I’m sure you all have had a scenario where something is broken, but you figure out some janky way to get things working. Instead of spending the extra 15 minutes it would take to do the damn thing properly, you continue to live in this pseudo-functioning world. The problem is infinitely worse when it’s just “your dirty little secret”. Nobody else cares or even uses the functionality so why bother?
Well, I’ve been that way with MAMP for a long time now. For those that don’t know, MAMP (Mac, Apache, MySql, PHP) is a one click solution for setting up a web development environment on your Mac. I don’t remember when I started using it, but I know it’s been a lot longer than I should be comfortable admitting.
For some reason I’ve yet to look deeper into why MAMP creates its MYSQL socket inside of its own directory path, /Applications/MAMP/tmp/mysql/mysql.sock. Normally this isn’t a big deal, but if you want anything else on your box to interact with MySQL, chances are it will be looking for MySQL’s socket file in the standard /tmp/mysql.sock location. The error you’ll typically see is (2002, “Can’t connect to local MySQL server through socket ‘/tmp/mysql.sock’ (2)”).
That’s not a major problem though. You can just create a symbolic link in the /tmp directory by starting MySQL via MAMP and then executing sudo ln -s /Applications/MAMP/tmp/mysql/mysql.sock /tmp/mysql.sock. For YEARS I’ve been doing this after every reboot. More accurately, I’ve been launching MAMP, launching whatever app I’m working on, getting the error message and THEN executing this command. Today I finally got fed up with it and decided to actually figure out how to get this to happen automatically. It took me all of 10 minutes and I’m frustrated that I hadn’t broke down and done this task sooner.
In the MAMP application folder /Applications/MAMP/bin there are two files, startMysql.sh and stopMysql.sh. The scripts are just executing the start and stop commands for the MySQL daemon. Those commands have a parameter called –socket. Simply change the path for that parameter from /Applications/MAMP/tmp/mysql/mysql.sock to /tmp/mysql.sock. Remember to do this in both files and voila. You’ll never have to create a symbolic link for MySQL again.
I’m hoping that most of you aren’t as lazy as I am and solved this problem a long time ago. But in case you haven’t, you’re welcome.
When I hear tech media pundits talk about the tablet race I laugh hysterically. It’s this sort of involuntary reaction, the kind I’m sure Apple executives have when rumors of the next product launch circulate the web. The idea that there are competitors to the iPad is severely misplaced and really shouldn’t be a secret to anyone. I’m not saying that there no other quality tablets out there, I’m saying that people don’t want tablets, they want iPads. The fallacy that there is a large market for Android tablet users stems from market share numbers that show a lead in Android sales over iPhone sales. But success in the phone market means absolutely nothing in the tablet game.
The problem with market share as a measure of success for mobile platforms is that it doesn’t take into account the quality of the consumer. Instead market share reports a simple percentage and assumes that all consumers are equal. Anyone that has been even a casual observer of Apple’s business practice knows that’s not the case. Despite Apple holding a dismal percentage of the PC market share pie, their revenues have far exceeded that of any other computer manufacturer. Apple consumers are worth more than the average PC user. I’m not going to debate motives behind it, but Apple consumers have shown a clear willingness to pay a premium for what they perceive is a quality good. In comparison, the average Android consumer is looking for a cheap alternative smartphone, but not specifically an Android phone.
To further dig this point in I’ll use an example. Think of an election. You may vote for the President, but you simply picked the Water Reclamation Clerk based on party loyalty. The Presidential candidate might get a donation from you, but the reclamation clerk probably won’t get a dime. Something similar is occurring in the mobile OS market. There is no clear OS choice, it’s just the OS that came on the phone when you purchased it. There are more iOS fans than there are Android fans. Looking at market share, this seems a bit counter-intuitive, but there is a difference between choosing an OS and being given an OS. I challenge you to go to your local cell phone retailer and find a non-iOS phone that isn’t running Android. The market is inundated with Android devices. There are plenty of consumers that love the endless configuration choices. But average consumers are making their choice of phone based on handset design, not around the Android platform. The consensus of course is “so what, they’ve got the phone!” but that logic doesn’t hold up for developers, which serve as the engine for any mobile platform.
Developers need engagement from their audience. They need people who are looking for apps and are willing to pay for them. Once you place the world in that context, the number of handsets becomes a moot point. Soccer moms who don’t want the iPhone simply want to check e-mail, weather and Facebook. If the purchasing experience is smooth enough, you might be able to talk them into downloading a game or two at a fair price. But usually that’s the extent of their experience in the paid app realm. Apple on the other hand is world famous for their ability to engage their customers and build a type of loyalty like no other company in history. Not only are Apple fans loyal, but they pay a premium for the privilege. Android’s fan base isn’t anywhere close in terms of brand loyalty. Most of the diehard Android fans I talk to are also Open Source junkies, which doesn’t give them a huge track record of paying for software. The numbers look even more dismal when you look at revenue generated by each app store, with Apple’s App store generating four times as much revenue as the Android Marketplace.
So far all we’ve talked about are mobile phones, but where does this leave our tablet war? It leaves it as a lop-sided victory. Like a classic pump and dump scheme, the numbers paint a picture of a market that doesn’t exist. Consumers don’t want Android tablets, because most of them don’t want a tablet at all. They bought Android phones simply because they wanted a smart phone, but for some reason didn’t want an iPhone. (Some may have enjoyed Windows Phone 7 had it actually been pushed on them) The iPhone however was crafted around the idea of a specific experience. When people are looking at the iPad, they’re looking for that same experience, but on a larger device. It just so happens that larger device resembles a tablet. Android users on the other hand are just looking to make phone calls, check Facebook and browse the web occasionally. The Android experience doesn’t compel the user to move it to a larger screen or device. The idea makes even more sense when you look at the number of Android users that own iPads!
While the phone war rages on, the tablet war should really be relabeled as a skirmish. The Android experience, number of handset options and openness make it a strong competitor to Apple’s iPhone. But in the tablet market, it’s simply not there. I know there are millions of things the Android tablets can do that the iPad will never allow you to do. You maybe reading this and creating a laundry list of all the awesome things you can do on your tablet. But if you stop and listen for just a second, you may hear the rest of the world whisper.
Sometimes in an environment you get one of those pesky error messages that only shows up occasionally. Recently for me, it’s been the System.OutofMemory exception. The error was being thrown by an IIS Website the server was hosting. My troubleshooting was specific to an IIS Application pool, but hopefully this post will give you some food for thought if you’re experiencing a similar situation.
The first thing was to do the obvious and check to make sure we weren’t actually out of memory. These days there are about 1000 different warning signs letting you know the server is close to being out of memory. Not to mention you’ll probably see performance issues across the board as opposed to a single application. So if you’ve got enough memory, why would you get an OutOfMemory exception?
Well first, let’s back up a bit and talk about the virtual memory manager of Windows. If you’ve never done any programming or operating system work, you may not know this. All processes are given their own virtual address space. What this means is that the executing process is ignorant to the actual memory available to the system. As far as the process is concerned, it has a whopping 4 gigabytes of memory. The upper 2 gigabytes of the stack are “reserved” by the operating system, while the lower 2 gigabytes are used by the process. These values can change based on boot switches, 64bit Windows or other settings, but we’ll use these numbers for our example.
The process is a bit ignorant to things like paging. The process simply requests memory and treats it as if all memory is physical ram, leaving those cumbersome paging and mapping details to the operating system. The other important thing to remember is that memory is allocated in contiguous chunks. That means if your program is requesting 5 megabytes of memory, not only must there be 5 megabytes of memory available, but there must be 5 megabytes of adjacent memory locations available. So even if you’ve got 1 gigabyte of memory available, if it’s totally fragmented into small chunks, you may not have enough contiguous memory available, which in turn will cause the System.OutofMemory exception.
How do you defragment a process’s virtual memory map? I’m not sure that you do. If you restart the process, a new map is created, but this isn’t always feasible. In reality, most of your applications won’t have this problem, but long running processes, for example IIS, might run into this situation. If you dump all your websites into your DefaultApp pool, it’s quite possible that all those applications are fragmenting the address space to the point where you can’t load large DLLs, such as the AjaxControlToolkit.dll file. (The culprit in my exercise)
Sometimes it’s difficult to get an idea of what the process virtual map looks like. I’ve discovered a helpful tool called Address Space Monitor. The application will let you know the largest amount of contiguous memory available to your process along with some other helpful bits of information.
I combined this, with another helpful tool called Procmon. With Procmon you can monitor all types of activities, including attempted file loads into memory. I simply ran Procmon with a filter for dll files that were being loaded by the process I was having trouble with to get an idea of what was going on when the OutofMemroy exception was being thrown. After seeing a 5.6 megabyte file being loaded by a process with 2 megabytes of contiguous address space available, it was pretty clear what my problem was.
This write-up is somewhat specific to my problem, but I hope it at least has given you some food for thought for troubleshooting your own issue.
I’ve been working on a web site using C# and the Microsoft MVC framework. So far I’m liking what I’m seeing, but I’m still digging into some of the actual implementation details. There are some things that happen with MVC that look like standard stuff you’ve seen before, but actually behave quite differently.
One of the hurdles I (and several other people) was facing was that I had a controller action, with a parameter, but the parameter was being passed into the controller with a value of NULL. Phil Haack put together a nice Routing Debugger utility to see how your routes were being interpreted by the MVC Framework. Everything looked good, I could see that the ID value was being properly populated, but my Controller action parameter still had a null value.
After doing some research I discovered that even though your Controller Action looks like a regular method signature, the way parameters are passed is actually quite different. The parameters are actually key/value pairs that exist in the HTTPRequest Object. If you’ve defined the following controller method signature
public ActionResult SaveADDomain(string Name, string Desc, string Id)
The MVC Framework will look for keys named “Name”, “Desc” and “Id” and populate the corresponding variables with the key’s value. If the key doesn’t exist, null is instead populated in the variable. I believe the collection for the key/value pairs is stored in the HTTPRequest Params property, but don’t quote me on that just yet. So how do we define these key value pairs? In the Global.asax file of course!
The Global.asax file is where the Routes for your application are registered. The default file should look something like the following:
public static void RegisterRoutes(RouteCollection routes)
{
routes.IgnoreRoute("{resource}.axd/{*pathInfo}");
routes.MapRoute(
"Default", // Route name
"{controller}/{action}/{id}", // URL with parameters
new { controller = "Home", action = "Index", id = UrlParameter.Optional } // Parameter defaults
);
}
You’ll notice in the routes.MapRoute call, there is a URL path definition of {controller}/{action}/{id}. The {id} is the beginning of the parameters calls. So if you wanted to also have a parameter like “sortorder” or something like that, you could modify the route pattern to look like {controller}/{action}/{id}/{sortorder}. Or you could use it to rename the {id} to something else.
As most of you know, FTP is the devil in terms of security. It sends clear text passwords over the wire, with most users being actual system users as well. It’s telnet’s equally evil little brother. In the *nix world, it’s not that big of a deal, because you have free tools that come shipped with the OS. But in the Windows world (at least in pre 2003) there is no secure solution out of the box. With budget crunches being what they are, most managers aren’t willing to switch to an SFTP solution which can run $1000+ per license.
For one of our smaller FTP implementations I decided to try and get Cygwin running on Windows, with an OpenSSH implementation. It’s not the greatest, but it is free, cheap and will execute as a service as opposed to some big control panel that needs a user logged in. (A common symptom of free Windows SFTP solutions) I also wanted to make sure that a user couldn’t get an interactive shell once they logged in, essentially giving them access to the entire machine.
Cygwin Installation
The first thing I had to do was go through the Cygwin installation. When you go through the setup, there will be an additional section that asks you for additional packages to install. Be sure that you install the OpenSSH packages, as those will provide your SSH and SFTP server binaries.
Once Cygwin is installed, we’ll need to add a new user to the system to handle the SSHD process. We don’t want to execute this as Local System, due to some security issues and hurdles. First we’ll create the user account. Drop to a command line (I’m a command line guy) and execute
net user sshd password /add /fullname:”Cygwin SSH Daemon”
Replaced “sshd” with whatever username you want to use and “password” with whatever password you want. You probably also want to make sure the password doesn’t expire, but I’ll let you make that call. Next you’ll need to add this user to the Administrators group. I still haven’t been able to find out the other security needs, but I know that my implementation didn’t work until I added the user to the Admin group. You shouldn’t worry though, we’ll be taking other precautions to ensure that the user is sufficiently limited.
net localgroup Administrator sshd /add
Next we’ll need to assign the SSHD account some appropriate rights, as well as take away some rights. I’m using the ntrights command which can be found in the Windows 2003 Resource Kit.
You’ll notice that we’ve chosen to deny the user interactive logon capabilities, meaning he is just a service account at this point.
Now we’ll want to setup one of the System Variables. Right click on My Computer and go to Properties. Click the “Advanced” Tab and then click “Environment Variables” on the bottom pane.
In the section marked “System Variables” click New and add a variable named “CYGWIN” and give it a value of “ntsec tty”. Click OK, and then OK again to save your changes.
Now you’ll want to launch your Cygwin shell and run the SSH configuration script.
ssh-host-config
You’ll want to answer Yes to all questions except the question “This script plans to use cyg_server, Do you want to use a different name?” You should enter the name of the user account we created earlier. (In our exampled it was “sshd”)
Next you’ll need to run
cyglsa-config
Warning: This will require a reboot!! This command enables Cygwin to use the Local Security Authority to help authenticate users. Once the command is complete, be sure to reboot your system. If your service fails to start after the reboot check the services failure section.
Next we need to create the group and passwd file. These files are necessary for Cygwin users to authenticate. They are the Unix counterpart to the SAM database.
You should be able to now accept incoming SSH connections. Make sure that all necessary firewall ports are open. Using Putty or your other favorite SSH client, connect to your machines IP with a username and password. Be sure that you’ve setup another user besides the SSHD user.
Preventing a User from Logging In
If you want to prevent a user from completely logging in, you can modify the /etc/passwd file in Cygwin. Chances are you don’t have VI installed, so your best bet is to browse to your Cygwin installation directory in explorer and open the passwd file in the etc folder. The very last portion of each line represents what shell should be launched for the user. The default is “/bin/bash”. If there’s a user you don’t want accessing the system, change that to “/bin/false”. Voila. The user can’t login via SFTP or SSH. However, you will have to change this every time you regenerate the password file. (Which is why this solution is only good for small implementations)
Only allowing SFTP for a User
Edit the /etc/passwd file on the system by browsing to it either in explorer or by launching VI in a Cywin window. Change the default shell for the user (the last portion of the user line) from “/bin/bash” to “/usr/sbin/sftp-server”. This will make SFTP the users default shell.
Setting up a CHROOT for a User
There is no true CHROOT concept in Cygwin, because it is implemented using Windows DLLs. If you need to do CHROOT, I’ve heard that scponly is a solution that works and can be compiled in Cygwin. I’ll be trying this soon, so details will follow.
Closing Remarks
Let’s be clear. Cygwin is not the ideal SFTP solution for Windows. Sometimes however, circumstances will warrant its use. The above solution without the CHROOT has one major flaw. The user has complete access to the disk. While they can’t necessarily execute code directly, there are things they could do to make life bad for you. You need to be diligent in your security setup if you intend to do this. Hopefully my CHROOT experience will go well and I can share the details with you at another time.
I’m a relative newcomer to Python, but drew interest in it when looking for a framework to develop a project I’ve been working on. I was introduced to DJango and liked the possibilities, but figured I should do a bit more with Python before diving in. So I started working on a Python script at the office to help me parse some JMeter log results. I’m trying to gather some statistical data on when errors occur during a load test and the JMeter reports don’t seem to have quite what I’m looking for.
The first problem I ran into was that there are multiple threads executing the load test. As a result, the log file is not organized in a linear fashion. I’ve put all of the data elements of the test into objects, but now I need to sort the list based on the TimeStamp property of each object. Not to mention the list is roughly 500,000 elements long, so I need it to be somewhat efficient. (Although performance isn’t a HUGE deal in this project) I opted to go with a min-heap data structure.
After doing some digging I came across the heapq library in Python. The problem was that there was no mechanism for sorting based on the property of an object. It basically would take a list of values and sort them. I really wasn’t in the mood to write my own library for this project, but I figured it was worth spending some time with the problem because it would most certainly come up again in my days with Python.
Ultimately I discovered that the heapq library would also accept a tuple as a parameter, sorting based on the first element in the tuple. So the problem came down to something as simple as:
Now when I iterate though my list, I’ll be returned a tuple. I just need to grab the second element out of the tuple for my actual object and do my worse with it. It was pretty simple.
There was another option however. We could overload the comparison operators on our objects.
__eq__
__ne__
__ge__
__gt__
__le__
__lt__
By overloading these comparison operators, I can define how the comparison would take place.
Now I’ll need to do this for all of the comparison operators, but once I’m done I’m golden. Some people suggested doing this using the Decorator Pattern but for my needs that might be overkill. My data set won’t be that dynamic and I know my needs at compile time. (Or design time as it were)
Also there was some chatter out there about using overloading __cmp__ , which would work, but is deprecated as of Python 3.0. Python 3.0 isn’t in wide use yet and future proofing in this case is easy enough. But if you’re really lazy and don’t have a need for 3.0 compatibility, there’s that route too.
I was at my monthly Chicago Comic Book Meetup and we started an interesting conversation about dysfunctional families in comics. The discussion finally made its way to the relationship between Alfred and Bruce Wayne. While their relationship obviously goes beyond master and butler, there still exists a certain rigidness about it. Alfred still refers to him as Master Bruce or Master Wayne. Someone at the meetup made the comment that despite the closeness of their relationship, Alfred would always adhere to his position in society in relation to Bruce Wayne.
But hasn’t their relationship become more like family? (Even though it’s not reflected in their addresses to one another.) So the idea of Alfred getting a weekly pay check from the Wayne Foundation for his service as a butler just seems odd to me. I’m sure it has never come up in conversations at DC, but I can’t imagine Alfred getting a regular direct deposit pay check. Sure he’s compensated, but I would imagine that just about anything and everything he does, goes on the black Visa card. Can you imagine Alfred “saving up” for a new flat screen television. I’m not sure what butlers make, but my guess is that it’s not baller status. But at the same time, I couldn’t imagine Bruce letting Alfred want for anything.
Here’s another angle. If Alfred gets paid like any other commoner, what happens if Bruce goes broke? Does Alfred move out and get another job? Does he work a register during the day and help Batman solve crimes at night? I think not. They’d be broke, in squalor together, trying to figure a way out of things. I’ve had some terrific bosses, but I’d never stick around for no pay if our relationship hadn’t somehow transcended employer/employee.
Here are a few things that regular employees would have to go through, that I can’t imagine Alfred ever dealing with.
Pitching into his 401k plan
Going through a performance appraisal
Filing an expense report
Getting a Purchase Order approved
Choosing between an HMO and a PPO
Missing the family reunion because gas rates are so high
The list is silly, but at the same time incredibly serious. When Alfred starts to collect a paycheck, it brings into focus the everyday problems that must come along with that. It also makes us separate what Alfred has and what Bruce Wayne has, which for years has been one in the same to me.
Do you think Alfred Pennyworth collects a pay check? Why or why not?
It’s rare that a game makes me downright emotional. The game will have a hard time living up to this trailer, but I already want to kick some zombie ass! For Jasmine!!!! Or whatever the girl’s name is….
I was at my monthly 
