Development virtual machines on OS X using VMWare and Ubuntu

Update 21st Febuary, 2010: If you get this error when installing VMWare Tools: mount: unknown filesystem type 'iso9660', it’s because you’re using Ubuntu 9.10 in JeOS mode. The solution is to upgrade your kernel to the generic, rather than virtual, kernel.

Update 26th November, 2009: Mike West has pointed out that the Linux headers required differ from version to version, so I’ve updated the instructions to reflect the header install required. Also, the JeOS version of Ubuntu 8.04 LTS is available at a different URL than the normal distribution.

Update 29th June, 2009: VMWare Fusion 2.0.5 has been released, which contains an updated VMWare Tools package for Ubuntu 9.04. This removes the requirement to edit the source files as outlined here.

Update 21st June, 2009: it’s been pointed out that Ubuntu 8.10, which this article was originally written for, has been replaced by the 8.04 LTS release. If you want a stable version of Ubuntu, rather than the bleeding edge, you’re best off using this release.

I’ve been using Linux as my primary development environment for a number of months now, despite being an abject Mac fanboy. Why?

• I often want to try new software (Python 2.6, Nginx, mod_wsgi, or CouchDB, for example), and neither MacPorts nor Fink are really appropriate to my needs, as it’s too easy to break your system using it (and to give you an idea of how invasive MacPorts is, just take a look at the uninstallation instructions!);
• OS X, being both a consumer desktop OS and a BSD–derived UNIX, is nothing like any of the commodity hosting available (usually Debian–derived systems like Ubuntu for me), so releasing stuff requires more testing (both locally and then in a “production–like” environment) before I can be sure it’ll work; but mostly
• The penalty for screwing up is too high. I live in fear of trashing the default Apache or Python on my Mac and not being able to recover from it.

I will continue to use my Mac as my main machine, because I love OmniFocus, and Safari, and VMWare Fusion, and all the other applications on my Mac. I love the toolset available to me, and value the user experience.

So what’s the solution? Virtualisation, of course. Thanks to VMWare’s shared folders feature, we can mirror our local folders into the VM, making it super–easy to run our code in Linux (and it also means we don’t have to use Samba).

Following are some (hopefully) bulletproof instructions for setting up JeOS (a virtual–machine friendly version of Ubuntu Server, as pointed out to me by Danny Amey) in VMWare in such a way as to make it an ideal development environment.

1. Firstly, get the latest Ubuntu Server from the download page (via BitTorrent or any other convenient method). Note that JeOS is listed as a feature of the distribution: Ubuntu Server Edition JeOS (pronounced “Juice”) is an efficient variant of our server operating system, configured specifically for virtual appliances;
2. Then start VMWare Fusion up, and create a new Virtual Machine. I recommend 256MB RAM and HDD of about 10GiB. Name it whatever you like (let’s say “JeOS” for this example);
3. When asked for a CD–ROM, point VMWare at the location of the ISO you just downloaded;
4. You’ll want to turn printers off, as JeOS doesn’t seem to deal very well with printers—probably because it’s quite minimal and doesn’t ship with printer capability;
5. Follow the other steps as normal.

You now have a fresh virtual machine. Polish the settings (change HDD size, for example), and start it up.

1. Initially you’ll see a language screen. Choose the appropriate language (“English” for me, obviously);
2. After choosing the language, you’re presented with an install menu. At this point, we need to differ from the usual Ubuntu install process and change our install mode. Hit “F4” (or “fn + F4”) to open the modes menu, and select “Install a minimal virtual machine”;
3. Follow the rest of the install process as usual (name your user, name your server etc.).

It’s worth nothing that at this point I don’t normally install any of the pre–configured server options—I just leave it as bare bones as possible.

Now the machine will start up, so we can install VMWare tools. This gives us the benefit of Shared Folders, which let us treat the machine as a local folder, so we can use TextMate or other local tools to edit files within the system. This is the real key to making the development VM usable.

After the machine starts up in a VMWare window, log in and type ifconfig. Take note of the IP address in the eth0 section alongside inet addr:. This is the IP address of your virtual machine (if you’re using NAT—if you’re using bridged it’ll be a non-local IP).

Since the terminal provided by basic Linux is a bit sparse on features, install SSH with sudo apt-get install ssh. When this completes, log out and open Terminal.app.

SSH into the system with ssh [username]@[ip address]. You’ll need to accept the dialogue about the MAC address fingerprint.

Now we need to update the system a bit. Run:

1. sudo apt-get update (this updates the package sources);
2. sudo apt-get upgrade (this updates the installed packages); and
3. sudo apt-get dist-upgrade (only if there’s a newer version of Ubuntu floating around—probably not at time of writing)—if you do this, see below.

Note: if you ran dist-upgrade above, you should restart your virtual machine now with: sudo shutdown -r now. Otherwise you may get clashes with kernel versions in the next step (thanks Mike Davies).

Now we can get on with installing VMWare tools.

1. sudo apt-get install linux-headers-$(uname -r) build-essential (installs some software required to build the components of VMWare tools);
2. In Fusion, install VMWare tools with the “Virtual Machine > Install VMWare Tools” menu dialogue. This causes a fake CD–ROM to be inserted into the machine;
3. Back in SSH, mount the CD with sudo mount /cdrom;
4. Create a new directory to compile source code: mkdir -p ~/src;
5. Enter the CD: cd /cdrom/
6. Copy the source code of VMWare across: cp VMwareTools-[HIT TAB TO COMPLETE] ~/src;
7. cd ~/src;
8. tar zxfv VMwareTools-[HIT TAB TO COMPLETE];
9. cd vmware-tools-distrib;
10. Now we can install VMWare Tools: sudo ./vmware-install.pl [--default].

At this point you can basically accept all the default dialogue options until it completes. Edit: as Mike West has rightly pointed out, passing the --default flag to the install command automatically accepts all defaults for you.

Now you’ve installed VMWare Tools, so it’s time to mount the remote folder:

1. In the VMWare Library, edit the Settings of your currently running virtual machine;
2. Select the “Sharing” settings option;
3. Add a new folder (in mine my code is stored in ~/Projects) and share it with Read/Write permissions;
4. Close the menu dialogue to save your choice.

This folder is now available to the VM at /mnt/hgfs/[NAME OF FOLDER], except there’s one issue: because of the way the mount works, the VM can’t write back to the shared folder (which is a problem if you’re compiling anything, for example). Here’s how you enable read–write for real:

1. Work out your user and main group ID by typing: id (this will usually be 1000 and 1000);
2. Edit your sharing settings with: sudo vi /etc/fstab;
3. Edit the line starting with: .host:/ /mnt/hgf to look like: .host:/ /mnt/hgfs vmhgfs defaults,ttl=5,uid=1000,gid=1000 0, where uid=1000 and gid=1000 are the actual user and group IDs we discovered back in the first step.

Now your VM can write back to the shared folder, and all is well.

Bonus topics

Link your shared folder into your home directory

If you want the shared folder reflected in your user home directory, just use a symlink:

ln -s /mnt/hgfs/Projects ~/projects

Run the VM in headless mode

So you don’t need to start VMWare each time, you can run the virtual machines as a separate process outside of VMWare Fusion. This is called “headless mode”. From a local terminal on your Mac, type:

defaults write com.vmware.fusion fluxCapacitor -bool YES

and restart VMWare. When you start your machine, choose “View > Enter Headless”. The screen will disappear for that machine, and it’ll keep running. Now you can close VMWare with a running VM and not have the machine stop. Use this script I wrote to start machines via the command line (so you can even bypass VMWare Fusion altogether).

Final notes

At this point it’s wise to use the snapshot feature of VMWare Fusion to save the state of your machine, in case you completely trash it and want to restore from scratch. This is one of the biggest wins of virtualisation in my opinion—making it easy and non–dangerous to experiment.

Also, thanks to Danny for helping me out a bit and Tim Huegdon for being my guinea pig.

Update: A note on Symfony

A few people have written and asked me about how to get Symfony running under this set up. Symfony has slightly peculiar requirements in that it needs to have a PHP/Apache writable directory in which it can dump the compiled YML files it uses for configuration. The solution to this is to patch /etc/apache2/apache2.conf as follows:

User [your username here] Group [your groupname here (usually the same as the username)]

and then reboot Apache.

This configuration means that Symfony runs with the same permissions as your primary user, which gives it full control over your project directories, and also inherits the permissions granted by HGFS. Needless to say: never do this on a public web server.