Apache can do “web-scale”. Sometimes.

Benchmark results can very wildly, depending on everything from the color of Ethernet cables used (not really) to the kernel revision, and the major/minor point release and patch set of the software tested. Nginx 1.2.1 may have very different performance characteristics as compared to the Nginx 1.1.19-ubuntu-1 package that comes with Ubuntu 12.04. Due to the unpredicable performance of EC2 instances, all of the benchmarks were run multiple times, and the fastest test results were selected for comparison.

The performance of web server software will be very different for various web traffic workloads. For example, Nginx is a much better choice for internet radio streaming compared to Apache. You should perform benchmarks with your application on various web servers to understand the unique performance characteristics of your site.

The goal of this benchmark was to test raw throughput and average response time under load of various web servers. This benchmark did not test websockets, long-polling, large file transfers, or streaming. Each of the web servers were tuned and configured to the simple task of returning the string: “Hello World”. All of the benchmarks were performed on c1.medium EC2 instances, running Ubuntu 12.04 64bit. The benchmarks were run using JMeter 2.7, and visualizations were produced by Loadosophia.org.

The Contenders:

Maximum throughput test

The goal of this benchmark was to test the average response times under various amounts of load. Ideally, a good web server will respond quickly and the standard deviation of the requests will be minimal. We want low latency and consistent response times.

This test was run for about 8 minutes, started out with 100 jmeter threads un-capped with no timer, then moved upto 500, 1000, and finally 2000. Between each thread group run, there was a small cool-down period.

Apache 2 – mpm_worker

mpm_worker was able to sustain about 3.1k Requests-per-second, with an average response time of 132ms. Not much to say here, this test was performed as a simple baseline.

Apache 2 – mpm_event

mpm_event was slightly slower than mpm_worker, coming in at 3k Requests-per-second, with an average response time of 136ms. However in the 500-thread test, the standard deviation of the test results was lower compared to mpm_worker.

These results were surprising to me, since mpm_event is generally considered to be faster than mpm_worker.

Nginx

Nginx was able to outperform Apache in these tests, but by a very small margin. The max requests-per-second hit 3.3k, and average response time was 117ms. However, the standard deviation was much higher than Apache.

Nginx is slightly faster than Apache in this test, but Nginx does not show nearly as consistent performance.

Brubeck b/w Mongrel2

The performance of Mongrel2 was the fastest of all servers, but it quickly it became overloaded. At 100 threads Mongrel2 was able to sustain the highest RPS of the test, 6k RPS, at 28ms latency! However, as soon as Mongrel2 hit a tipping point, it quickly fell over.

With the default configuration, running as non-root, Mongrel2 hits a limit of file descriptors. I increased the file descriptors to 65535, and found that the max requests-per-second that Mongrel2 was able to serve got cut in half from 6k down to 3k.

 
Zed Shaw wrote an article about the polling mechanism in Mongrel2, and Willy Tarreau (author of haproxy, and epoll advocate) explains why adding more file descriptors actually slows down Mongrel2 in this article.

Even in the test with the increased file descriptors, Mongrel2 had really good performance up to through 500 threads, but quickly fell over when 1000 jmeter threads came stampeding through. I was told that this performance limitation was probably related to internal ZeroMQ configs that Mongrel2 does not currently expose for tuning. See also 6k RPS Data here: https://loadosophia.org/examples/11579/

Brubeck WSGI

Until recently, Brubeck required running a Mongrel2 instance in order to serve web traffic. But to ease the Brubeck development workflow, they recently added WSGI support. This benchmark shows the WSGI interface is highly performant and stable (I encountered no errors up until the 2000 thread test at the end of the test).

My first test followed the “hello world” example on the front of brubeck.io. I shared the results of the test with the development team in #brubeck,. They asked me to rerun the tests using function-based views.

By switching from class-based to function-based views I was able to increase the performance of Brubeck by 30%.

In the function-based views test, Brubeck WSGI was able to maintain an average 2.6kRPS, and response times had the lowest standard deviation observed throughout the entire test at 100 and 500 threads.

My theory is Brubeck has the most consistent performance of all the servers tested because it’s the only server that uses green threads. Context switches are extremely slow in virtualized environments like EC2, much slower than on real hardware. By using python-managed threads instead of OS-managed threads like the other servers tested above, Brubeck is able to handle many more requests in a single scheduled clock cycle. For more information about gevent, and green threads, go watch Bob Hancock’s tutorial from PyCon 2012 about concurrency.

See also: Brubeck class-based views data: https://loadosophia.org/examples/11575/

Tornado

The design of Brubeck was inspired by the Tornado web server. However, where Brubeck differs is that Tornado prefers to use callbacks instead of gevent for concurrency. This model of programming can be confusing to implement, because you essentially have to program in reverse.

The performance of Tornado was not much faster than the traditional Apache/Nginx servers for this test, and was only able to sustain an average of 2k RPS, at 344ms. Tornado was not designed to handle this type of rapid ballistic traffic pattern, and it is much better suited for long-polling, or streaming data.

If you already use Virtualenv, great! I don’t have to tell you why it’s a good thing to use. But if you’re not using Virtualenv yet, give me two inches of screen space to explain why you should.

Virtualenv allows you to store projects in complete, self-contained folders. This means that when Django 1.4 came out last month, you can quickly try it out without breaking all of your Django 1.3 projects. You might think, “I only have one project!” This will change quickly. Using Virtualenv also allows you to git-clone other people’s projects, read their code, run their programs, and become inspired by their really smart ideas.

If you build an easy system to manage your Virtualenv projects, it’s actually easier to use Virtualenv, than not.

Now that I’ve convinced you to use Virtualenv to manage your python projects, I’ll explain my favorite way to work with Python and multiple Virtualenvs.

I primarily use:

1. OS X Lion (but Linux is a fine choice as well)
2. MacPorts (Yes, I use a BSD-style package manager on a BSD system. If you’re using some ruby-based SYSV style package manager, that’s probably fine as well.)
3. Python27 (from Ports)
4. virtualenv & virtualenv wrapper
5. Setup your virtualenv bash environment variables
6. Some clever pre/post hooks for virtualenv wrapper
7. Dropbox, to keep this all sync’d from my laptop/desktop

The Basics

Assuming that you’re using OS X Lion, with Macports, this next section will be applicable to you. If you’re not using those, I assume you have enough Google skills to install Python, Virtualenv and Virtualenvwrapper. Skip to the advanced section below for my pre/post hooks and bash magic.

1. OS X
Make sure all Apple updates are installed. As of writing this tutorial, the latest OS X is 10.7.3.
Apple recently split out the command-line compilers as a peripheral download, called “Command Line Tools for Xcode”, search around on https://developer.apple.com/downloads/ for it, download it, install it. (should be about 171mb.)

If you have already installed Xcode from the Apple Appstore, open Xcode, open the preferences, and check the downloads tab to install the latest version of the command line tools. The fact that Apple split out these tools is interesting, and I’ve written about what I think that it means for the future of OSx in another post.

2. MacPorts
Many people are starting to use Brew instead of MacPorts. I prefer MacPorts because it does not rely on as many OS libraries, thus is less susceptible to Apple’s bugs and secretive updates. However, the reason why I like MacPorts more than Brew is a topic for another article.

• Install MacPorts for Lion from the DMG disk image at (http://www.macports.org/install.php)

The MacPorts installer will modify your UNIX path setting in the .profile file in the base of your home directory. (This file is hidden from the finder’s default view, use the Terminal to see it) Rename .profile to .bash_profile, or if you already have a .bash_profile, edit this file to make sure that the Macports setup correctly installed the path statement:


# MacPorts Installer addition...
export PATH=/opt/local/bin:/opt/local/sbin:$PATH


• Configure Spotlight to exclude /opt/local from indexing
  system prefs -> spotlight -> privacy -> add /opt/local

3. Python 2.7

• open a new Terminal
• sudo port selfupdate
• sudo install python27
• sudo port select –set python python27

(ignore the errors about missing xcodebuild, since this is only needed if you’re going to be building OS X native apps with MacPorts. Consider installing Xcode if you think you might be doing this in the future.)

4. Pip/Virtualenv

• sudo easy_install pip
• sudo pip install virtualenv virtualenvwrapper

The Advanced stuff:

Now that you’ve installed some of the basics, let’s configure out UNIX environment for Virtualenv.

5. Bash environment variables for Virtualenv
Edit the .bash_profile folder in your home directory. (whenever you see ~/, that’s an automatic bash shortcut to refer to the location of your home directory)

vim ~/.bash_profile
Then add:

if [ -f ~/.bash_virtualenv ]; then
source ~/.bash_virtualenv
fi

This will first test to see if the file .bash_virtualenv exists, and if it does, it will ‘include’ this file every time we open a new terminal.

Now create the .bash_virtualenv file:

vim ~/.bash_virtualenv

This will setup several Bash environment variables that will be read by pip, virtualenv, and the virtualenv wrapper script. Some of the important variables set here are the pip download cache, so that pip doesn’t re-download packages, and the WORKON_HOME. Feel free to change the PY, and the VIRTUALENV_PROJECT… folders. I like to store git repositories in my Dropbox folder, and then push/pull to them in order to sync my work between multiple computers.

# for virtualenv
if [ -f /usr/local/bin/virtualenvwrapper.sh ]; then
PY=~/Documents/python/virtualenvs
export WORKON_HOME=$PY
export PIP_VIRTUALENV_BASE=$PY
export PIP_RESPECT_VIRTUALENV=true
export VIRTUALENVWRAPPER_VIRTUALENV_ARGS='--distribute'
export PIP_DOWNLOAD_CACHE=$HOME/.pip_download_cache
export VIRTUALENV_PROJECT_GIT_ROOT=${HOME}/Dropbox/research/python/git
source /usr/local/bin/virtualenvwrapper.sh
fi

To test this file, do:
source ~/.bash_virtualenv

And you should see something like:
virtualenvwrapper.user_scripts creating /Users/rterhaar/Documents/python/virtualenvs/initialize
virtualenvwrapper.user_scripts creating /Users/rterhaar/Documents/python/virtualenvs/premkvirtualenv
virtualenvwrapper.user_scripts creating /Users/rterhaar/Documents/python/virtualenvs/postmkvirtualenv
virtualenvwrapper.user_scripts creating /Users/rterhaar/Documents/python/virtualenvs/prermvirtualenv
virtualenvwrapper.user_scripts creating /Users/rterhaar/Documents/python/virtualenvs/postrmvirtualenv
virtualenvwrapper.user_scripts creating /Users/rterhaar/Documents/python/virtualenvs/predeactivate
virtualenvwrapper.user_scripts creating /Users/rterhaar/Documents/python/virtualenvs/postdeactivate
virtualenvwrapper.user_scripts creating /Users/rterhaar/Documents/python/virtualenvs/preactivate
virtualenvwrapper.user_scripts creating /Users/rterhaar/Documents/python/virtualenvs/postactivate
virtualenvwrapper.user_scripts creating /Users/rterhaar/Documents/python/virtualenvs/get_env_details
virtualenvwrapper.user_scripts creating /Users/rterhaar/Documents/python/virtualenvs/premkproject
virtualenvwrapper.user_scripts creating /Users/rterhaar/Documents/python/virtualenvs/postmkproject
virtualenvwrapper.user_scripts creating /Users/rterhaar/Documents/python/virtualenvs/prermproject
virtualenvwrapper.user_scripts creating /Users/rterhaar/Documents/python/virtualenvs/postrmproject

6. Edit those pre/post virtualenv wrapper hooks, add some magic

mkdir -p ~/Documents/python/virtualenvs
source ~/.bash_virtualenv (or if you get any error messages, just open a new terminal)
cd ~/Documents/python/virtualenvs

The four files that you should edit are:

• postactivate
• postdeactivate
• postmkvirtualenv
• premkvirtualenv

First, postactivate


# Global virtualenvwrapper postactivate, lives in $WORKON_HOME/postactivate
# from http://hmarr.com/2010/jan/19/making-virtualenv-play-nice-with-git/
PROJECT_DIR=”${WORKON_HOME}/$(basename $VIRTUAL_ENV)”
if [ -d $PROJECT_DIR ]; then
# If we aren’t already within the project dir, cd into it
if [[ ! `pwd` == "$PROJECT_DIR*" ]]; then
export PRE_VENV_ACTIVATE_DIR=`pwd`
cd “$PROJECT_DIR”
fi
fi
unset PROJECT_DIR

Now edit postdeactivate:

if [ $PRE_VENV_ACTIVATE_DIR ]; then
cd $PRE_VENV_ACTIVATE_DIR
unset PRE_VENV_ACTIVATE_DIR
fi

Then edit premkvirtualenv:


source ~/.bash_virtualenv

if [ -d $VIRTUALENV_PROJECT_GIT_ROOT ]; then

if [ -d ${VIRTUALENV_PROJECT_GIT_ROOT}/${1}.git ]; then
echo “Git root already created”
else
echo “Creating git root”
cd $VIRTUALENV_PROJECT_GIT_ROOT
git init –bare ${1}.git
fi

cd ${WORKON_HOME}/${1}
git init
git remote add origin file://${VIRTUALENV_PROJECT_GIT_ROOT}/${1}.git
git config branch.master.remote origin
git config branch.master.merge refs/heads/master
git pull

if [ -f .gitignore ]; then
echo “.gitignore already exists, skipping”
else
echo “bin/*
include/*
share/*
lib/*
man/*
.Python
.*.swp
*.py[co]” > .gitignore
git add .gitignore
git commit .gitignore -m “init”
git push origin master
fi
fi

Now finally postmkvirtualenv:

if [ -f requirements.txt ]; then
echo “requirements.txt already exists, skipping”
else
pip freeze > requirements.txt
git add requirements.txt
git commit requirements.txt -m “adding basic pip reqs file”
git push origin master
fi

After adding these pre/post hooks to these files, you will have a really powerful and easy to use Virtualenv system. Here’s some tips for working with your new virtualenv system:

To start a new project, make a new virtualenv using:
mkvirtualenv my_project

This will create several folders, and “activate” your terminal session for the new virtualenv directory. When you’re done working, close the terminal or type “deactivate”

To continue working on an existing virtualenv:

workon my_project
(Tab completion works, so if you type ‘workon’ then hit the tab key twice, you’ll get a list of existing projects)

Once you’ve activated a virtualenv, install software inside this virtualenv by typing:

(my_project)users-Mac:~ user$ pip install ipython
(notice! no sudo. We are using a special version of pip that will install ipython into the virtualenv, and not to your OS.)

That’s it! Hope you enjoy using Virtualenv to manage your Python projects. Please leave a comment if you have any problems, or suggestions.

The Problem
When you create an Amazon EC2 instance, you are allowed to throw a small amount of data (64k ?) into instance metadata. This metadata can then be read back by the instance by querying a special URL. This feature has been used for server-classification and configuration of systems.

The problem is that everyone was using this metadata space in a different way. Some people were just throwing a single trigger word like “database” into this metadata space, and some people were storing large procedural configuration scripts that would be downloaded and executed by the instance. There has been much fragmentation onto how to use this metadata space for server configuration.

Cloud-init as a solution
Let’s say that you want an Ubuntu machine to boot up, upgrade all available packages, install Puppet, then check-in to your PuppetMaster:

#cloud-config
apt_upgrade: true
packages:
- puppet
puppet:
conf:
agent:
server: "puppetmaster.example.org"


I look forward to software like cloud-init becoming common and universally supported by more cloud providers. The concept is simple, easy to work with, and powerful.

Read more about cloud-init at the Ubuntu Docs Page for cloudinit

The Evil Template-Based Workflow Example

1. Configure your base system, install some good software
2. Take a Snapshot (somehow… re-bundle an AMI? Hit the snapshot button in VMWare?)
3. Shutdown your base system
4. Spin up instances based on this snapshot
5. Attempt to manage the chaos as configurations drift
6. Spin up your base system, apply various updates, and take a new snapshot
7. Spin up new instances based on new snapshot
8. Chaos!!

You can see how without a dynamic configuration management system, the template-based workflow is impossible to manage. Outdated templates and manual system configuration result in complete infrastructure chaos.

Configuration Management Systems (use them!)

Carl realized that the smarter alternative to the template-based workflow is to use a configuration management system, but there’s so many options! Carl wasn’t sure which one to use, how to begin using any of them, or how all of the pieces fit together. Carl told me that one of the most confusing things in his research was to understand where the responsibilities of the various management system begin and end.

A Nebulous Onion

I drew the following diagram for Carl to help him understand how all of the current tools available fit together into a bigger picture of Cloud Management. This workflow example is bias to Cloud Providers with advanced APIs, and Linux/UNIX operating systems- but why would you ever use anything else?

1. Creation of Servers: Some event occurs, and servers are created. Maybe this is an auto-scaling request from your load balancer, or maybe John in Marketing says that your website is going to be mentioned on Oprah later this afternoon. (woo!)
2. Servers are created programmatically: A bit of configuration metadata is stored somewhere in an API that will be referenced in step 3. e.g., these 10 new LARGE servers should be WEB servers in the PRODUCTION cluster.
3. The OS boots: The instances boot, and cloud_init (or some custom script you spent all afternoon writing) reads the metadata that you stored back in step 2, and basic configuration begins. Puppet is installed, and your new instances prepare for their first connection to your Puppet master for configuration.
4. Your Systems are 100% Configured: A good configuration management system will leave your systems in a fully-configured state. No manual intervention should be required.
5. Connection to External Data: This could be the topic of a larger blog post, but the final step in configuration is to connect your instances to external data sources. This could mean mounting an NFS share, connecting an EBS volume, or joining a Gluster or Casandra ring. This is really upto your imagination, and your application’s needs.

Summary:

There are many tools for Wrangling the Cloud, and I’ve only described one example workflow from a very abstract level. There is no one-size-fits-all solution, however that doesn’t mean that you should reinvent the wheel. We’ve built a lot of systems like this for our clients, contact us for help with yours.

Luckily, Debian has packaged a new version of Postgres that will use the libreadline6 library if it has been installed instead of the inferior (but more open-licensed) libedit. Unfortunately, the new Postgres package is only currently available in Debian Backports so you’ll have to enable the backports repo first.

Once you’ve installed Postgres8.4 > r114 from backports, all you have to do is install readline:
sudo apt-get update && sudo apt-get install libreadline6

In order to move the swap file from the OS drive to the data drive:

zfs create -V 4G tank/swap
zfs set dedup=off tank/swap
swap -a /dev/zvol/dsk/tank/swap
swap -d /dev/zvol/dsk/rpool/swap


And then to make this change persistent across reboots, also edit the swap path in your /etc/vfstab.

/dev/zvol/dsk/tank/swap - - swap - no -


Using a shared swap device is fine for development and small office servers, but you should consider using dedicated and mirrored swap if this server will be under any serious load.

First find two physical disks for your new mirrored swap

root@gyro:~# format
Searching for disks...done

AVAILABLE DISK SELECTIONS:
0. c7t0d0
/pci@0,0/pci8086,3472@1f,2/disk@0,0
1. c7t1d0
/pci@0,0/pci8086,3472@1f,2/disk@1,0
2. c7t2d0
/pci@0,0/pci8086,3472@1f,2/disk@2,0
3. c7t3d0
/pci@0,0/pci8086,3472@1f,2/disk@3,0
4. c9t0d0
/pci@0,0/pci8086,25f7@2/pci8086,350c@0,3/pci11ab,11ab@2/disk@0,0
5. c9t1d0
/pci@0,0/pci8086,25f7@2/pci8086,350c@0,3/pci11ab,11ab@2/disk@1,0
6. c9t2d0
/pci@0,0/pci8086,25f7@2/pci8086,350c@0,3/pci11ab,11ab@2/disk@2,0
7. c9t3d0
/pci@0,0/pci8086,25f7@2/pci8086,350c@0,3/pci11ab,11ab@2/disk@3,0


Let’s pick the last two drives for our example

zpool create SWAP_ZPOOL mirror c9t2d0 c9t3d0
zfs set mountpoint=legacy SWAP_ZPOOL

Setting mountpoint=legacy will keep Solaris from mounting this drive automatically upon import. From here, follow the steps above to add this device to configure this device as swap and add to vfstab.

So how can Puppet help us with this task?

#!/usr/bin/puppet

package{$package_list:
ensure => 'latest',
provider => 'apt',
}

Then make sure to chown this file:

$ chmod +x packages.pp
$ ./packages.pp

Now, when you run this file, Puppet will use the built in ‘apt’ package provider to install the list of packages defined at the top of the file.

It’s actually much easier to write a complex script using a high-level language such as Puppet, rather than Bash, despite this example that requires more code to accomplish the same task using Puppet than Bash.

Using Puppet instead of Bash for basic system scripting gives you many advantages such as better debugging, better error handling, standard formatting, better readability, and is generally much easier to perform complicated tasks than Bash or perl.

It also is a great introduction if you’re just getting your feet wet in the Puppet world. Good luck, and check back soon for more Puppet tutorials.

Problem symptoms included Tomcat freezing while initializing, then after 10mns timing out with error messages saying that it was unable to contact the WSDL URL. Before we could connect to that URL, we had to tell Java’s SSL Libraries to allow unsafe renegotiation.

How to fix this:

Short Answer:
In /etc/default/tomcat6 (Debian/Ubuntu), add a new line to your ‘CATALINA_OPTS=’ section.

-Dsun.security.ssl.allowUnsafeRenegotiation=true


noatime
A very typical UNIX/Linux tweak is to turn off ‘atime’ on your system drive. As part of the POSIX, your OS must write ‘last accessed’ time to each time upon read. On SSD disks, the default ‘atime’ feature can cause pauses and high latency due to excessive small writes. Turning off ‘atime’ may affect HD clean-up tools that check for infrequently accessed files, and may also affect a few UNIX applications, such as older versions of Mutt.

In UNIX and Linux, the noatime setting is easily configured via /etc/fstab, but in OS X, a launchd file must be configured. The idea and implementation for this is from: http://blogs.nullvision.com/?p=275

Open your terminal (/Applications/Utilities/Terminal) and type:

sudo nano /Library/LaunchDaemons/com.nullvision.noatime.plist

And inside that new file, paste the following:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN"
        "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
    <dict>
        <key>Label</key>
        <string>com.nullvision.noatime</string>
        <key>ProgramArguments</key>
        <array>
            <string>mount</string>
            <string>-vuwo</string>
            <string>noatime</string>
            <string>/</string>
        </array>
        <key>RunAtLoad</key>
        <true/>
    </dict>
</plist>

Hold CTRL and hit X to exit, make sure to save when asked. Then reboot your computer, and open the terminal again. We will now confirm that your system has ‘noatime’ enabled.

mount | grep ” / “

You should see something simular to:

/dev/disk0s2 on / (hfs, local, journaled, noatime)

Success! your operating system drive has been mounted with ‘noatime’.

Move your Swap
If you have a system with both an SSD as well as a standard magnetic HD, you may consider moving your swap file off the SSD and onto the HD. There are various arguments for making this change- by moving your swap to the slower/larger storage device, paging operations to disk will obviously be much slower, however if your system has ample RAM, the swap file should be mostly untouched anyhow. Your upgrade priorities should be reevaluated if you have an SSD in a system that has less than 4GB of RAM.

To move your swap to a new location, we will modify a built-in system launchd file, and create a wrapper script. Editing base files installed by the OS is scary, and dangerous, so make sure you have a backup before attempting this step. This step also requires that the Xcode developer tools have been installed, since the plist editor is not included in the base OS X install. (implementation modified from http://hintsforums.macworld.com/showthread.php?p=452409 )

First, we’re going to create the wrapper script:

mkdir -p /opt/local/bin/
sudo nano /opt/local/bin/dynamic_pager_init.sh

And now inside that file:

#!/bin/bash
#launch Apple’s dynamic_pager only when the swap volume is mounted

MOUNTPOINT=$1

#check to see if swap storage has been mounted yet
/sbin/mount | /usr/bin/grep -q $MOUNTPOINT

#if $MOUNTPOINT is mounted, then create swap file, otherwise exit1
if [ $? -ne 0 ];then
echo “Swap storage not mounted yet, waiting”;
exit 1
else
echo “Launching dynamic pager on volume Swap”;
/bin/mkdir -p $MOUNTPOINT/.vm
/sbin/dynamic_pager -F $MOUNTPOINT/.vm/swapfile;
fi

Hit CTRL-X and save the file, and then set execute permissions on the file:

sudo chmod +x /opt/local/bin/dynamic_pager_init.sh

Now that the wrapper script is in place, we’re going to tell the launchd config to use this script to create the swap file at boot. Launchd will continually rerun the script until your selected swap volume is mounted, and dynamic_pager runs successfully.

Open your terminal, (/Applications/Utilities/Terminal) and type in:

sudo open /Developer/Applications/Utilities/Property\ List\ Editor.app /System/Library/LaunchDaemons/com.apple.dynamic_pager.plist

You should see the following plist settings:

In order to use the wrapper script instead of calling the dynamic_pager_init binary directly, make your plist look like this:

Make sure to change the value at “item 1″ to the name of your mounted HD. The HD on my system is mounted at “/Volumes/Data”. In order to find out the name of yours, check in the OS X “Disk Utility” or from the Terminal, type in “mount” and find the name of your HD in the list.

I then had to save this file to my desktop, and then copy the file in place via:

sudo cp ~/Desktop/com.apple.dynamic_pager.plist /System/Library/LaunchDaemons/

Now reboot, and test to make sure your swap has been successfully relocated by typing: