Unleash Your Cloud Load Stress Monster

A common question when you prepare your system for the slash dot effect is "How do I check that my system is capable to hold these numbers?"

Location. Location. Location.
You may have the following options:

1. Buying a lot of hardware and setup a one time (or more) lab and check your system capabilities. Pros: its your own servers and you will always find something to do with these extra servers. Cons: It will burn your budget, keep your staff night and days to setup it and will require several days to months to get all installed.
2. Rent a lab and do your things there: Pros: You really don't need to put that amount of money Cons: Schedule the lab, making sure that the hardware and networking meet your needs, making sure software licenses are available... Most important, if you find a major issues in the first day, you will have to close the lab and reschedule another test (and pay again).
3. Setup a cloud based lab. Pros: No setup fees, no need to schedule, no need to commit, and you can save your environment, shut it down, and turn it on when you will need it again. Cons: You don't really own the servers, but hey, who really wants to own servers?

The Tools
OK, so we chose the cloud again. What about the tools? should we choose HP Software LoadRunner or Radview WebLOAD? If so get ready to write a 6 digits number check.
However, the smart choice is selecting the open source tool: Apache JMeter, that can generate HTTP stress (it's a world wide web world after all) at the price of $0. This tool requires you to build the stress scripts in a manual manner using drag and drop, parameters configuration and BASH scripts and it supports visualization using graphs and reports (it also support SOAP, HTTPS, LDAP, JMS, IMAP, JDBC...)
One last thing, JMeter support "bot network" mode, where several JMeter instances can load a single system and provide a unified reporting.

Decisions. Decisions. Decisions.
So we chosen the cloud environment (Amazon AWS currently provides the best offer) and JMeter... now just before launching instances, lets make several decisions that will help us keep costs as low as possible.

Getting Best Prices

1. Windows or Linux: since JMeter is Java based, it's platform independent and Linux will be the smart choice.
2. Spot prices: by using  spot request, you can save about 60% of your CPU cost, and 30% of the total cost.
3. Install both stress loaders and the system in Amazon to avoid paying for traffic.

Stop talking. Start Working.

Lets start with several basic steps:

1. Download Client Tools that will be used to connect using SSH from Windows host :
1. Download WinSCP
2. Download Putty 
3. Download PuttyGen
2. Sign up to Amazon AWS and prepare your user:
1. Sign up
2. Gen a KeyPair from the AWS management console (can be done using CLI if you prefer so).
3. Download the KeyPair (PEM file) and create a private key (PPK file) that can be used by Putty and WinSCP:
1. Open PuttyGen
2. Conversions > Import Key to import your .PEM file
3. Click on "Save private key" to create your private key file (PPK)
   
3. Launch your instance and connect to it
1. Launch an image from EBS based image (if you prefer to keep your work for next time). Use spot request to save some money. Please notice that the default Linux flavor is Fedura.
2.  Connect using WinSCP and the PPK file. CLI should be done by starting Putty from within the WinSCP.
4. Install JMeter and its dependencies:
1. Download and install JMeter
1. Download the JMeter tar file from the site using wget
2. Unzip the file using tar -zxvf file.tar.gz
2. Download and install Java
1. Download java using wget from http://java.com/en/download/manual.jsp and install it
2. Set X permissions on the Java: chmod a+x jre-6u-linux-i586.bin
3. Run jre-6u18-linux-i586.bin to install Java
3. Set environment variables:
1. Set Path: PATH=$PATH:/etc/java/jre1.6.0_18/bin (update it according to path where you installed Java).
2. Set Path: PATH=$PATH:./
5. Launch your JMeter
1. jmeter -n -t my_test.jmx -l log.jtl
2. You may find full details of this syntax in the Apache Jakarta JMeter page:
1. -n: nongui mode
2. -t: the script file you built before
3. -l: the results file

Last Words:
Finally we a stress lab in the cloud, all that left is writing your stress script, installing your system and start stressing it...

Keep Performing,
Moshe Kaplan