MySQL CentOS and Amazon Web Services (AWS)

Many Linux experts prefer CentOS. The reason is clear, Redhat's RHEL is probably the best Linux server distribution and CentOS is very much a like (except for the yearly fees).
This distribution (and Fedora as well) is very common for MySQL installation, since there are many predefined packages for it. This is unlike other distributions such as Ubuntu.

Therefore, when you will perform your database migration to the cloud, I recommend that you will first verify your needs. The following items will help you implement your systems right:

1. Select the Linux flavor
1. CentOS: Detailed options are available bellow. Don't forget to verify the recent CentOS version: Usually there are two (an early major version such as 6.0 and an old advanced minor version such as 5.7) and you should select the right version for you.
2. Fedora: Some free and good AMIs are available.
3. RHEL: There is a premium AMI for it
4. Ubuntu: No formal MySQL distribution exists to this flavor.
5. Debian.
2. Select the Storage Type. Amazon AMIs can store their information on one of two storage type:
1. Instance storage: this is a server attached disks storage that is great from plain performance issues (no network traffic and latency for storage). Yet, it lacks the option to extend the disk seamlessly to a larger size and backup must be done through the network instead on an instant disk duplication (shorter downtime).
2. EBS: this is Storage Area Network based disks that easily can be extended and backed up with minimal server downtime. Yet, this storage costs both in money (~$0.15/GB per month) and network latency.
3. Select the right image for you:
1. RightScale Image: (last update Jan 2009. If you don't use RightScale, don't forget to remove their management tools and don't forget to update your version to a recent one. RightScale has the two types of images.
2. Amazon Linux AMI: The AWS image is based on CentOS image, yet it includes some tweaks. The plus is having an Out of the Box solution with Amazon support. The minus is that it's not a 100% RHEL compatible anymore.
4. Install your database (MySQL probably) and web server (Apache probably) on the machine.
1. The recommended MySQL version is MySQL 5.5 with major performance enhancement since MySQL 5.1.

Bottom Line

Now, you just need is to choose the configuration that most fit your needs.

Keep Performing,

Moshe Kaplan

Moshe Kaplan is an entrepreneur and a cloud systems consultant

At My Command Unleash Version!

Click on the post title to read the full post and the comments.
 
As you may know I'm a big fun of Continuous Integration, a.k.a CI, (running every trunk commit with some unit tests and integration tests, finding that you ruined the build and earning the blame T-shirt is such a good way to start the morning :-).

Cool things you can do with CI
However, today's CI is much more from just running several tests. These days you can do almost everything with it:

1. Get your latest version from trunk.
2. Build the version.
3. Launch a new virtual machine based on a predefined image or by installing it almost from scratch.
4. Install the version on the new machine.
5. Run smoke tests to provide an instant validation of the code stability.
6. Run nightly builds to carefully check that the version is clean.
7. Run stress tests to validate capacity and long multiple days tests to validate memory leaks.
8. Notify key players with the results.

Cloud Related Cool Scenarios
Cloud computing is basically about obtaining on demand resources through API (the GUI is a fancy stuff that can be useful but is not really a must). Therefore, you can use CI to automate every step in your testing, upload to production and rollback when needed. Some of the scenarios are:

1. Performing Software Upgrade: Launching a new instance, installing the latest software version and creating an Image (AMI in AWS).
2. Launching an entire new integration environment that is identical to the production environment and check its functionality as well as its correspondence with the non functional requirements.
3. Launching a new environment for marketing needs.
4. Managing your new version upload, by creating a script that will prepare new servers, upgrade your database servers (or RDS), add new application and web servers to the load balancer farms, and finally will remove the old server and shut them down.
5. Implement Failover scenarios (DRP) 

Industry Best Practices
According to Amazon (see Jinesh Varia, Technology Evangelist @Amazon Web Services document), CI and automation is the best practice to manage your cloud environment:

1. Create a management server (like CruiseControl or regular CRON jobs).
2. Create a build-test-deploy cycle script using the chosen management server that will generate:
1. New AMI
2. Deploy new servers based on these AMI
3. Add these servers to existing load balancer
4. Attach elastic IPs
5. Generate static files to S3 and CloudFront
6. Generate new queues and notifications using SQS and SNS
7. Remove old servers and unneeded AMIs

Using this method can save you a lot of time in configuration and optimization, as well as many production issues due to human errors.

What CI to choose?
There are many CI products in the market, some of them are open source and other are commercial. Here are some of leading tools, thanks to Alon Nativ advice:

1. CruiseControl: a leading open source product with dozens of plugins and 3rd parties.
2. Apache Continuum: an open source product with many features.
3. Final Builder: a commercial and major tool in the market with great visual interface and many built in scripts and API
4. JetBrains TeamCity: a commercial tool with a built in integration to Amazon AWS.

You can also take a look at RightScale product and services. RightScale provides semi CI in a SaaS manner with many features we discussed before and more. I included here a RightScale video, which is one of the best industry resources regarding this issue:


Bottom Line
Now after going over the details, I recommend you to:

1. Design your cloud environment.
2. Design your baseline AMIs/Images.
3. Design your upgrade, rollback and test-build-deploy scripts based on these basic AMIs.
4. Implement them using your chosen Continuous Integration tool.
5. Roll it out to production.

Keep Performing,
Moshe Kaplan

Managing Your Resources in the Cloud

One of the major issues while working in the cloud is managing your resources.

Why?
When you have your servers in your own data center, you can touch them, see them and count them. And of course you pay a lot of equity for them.
When migrating to the cloud your CapEx is finally zero, however, if you'll not manage your resources wisely, your OpEx will soon right and be larger than your old CapEx budget. Moreover, when working with a contractor (yes, your cloud computing provider is now your contractor), you should manage it as a contrator. meanning that you should monitor your resources and verify the service level.
Moreover, many cloud clients are soon will find themselves runing dozens of servers, which in enterprises usually require a significant effort including command and control systems, NOC, helpdesk and so on.

What should be my requirements?

So what can I do?
Well you have several options:
A. Develop your own managment consule based on the chosen cloud computing provider API, monitoring each server resources.
B. Use enterprise world command and control systems such as CA Unicenter, IBM Tivoli suite, HP OpenView suite, BMC or Microsoft.
C. Use cloud monitoring niche systems. The largest and most significant player in this field is RightScale which provides both monitoring and auto scaling service. Another new player in this market is cloudkick. This player backed by the VC botique Y Combinator, provides its monitoring, graphs and alerting system free of charge.

Bottom line:
Option A (Develop your own solution) is too tedious and will walk you out of your main course of business. Option B is nice, but most of these players are fouces in an on premise market, and it will take them time to get into the cloud market. Moreover, most cloud computing clients are start ups and SaaS providers, that these massive C&M systems will not fits their needs.
Option C in the current time seems to be the right one if you established your operations in the cloud. From our analysis cloudkick is a nice start for a free service, but there is way to go. For exmpale, it still missing the auto scalling feature (if my CPU is over 80%, provide me ASAP new EC2 instance). So our recommandations in the current is usign RightScale.

Keep Performing
Moshe Kaplan. RockeTier. The Performance and Cloud Experts.