Scale Out Patterns for OpenStack (and other Cloud) based Systems

It is a common question these days how one should design their next system to support elastic and growth requirements in the cloud era.

As I got this specific query today, I would like to share with you my answer based on various materials I created in the last few years:

1. Avoid File Storage SPOF: Use AWS S3 or its equivalent open source OpenStack Swift as a central file server repository when needed
How to use OpenStack Swift for your business case

2. Avoid Data Store SPOF: Use clustered data store that recovers automatically w/o a DBA or an operator intervention. Fine examples are AWS MySQL RDS, MongoDB and Cassandra
MongoDB HA concepts

3. Avoid Static Servers: Leverage Autoscale features or customize it for your needs
How to implement your application logic to auto scale your app

4. Avoid Messing with Cache: Use a central sharded cache to avoid cache revocation and reduce data stores load using MongoDB, CouchBase or Redis.

VP R&D Open Seminar: Caching from Moshe Kaplan

5. Offload your servers sessions: in order to avoid users log off and lost transactions:
How to implement a session offloading using a central store

6. Avoid Service Downtime due to Servers Downtime: More issues that can be found in my extensive presentation:
 - Use DNS Load Balancing for Geo LB and DRP (Slide 10)
 - Use CDN to offload network traffic (Slides 16-19)
 - Perform Session Offloading by cookies or a central store (Slides 62-65)

Web systems architecture, Performance and More from Moshe Kaplan

Bottom Line
You can scale your app! just follow the right recommendations

Keep Performing,
Moshe Kaplan

Consider Using SSL? Don't Forget Choosing the Right CDN for that!

Fact #1: More and More Sites are using SSL to Secure Their Users' Transactions
Everybody requires security and privacy these days.
If you don't believe it, take a look at Google, Facebook and Twitter. All them are using HTTPS and SSL to secure all their webpages and API calls including simple feed presentation and search page presentation.
And yes, this fact is still valid even in the post Heartbleed era.

Fact #2: Sites and Widgets are Required for Quicker than Light Loading Time
In the online business, time is money. And faster webpage load times worth a lot of money.

Fact #3: Webpages Secured by SSL have Poor Performance
When you surf for the first time to a website (or take a look at a widget), you are required to have several phases in order to view the website:

1. Resolve the Site DNS.
2. Call for the first Web page.
1. Perform a TCP handshake.
2. Perform a SSL handshake.
3. Retrieve the page itself (+ encryption overhead)
3. Call the embedded resources: images, CSS and JavaScript files (+ encryption overhead).

As you can see, the initial loading of a regular webpage is not short at all. Adding the SSL handshake to this process as well as the encryption and decryption and the overhead on the content, results in even longer times.

What Can be Done?

A common solution is choosing a SSL offloading device such as Radware's Alteon. This device will shorten the encryption and decryption times at the server side. However, it will not reduce the SSL handshake time or shorten the time needed to transfer the page encryption overhead.

The only way to shorten this time is shortening the round trip time between the users and your servers. If this is sounds like a CDN case study, you are right.

CDN is a Key Solution to Managing HTTPS Traffic

Modern CDN solutions support SSL termination at the edge. Therefore, the SSL handshake time can reduced from up to 1 second to several dozens of ms (see in the figures).

This is a major plus to the benefit of shortening the static files serving time by serving a cached copy from the CDN edge.

The good news are that this benefit is valid for both static files and dynamic calls.

Figure 1: Lightweight HTTP REST API Call: Local (Left waterfall) vs Remote (right waterfall), where local call was done from a server located at the same data center as the web server and the remote was done from a remote location with a 200ms round trip to the web server. As we can see most the time is due to the network round trips rather than server processing. Please neglect the initial DNS resolve time.

Figure 2: Same call this time using HTTPS. We can see that the original waiting time was split to two, while SSL connection time doubled it. We can see that in this case as well the SSL processing time at server is neglectable (22ms) while the round trip costs us about 420ms. Please avoid the DNS resolve time in this case as well.

CDN Selection for HTTPS Traffic Cases

While many CDN services support SSL offloading to their own domain (e.g https://your_domain.cdn_provider.com), you probably would like to use your own domain name (e.g https://your_domain.com). Therefore, you should verify that the CDN provider supports custom SSL certificates. Common cloud CDN providers such as AWS and MaxCDN are known to support it, while providers like Microsoft Azure don't. 

Bottom Line

CDN are a corner stone to every web scale deployment these days, and many times you will find they solve issues you were not expecting for them before.

Keep Performing,

Moshe Kaplan

Can OpenStack Object Store be a Base for a Video CDN?

Video CDN are amazing from technical aspect as we are talking about high scale systems with some unique business cases.
I would like to share with you some design aspects regarding these systems.

The Video CDN Case Studies
Video CDN includes two main case studies:

1. VOD Case Study: This is a long tail/high throughput scenario where you need a high capacity disks, where only a small portion of it will be used extensively. In order to create a cost effective solution you should have:
1. High capacity storage system with low IOPS needs. Servers with 24-36 2-3TB SATA disks will provide a up to 100TB raw storage with a tag price of $15K.
2. Replication and auto failover mechanism that can distribute content between several servers and can save us from using expensive RAID and Cluster solutions.
3. Caching/Proxy mechanism that will serve the head of the long tail from the memory.
2. Live Broadcast Case Study: This is a No Storage/high throughput scenario where you actually don't need any persistent storage (if a server fails, as soon as its get up again, the data will be no longer relevant). In order to create a cost effective solution you should have:
1. No significant storage.
2. High capacity RAM that should be sized according to:
1. The number of channels you are going to serve.
2. The number of resolutions your going to support (most relevant when you plan to support handhelds and not just widescreens).
3. The amount of time you are going to store (no more that 5 min are needed in case of live, and no more than 4 hours in case of start over).
3. In memory rapid replication (or ramdisk based) mechanism, that will replicate the incoming video to several machines.
4. HTTP interface to serve video chunks to end users.

Serving Static Content rather than Dynamic
Modern Video Encoding systems (such as Google's Widevine) support "Encrypt Once, Use Many", where the content is encrypted once, and decryption keys are distributed to secured clients based on a need to know basis.

Why OpenStack Swift/Object Store?
In short, OpenStack Swift is the OSS equivalent for Amazon propriety AWS S3: "Simple web services interface that can be used to store and retrieve any amount of data, at any time, from anywhere on the web."

OpenStack Swift/Object Store Benefits

1. Open stack is OSS and therefore easy to evaluate.
2. Active large scale deployments including RackSpace and Comcast.
3. Built in content distribution method that let you distribute the load between multiple servers based on rsync.
4. High availability based on data replication between multiple instances. On a server failure, you just need to take it out of the array and replace with a new one, while other servers keep serving users.
5. This mechanism help you avoid premium hardware such IO controllers and RAID mechanism.
6. Built in HA and DRP based on 5 independent zones.
7. Built in web server, that enables you serving static content as well as HTTP based video streams from the server itself, rather than implementing high end SAN.
8. Built in reverse proxy service that minimizes IO and maximizes throughput, based on Python and Memcache.
9. Built in authentication service
10. Target pricing of $0.4 per 1M servings and $0.055/GB per month if we take AWS as a benchmark.

OpenStack Object Store Architecture
The OpenStack Object Store architecture is well described in two layers:

1. The logical: Accounts (paying customers), Containers (folders) and Objects (blobs).
2. The physical: Zones (Independent clusters), Partitions (of data items), Rings (mapping between URLs and partitions and locations on disks) and Proxies.

Key Concepts

1. Account is actually an independent tenant, as it has its own data store (implemented by SQLite).
2. Replication is done based on large blocks, quorum and MD5.
3. Write to Disk before Expose to Users: When files are uploaded, they are first committed to disk at least at two zones, and then database is updated for availability (so don't expect sub second response for a write).

System Sizing 

Before testing the system you should be familiar with some key sizing metrics obtained by Korea Telecom:

1. Object Store Server Sizing: High capacity storage (36-48 2-3TB SATA drivers that will provide us up to 100TB per server), memory to cache the head of the long tail (24-48GB RAM), 2x1Gbps Eth to support ~500 concurrent requests for long tail request. A single high end CPU should do the work.
2. Proxy Server Sizing: Little storage (500GB SATA disk will do the work), memory to cache the head of the long tail (24 RAM), 2x10Gbps Eth to support ~5000 concurrent requests to support the head of the long tail requests.
3. Switches: you will need a nice backbone for this system. In order to avoid a backbone that is too large, splitting the system to several clusters is recommended.
4. Load Balancing: in order to avoid high end LB, you should use DNS LB, where frequent calls to the DNS are neglect-able relatively to the Media streaming

The Fast Lane: How to Start?
OpenStack Object Store is probably cost effective when looking for large installations as you may need at least 5 physical servers for object and containers store and another 2 for proxies. 
However, you can check the solution based on a single server installation (SAIO):

If you take the fast lane and AWS is the fast lane to your POC, feel free to use the following tips:

1. In the initial installation, some packages will miss in yum so:
1. sudo easy_install eventlet
2. sudo easy_install dnspython
3. sudo easy_install netifaces
4. sudo easy_install pastedeploy
2. No need to start the rsync service (just reboot the machine).
3. Start the service using sudo ./bin/startmain
4. Test the service using  the supload bash script to stimulate a client.

Working with the Web Services
There are 3 main ways to work with Swift web services:

1. AWS tools, as Swift is compliant with AWS S3.
2. HTTP calls as it is based on HTTP.
3. Swift client that streams your major needs.

Working with Swift Client

In the following example we assume a given user test.tester was defined with the password testing, and data is served over the proxy at port 8080

Get statistics:
swift -A http://test.example.com:8080/auth/v1.0 -U test:tester -K testing stat

Upload a file to the videos container:

sudo swift -A http://test.example.com:8080/auth/v1.0 -U test:tester -K testing upload videos ./demo.wvm

Provide read only permissions (Please notice that r: is stands to referral domain, so specifying any other * can help you save bandwidth and minimize content stealing):

sudo swift -A http://test.example.com:8080/auth/v1.0 -U test:tester -K testing post videos  -r '.r:*'

Download the file (where AUTH_test is the user account, videos is the containter and anonymous access was provided as detailed below):
curl http://test.example.com:8080/v1.0/AUTH_test/videos/demo.wvm

Public Access
In order to implement a read only public access your will need to take care of the following items

1. User Management
2. Define anonymous access: delay_auth_decision = 1
3. Configure folder ACL
4. Enable delayed authentication at the proxy configuration (/etc/swift/proxy-server.conf):

[filter:authtoken]
paste.filter_factory = keystone.middleware.auth_token:filter_factory
# Delaying the auth decision is required to support token-less
# usage for anonymous referrers (‘.r:*’).
delay_auth_decision = 1

Working with Direct HTTP Calls

Get user/pwd

curl -v -H 'X-Storage-User: test:tester' -H 'X-Storage-Pass: testing' http://test.example.com:8080/auth/v1.0

> X-Storage-Url: http://127.0.0.1:8080/v1/AUTH_test

> X-Auth-Token: AUTH_tk551e69a150f4439abf6789409f98a047

> Content-Type: text/html; charset=UTF-8

> X-Storage-Token: AUTH_tk551e69a150f4439abf6789409f98a047

Upload file
curl –X PUT -i \

    -H "X-Auth-Token: AUTH_tk26748f1d294343eab28d882a61395f2d" \
    -T /tmp/a.txt \
    https://storage.swiftdrive.com/v1/CF_xer7_343/dogs/JingleRocky.jpg

Bottom Line
OpenStack Object Store (Swift) is exciting tool for anyone who is working with large scale system and especially when talking about CDNs. 

Keep Performing,
Moshe Kaplan

Faster Website? CDN Might be the Answer!

Why Our Japanese Users are Angry?
A common question when you setup a new SaaS or just a fancy website is "Why do my remote users have hard time with my application?" while when I'm in a physical location that is close to my hosting provider I get a relatively good user experience.


It's the TCP/IP Doctor Watson
Window Size is a significant parameter in the TCP/IP stack. This parameter controls how much data can be sent by either party before receiving an ACK from the other. If your Window Size is set to 8KB and the round trip (RTT) between you and the server is 20ms, you could get 400KB/s (1sec/20ms*8KB) or ~4Mbps. 


What If You Were in the Other Side of the Globe?
Well, if your RTT is more like 250ms, you could hardly get 32KB/sec (1sec/250ms*8KB), meaning that you could download a 1MB image in 30sec... Not something to proudly report back home.


CDN. The Question is: How Can I Solve This?
CDN (Content Delivery Network) helps bringing web content closer to the end user. This way it maximizes the throughput controlled by TCP/IP Window size.


How Does CDN Work?

1. The CDN provider places many edge servers around the world (as many as dozens of thousands of locations in some cases).
2. The CDN provider hosts your DNS record in a DNS that supports Anycast. Anycast enables the same DNS record to point to different locations around the world based on the network topology.
3. The end user is directed to the closest edge server based on the Anycast DNS.
4. The edge server tries to serve the file if it is cached on it (images, JavaScript files, static HTML, Streaming Video or just a static copy of your favorite news site are good examples).
5. If the requested file is dynamic or not cached, the edge server requests the file from the origin server using a large TCP window size. This way the 230ms RTT (250ms-20ms) is done in a much larger Window Size (say 64KB) and the throughput will be much larger.

Bottom Line
CDN can help you better your end user experience, use it when needed.


If you need assistance, don't hesitate to communicate,
Keep Performing,
Moshe Kaplan

Cloud Computing Design and Best Practices

Today I presented "Cloud Computing Design and Best Practices" in CloudCon, one of the largest cloud events in Israel.

It was a great lecture and I would like to share with you some of the lecture's insights:

1. Assumptions: Don't refer cloud computing as a brave new world. It is, but when you design your cloud system, don’t forget to keep care of the basic rules you had before.
2. Create A Road Map: You probably will be able to turn your system into a 1 Billion users system in day one. Therefore, design your road map and understand how you will reach your target. In order to get there, you should define the various system parts and you should understand how each of them will be scaled out/removed/replaced in the future to meet your road map.
3. Start Fast: People love success. Your investors love it, Your Marketing guys Love it, Your Customers love it and even Your Development guys do. Therefore, start safe and fast. If your development is great in C#, start with it. If you have an existing software product, start with it as well. It is always better to start with a model based on technologies and products that you are good at, then getting into an adventure that you cannot control its risks.
4. Minimize Costs: After you decided to take the fast track, you should control your costs, and most important: your growth costs. You should go over your business plan and turn it into technical requirements plan and find your bottlenecks. Based on these bottlenecks, define solution for each of them: if you are in the online ads market, take care of your impressions module; if you are in the video business, take care of your video processing module. Why? In the viral and online business the growth is exponential. Therefore, cost growth is exponential, and you have to take care of it before your budget will run away.
5. Best Strategies:
1. Scale out: think of sharing nothing, understand how to take each server and split it to infinite number of servers. If a larger server procurement is your best solution, you probably in the wrong direction.
2. Sharding: Data is usually the largest obstacle for scaling out, as conservative designs concentrate the data in a single place. If you have a similar case, take the path the giants already taken: Shard your database either if it is MySQL or SQL Server (read my Best Seller Sharding Post) 
3. In Memory Database: In Memory is X5-X10 faster than on disk. Therefore, analyze your system, and understand what you can do without going to disk. In case you may be able avoid neglected number of transactions, use this technique to cut your costs.
6. Refactor on the Run: as a player in a growing business, you don't have the option to rest. A 100 users system is different than a 100M users system, and as the system grows, smaller modules that were neglected in the first phase will become more important in terms of bottleneck, cost or business sensitivity. Your way to handle it should be refactor the system, step after step to meet the business goals.
7. Define the Exit Strategy: You should always remember that your cloud operator is still a vendor and your best partner in the early days may become an obstacle when you become a giant. Therefore, choose carefully your cloud provider tools. I would recommend you to think twice before you choose propriety data stores like SimpleDB, and if you do: Create your own interface and have an exit strategy when needed.
8. Everybody is using Open Source. What if your organization is not an expert in this field? As written before, you should start fast. There are plenty of cloud providers that support Windows and .Net, and you can get a head start if you will use the technology you are familiar with. When you grow, you may refactor your product and add technologies to remove your bottlenecks such as Erlang for Push or LAMP to handle your most common processes that result in 90% of your costs.
9. Looking for more strategies?
1. CDN: extract your static and streaming content to CDN provider. This move will cut your server and network utilization and will improve your end user experience.
2. Smart Clients: turn your end user client to be sensitive to network failures. If you are using Gmail and seen the loading... label instead of 404 page, you probably understand what I'm talking about (otherwise Google for JQuery).
3. Elastic Growth: if your system pattern usage is not uniform, consider turning on and off some of your instances to keep costs down and meet the spikes.
4. Replication: don't forget to keep your data safe and meet failures. Make sure you do it using commodity hardware and software.
5. Prepare for downtime & upgrades: make sure that you can always go on. Downtimes will come, since in large scale everything happens, even what should have not. Make sure you never really shut down you whole system even when you upgrade it.
6. NoSQL and SQL: choose SQL as a start if you are great at it. But don't neglect NoSQL when you will get larger.
10. Risk Management: You are going to do a bold move ahead, and you should prepare yourself as I mentioned before:
1. Choose your vendors carefully and know your exit strategy. Your Cloud Operator is a provider as well.
2. Hedge costs and take care of your bottlenecks.
3. Stress your system all the way to guarantee you can get to the next level.
4. Think one move ahead and keep aligned to your strategy.
5. Listen to your users feedback. Your business depends on your clients; take care of them.

Cloud Computing Design Best Practices

View more presentations from Moshe Kaplan.

Bottom Line
Working according to these rules can help you reach the 1 Bill users system you were looking for


Keep Performing,
Moshe Kaplan

The Path to the Cloud

You decided to make the big decision and migrate to the cloud. 

What Should You Do Now?

Well, migrating an active system between two hosting providers (and even from your small desktop under your desk), can be risky and may affect the business due to downtime or user frustration. Migration to the cloud that enables a new paradigm can be even more complex since it requires changing concepts and deal with a new brave world..

So How Should We Start?

The first rule is to not think of the cloud as a cloud. You should prepare for a simple migration (think of the cloud provider just as another hosting provider). This way you can minimize the major risk of incompatibility. Follow these steps to accomplish the first step:

1. Prepare your integration environment. If you don't have one, it's time for you to have one. If you do have, it's time to make sure that it's 100% compatible with your production environment. We'll use this environment in order to stimulate the migration.
2. Arrange your DNS records. Migrating to another location will probably require modifying your DNS records to reflect the new location. Some hosting providers do not allow naming for external locations (for example GoGrid is an example for such a provider). So you should verify that you know how to change your DNS records, or migrate the records to another register.
3. Implement a DRP environment. You should have one first in your current provider environment in order to eliminate issues such as access lists and network latency. Why is it so important?
1. Hosting migration is risky. You would like to have easy to roll back in case of non successful migration
2. Data Migration take a lot of time. If your databases (or NoSQL) are large, it will take a long time to migrate them between locations (even using a 100Mb channel, it will take a 3 hours downtime to migrate a 100GB database). The most efficient way to shorten this time will be by using Log shipping between the databases, or implement a replication between 2 NoSQL sites. When you'll choose to perform the migration to the new site, all you'll need to do is stopping the primary instance and turn the passive instance to primary one. Since the data is being replicated between sites all the time, the data migration time slot will be minimized to at most few minutes. 
4. Migrate your DRP environment to the new cloud provider. Start with the integration environment in order to verify there are no network issues in implementing it.
5. Verify the new provider stability. Now that you have servers in the new hosting provider, it's time to verify it stability, its network performance and any other issues that may arise and you were not expected to.
6. Implement a Reverse Proxy Solution. Some of your client will be ready for the migration. They may have old stale DNS records or they might use your IP address for some kind of reason. Using reverse proxy can help your routing traffic from your old location to the new one, minimizing lost traffic and downtime.
7. Perform migration test in the integration environment. You should turn the passive site into active one, and check that user can continue work after the downtime. You should document the process, and if times are not good enough, you should exercise the process. After you are satisfied with results, it's time for prime time.
8. Migrate your production DRP site the new hosting provider. Perform the process in a similar manner to the integration DRP migration.
9. Prepare for The big day. Transform your DRP (or the new hosting environment) into a production one.
10. Verify. Wait a few days and verify the system was stabilized, if so make few steps before closing your old hosting facilities and cut costs:
1. Implement a new DRP site both for the integration and the production environments.
2. Verify that the routed traffic from the old site is neglected.
11. The big day. it's time to shut down your old server, close the site and say bye-bye to your old provider.

Just Scale It
After doing so much work, it's time to decide what cloud items do you want to you. It may depend on the cloud provider offering, and your decision how and if you want to avoid vendor lock in:

1. CDN: if you have major static files traffic, or if you are able to turn some of the traffic into static one (think of Linkedin profiles for example), you could store it in CDN facilities that offer can reduce your web servers traffic and better prepare your for peaks. It also can help you 
2. Queues: Avoid taking care of queues availability and let this internet giants take care of it.
3. NoSQL stores: you could implement open source solutions such as Cassandra or chose the provider key value store.
4. Rational Databases: Some cloud providers offer these days a database as a Service that let you get rid of these log shipping, clustering and care of daily backups. If you are interested, drop me a note and I refer you to some interesting companies.
5. Load Balancing: You could choose using software based load balancers such as HAProxy, or choosing the cloud solution.
6. Monitoring: Some cloud provider offer you a system monitoring, saving you major effort to monitor your servers.

Take It One Step Further

Now that you chosen the tools you just have to make sure your system can scale out:

1. Separate back office services from web interfaces, creating pools of small machines that each can take care of their tasks.
2. Parallelize your back end services, making sure that no single service turns into a future bottleneck.
3. In case of push systems and communication between parties, create a directory system that can route events between different servers.

Bottom Line

If you felt that you are bounded to your current hosting provider, it's time to think again. Using this simple, yet not so short method, you could reduce risks and accomplish the change.

Keep Performing,

Moshe Kaplan

Lessons from Facebook:

FB maybe is not the best example for quality design (probably each of us is likely to have at least a single error message a day).

However, since it's the 2nd largest site in the world with contsant exponential growth, it's a good lesson to have. Most interesting that their tech is team is only about 250 people and 30K servers, pretty amazing.


Not long ago a Jeff Rothschild, the Vice President of Technology at Facebook gave a presentation in UC San Diego. You can find a detailed summary by Prof. Amin Vahdat, but I'll make several top comments that I found useful.

1. FB Software development: mostly PHP (complied), but other languages are used as well.
   
2. Common interface between services using an internal development that was turned to open source: Thirft. This interface enables easy connections between the different languages.
   
3. Logging Framework that is not dependent on a central repository or its availability. FB is using Hadoop and well as Hive (that was developed there as well). The log size is growing at a 25TB a day.
   
4. Operational Monitoring: Separated from the logging mechanism 
5. LAN can be a bottleneck as well: expect for packet loss and packet drops in the LAN if you stress it too much.
   
6. CDN: Facebook is using external CDN to images distribution.
   
7. Dedicated file system named Haystack that combines simple storage along with cache directory: file system is accessed only once to get images from it, while directory structure is retrieved from the cache.
   
8. Most data is served from Memcached. Database is used mostly for Persistency and data replication between sites (Memcached is being heated by the MySQL itself):
1. Top challenge: keeping data consistent since Memcached can be messed easily (No search for keys is available).
2. Mixing information including sizes and types is better - making sure that load on CPU, Memory and etc is distributed equally.
   
9. Shared nothing - Keep your system independent - avoid a single bottleneck. Therefore, data is saved in a sharded MySQL from day 1. However, MySQL is used mostly for data persistency and not for conservative database usage pattern:
1. No Joins in the MySQL
2. Chosen due to good data consistency + Management Software
3. 4K servers
   
4. Data replication between sites is based on MySQL replication
5. Memcached is being heated based on the MySQL Replication using a custom API

One last thing, if you are interested in Facebook financials, as well the storage machines (NetApp low end 3070), sizing, traffic, servers, storage procurement and data center costs, take a look at TechCrunch's Michael Arrington post.

Keep Performing,
Moshe Kaplan. Performance Expert.

Load Balancer: Pay less Do More

When we face a buildup of a new mega system that needs to handle dozens of Gb/s of traffic, supporting both HTTPS and HTTP and perform advanced logic such as Comet our main obstacle is how to design it to keep performance high and price low.

Well if you are dealing with such a system (I seen several of these in the last few months) you probably already familiar with hardware load balancers, software load balancers, Acceleration servers (SSL Proxies), caching servers, DNS round robin and CDNs. If you are not sure yet what to do with these components and why, stay with us...
However, please notice that the following is a short list of various technologies available in the market with a bottom line, this is not a complete list and each case should be analyzed according to the case (I think I should hire a lawyer next time...). If you want to learn more feel free to read Willy Tarreau article: "Making applications scalable with Load Balancing" which covers the overall aspects of these technologies.

DNS Round Robin: 
Major Pros: Cheap and simple.
Major Cons: Not dynamic (unless you monitor and manually change it) and if a server fails clients will have major problems acquiring new server address.
Bottom Line: Use it only when servers are for sure up and running (meaning that every DNS node is HA). Very useful to balance several data centers worldwide.

Hardware Load Balancers:

Major Pros: High throughput (10Gbs and counting) low latency and built in HA using VRRP.
Major Cons: Price ($$$).
Bottom Line: Use it for high throughput low latency load balancing or in other words for layer 3/4 load balancing in the data center gateway to balance the traffic between the servers. Cisco, Radware and F5 are good examples.

Accelerators (SSL Proxies and Compression):

Major Pros: Remove overhead from the application servers and keeping traffic as small as possible.
Major Cons: Another layer in the system.
Bottom Line: Use a stack of Apache+ModSSL servers to encrypt/decrypt and compress/decompress behind the HW load balancer keeping cost low and performance high.

Software Load Balancers:

Major Pros: Low cost.
Major Cons: Slow and low throughput.
Bottom Line: Use it when load balancing is almost in the application level (Layer 7) like HTTP Redirect. HAProxy and Apache mod_proxy are good examples.

CDN (Content Delivery Network or Static Files):

Major Pros: Reduce the number of servers in the system
Major Cons: Another layer in the system.
Bottom Line: Use commercial CDN (Amazon S3 and CloudFront are good examples) or dedicated lightweight HTTP like lighttpd to serve static content. It is also recommended to convert dynamic content to static one if possible.

Application Server:

Major Pros: Well we hope at least part of your system is dynamic...
Major Cons: You know, performance boosting is not a joke after all...
Bottom Line: Use optimized system from system to application level, including caching and in memory database to server the dynamic part of the application.

Database:

Major Pros: Well we hope at least part of your system is dynamic... (X2).
Major Cons: You know, performance boosting is not a joke after all... (X2).
Bottom Line: If you plan a very large database, we recommend planning for HA and Sharding from day 1. It's not so difficult and it'll save you a lot work and sleepless nights in the future. And yes, you can use commodity databases such as MySQL.

The bottom line: a wise use of each component can lead you to highly available system while keeping your budget low.

Keep Performing,
Moshe Kaplan. RockeTier. The Performance Experts.