Showing posts with label Load Balancers. Show all posts
Showing posts with label Load Balancers. Show all posts

Nov 5, 2013

Apache Session Persistancy Using MongoDB

Your app is booming, you need more web servers and you need to serve users and keep their user experience. When you had a single server you just used session for that, but now, How do you keep sessions across multiple web servers? 
Using a session stickiness load balancer is a good solution, but what happens when a web server needs a restart?

Session Off Loading
Many of you are familiar with the concept of session off loading. The common configuration for it is using Memcached to store the session object, so it is accessible from all web servers. However, this solution lacks the features of high availability by replication and persistence.

The Next Step: High Availablability
Offloading web servers sessions to MongoDB looks like a great solution: key-value store, lazy persistency and built in replication and auto master failover.

Step by Step Solution for LAMP Session Off Loading using MongoDB
  1. Install MongoDB (preferably in a replica set with at least 3 nodes)
  2. Install MongoDB driver for PHP: 
  3. Configure the php.ini: extension=mongo.so
  4. Download and integrate Apache session handler implementation using MongoDB.
  5. Configure your MongoDB setting in the module:
    1. Set the list of nodes in the connectionString setting: 'connectionString' => 'mongodb://SERVER1:27017, mongodb://SERVER2:27017',
    2. Do the same to the list of servers (see at the bottom of this post)
    3. Configure the cookie_domain (or just place a null string there to support all): 'cookie_domain' => ''
    4. Don't forget to enable replica set if you are using one: 'replicaSet' => true,
  6. Add the MongoSession.php to your server and require it: require_once('MongoSession.php');
  7. Replace the session_start() function with $session = new MongoSession();
And that's all. Now you can continue using the session objects as you used to create new great features:
if (!isset($_SESSION['views'])) $_SESSION['views'] = 0;
$_SESSION['views'] = $_SESSION['views'] + 1;
echo $_SESSION['views'];


Bottom Line
Smart work can tackle every scale issue you are facing off...

Keep Performing,

Appendix: 
'servers' => array(
array(
'host' => 'SERVER1',
'port' => Mongo::DEFAULT_PORT,
'username' => null,
'password' => null,
'persistent' => false
),
array(
'host' => 'SERVER2',
'port' => Mongo::DEFAULT_PORT,
'username' => null,
'password' => null,
'persistent' => false

)

Oct 11, 2010

The Path to the Cloud

You decided to make the big decision and migrate to the cloud. 

What Should You Do Now?
Well, migrating an active system between two hosting providers (and even from your small desktop under your desk), can be risky and may affect the business due to downtime or user frustration. Migration to the cloud that enables a new paradigm can be even more complex since it requires changing concepts and deal with a new brave world..

So How Should We Start?
The first rule is to not think of the cloud as a cloud. You should prepare for a simple migration (think of the cloud provider just as another hosting provider). This way you can minimize the major risk of incompatibility. Follow these steps to accomplish the first step:
  1. Prepare your integration environment. If you don't have one, it's time for you to have one. If you do have, it's time to make sure that it's 100% compatible with your production environment. We'll use this environment in order to stimulate the migration.
  2. Arrange your DNS records. Migrating to another location will probably require modifying your DNS records to reflect the new location. Some hosting providers do not allow naming for external locations (for example GoGrid is an example for such a provider). So you should verify that you know how to change your DNS records, or migrate the records to another register.
  3. Implement a DRP environment. You should have one first in your current provider environment in order to eliminate issues such as access lists and network latency. Why is it so important?
    1. Hosting migration is risky. You would like to have easy to roll back in case of non successful migration
    2. Data Migration take a lot of time. If your databases (or NoSQL) are large, it will take a long time to migrate them between locations (even using a 100Mb channel, it will take a 3 hours downtime to migrate a 100GB database). The most efficient way to shorten this time will be by using Log shipping between the databases, or implement a replication between 2 NoSQL sites. When you'll choose to perform the migration to the new site, all you'll need to do is stopping the primary instance and turn the passive instance to primary one. Since the data is being replicated between sites all the time, the data migration time slot will be minimized to at most few minutes. 
  4. Migrate your DRP environment to the new cloud provider. Start with the integration environment in order to verify there are no network issues in implementing it.
  5. Verify the new provider stability. Now that you have servers in the new hosting provider, it's time to verify it stability, its network performance and any other issues that may arise and you were not expected to.
  6. Implement a Reverse Proxy Solution. Some of your client will be ready for the migration. They may have old stale DNS records or they might use your IP address for some kind of reason. Using reverse proxy can help your routing traffic from your old location to the new one, minimizing lost traffic and downtime.
  7. Perform migration test in the integration environment. You should turn the passive site into active one, and check that user can continue work after the downtime. You should document the process, and if times are not good enough, you should exercise the process. After you are satisfied with results, it's time for prime time.
  8. Migrate your production DRP site the new hosting provider. Perform the process in a similar manner to the integration DRP migration.
  9. Prepare for The big day. Transform your DRP (or the new hosting environment) into a production one.
  10. Verify. Wait a few days and verify the system was stabilized, if so make few steps before closing your old hosting facilities and cut costs:
    1. Implement a new DRP site both for the integration and the production environments.
    2. Verify that the routed traffic from the old site is neglected.
  11. The big day. it's time to shut down your old server, close the site and say bye-bye to your old provider.
Just Scale It
After doing so much work, it's time to decide what cloud items do you want to you. It may depend on the cloud provider offering, and your decision how and if you want to avoid vendor lock in:

  1. CDN: if you have major static files traffic, or if you are able to turn some of the traffic into static one (think of Linkedin profiles for example), you could store it in CDN facilities that offer can reduce your web servers traffic and better prepare your for peaks. It also can help you 
  2. Queues: Avoid taking care of queues availability and let this internet giants take care of it.
  3. NoSQL stores: you could implement open source solutions such as Cassandra or chose the provider key value store.
  4. Rational Databases: Some cloud providers offer these days a database as a Service that let you get rid of these log shipping, clustering and care of daily backups. If you are interested, drop me a note and I refer you to some interesting companies.
  5. Load Balancing: You could choose using software based load balancers such as HAProxy, or choosing the cloud solution.
  6. Monitoring: Some cloud provider offer you a system monitoring, saving you major effort to monitor your servers.
Take It One Step Further
Now that you chosen the tools you just have to make sure your system can scale out:
  1. Separate back office services from web interfaces, creating pools of small machines that each can take care of their tasks.
  2. Parallelize your back end services, making sure that no single service turns into a future bottleneck.
  3. In case of push systems and communication between parties, create a directory system that can route events between different servers.
Bottom Line
If you felt that you are bounded to your current hosting provider, it's time to think again. Using this simple, yet not so short method, you could reduce risks and accomplish the change.

Keep Performing,

Dec 19, 2009

Load Balacing Support in Dynamic Environments

Hi,

The Mission
These days we face a challenging task: designing a very large system of scalable instances. Each of these instances may be in a different geographic location, and many of them are on demand instances that are  being started and shutdown instantly.
Another requirement in this system that a given client will be direct to a defined instance, due to system restriction (a round robin is not an option is this case).

One Step Further
Since the number of IP addresses in the internet is limited, we would like to use as few as possible Public IP addresses. This can be done using a load balancer or a proxy.
In the current state we would like to avoid using hardware load balancers in order to keep initial fixed costs minimal, but we may consider to use them in the future.

Is Amazon Cloud Load Balancer Service (AWS) is an Option?

AWS EC2 instances is a feasible option for on demand instances hosing. However, AWS charges $0.025 per a single load balancing rule per hour (+traffic). Therefore, it can be used, but for a large number of rules (>7) or high traffic, better solutions can be found in the market.

So What Can Be Done?
We left with software load balancers. The major ones Apache mod_proxy and HAProxy. Supporting large number of instances behind the load balancer can be done in one of the following two options:
  1. Pre register a large number of DNS addresses (sub domain) and associate them with the load balacner IP. The load balancer will simply redirect the request to the defined instance, based on the a simple rule in the load balancer. For example: http://instN.foofarm.com  Pros: simple. Cons: not fully dynamic, requires additional DNS registration once in a while to keep up with the application growth.
  2. Performing ProxyPass in the Load balancer: Every request will include in its path an instance identification for example: http://foofarm.com/instN/. This method does not require mass DNS declarations, but it requires specific definitions in the load balancers that may be more CPU consuming. In Apache the definition is pretty trivial, however, this product is less scalable from HAProxy. In HAProxy the task can be done as well based on a two phases: switching to the server and rewrite the URI:
    In order to switch to the server, you have to use ACLs to match the path,
    then a use_backend directive to select a server farm ("backend"). Your
    farm may very well support only one server if you want.

    Then in this "backend", you can use a rewrite rule ("reqrep") to replace
    the request line.

    This would basically look like this :

    frontend xxx
           acl path_mirror_foo path_beg /mirror/foo/
           use_backend bk_66 if path_mirror_foo

    backend bk_66
           reqrep ^([^: ]*\ )/instN/\(.*\)  \1/\2
           balance roundrobin
           server srv66 192.88.77.66:80

However, Willy Tarreau, HAProxy author who kindly provided this hint for me, recommends that you avoid the second part (rewriting) because :

 1) it requires good regex skills which sometimes makes the configs hard
    to maintain for other people

 2) rewriting URIs in applications is the worst ever thing to do, because
    they never know where they are mapped, and regularly emit wrong links
    and wrong Location headers during redirections.

Willy Tarreau also advices that the best thing to do clearly is to correctly configure your application to be able to respond with the real, original URI. Remapping it can be used
as a transitional setup in order to ease a graceful switchover, though. Bottom line: Pros: No DNS configuration and fully scalable solution, with no dependence on DNS replication. Cons: CPU Consuming and error prone declarations.

So, What to Choose?
The answer is based on your needs, and your believe in your people regex capabilities. We made our choice.

Keep Performing,
Moshe Kaplan. Performance Expert.

Sep 26, 2009

Load Balancer: Pay less Do More

When we face a buildup of a new mega system that needs to handle dozens of Gb/s of traffic, supporting both HTTPS and HTTP and perform advanced logic such as Comet our main obstacle is how to design it to keep performance high and price low.

Well if you are dealing with such a system (I seen several of these in the last few months) you probably already familiar with hardware load balancers, software load balancers, Acceleration servers (SSL Proxies), caching servers, DNS round robin and CDNs. If you are not sure yet what to do with these components and why, stay with us...
However, please notice that the following is a short list of various technologies available in the market with a bottom line, this is not a complete list and each case should be analyzed according to the case (I think I should hire a lawyer next time...). If you want to learn more feel free to read Willy Tarreau article: "Making applications scalable with Load Balancing" which covers the overall aspects of these technologies.




DNS Round Robin: 
Major Pros: Cheap and simple.
Major Cons: Not dynamic (unless you monitor and manually change it) and if a server fails clients will have major problems acquiring new server address.
Bottom Line: Use it only when servers are for sure up and running (meaning that every DNS node is HA). Very useful to balance several data centers worldwide.

Hardware Load Balancers:

Major Pros: High throughput (10Gbs and counting) low latency and built in HA using VRRP.
Major Cons: Price ($$$).
Bottom Line: Use it for high throughput low latency load balancing or in other words for layer 3/4 load balancing in the data center gateway to balance the traffic between the servers. Cisco, Radware and F5 are good examples.

Accelerators (SSL Proxies and Compression):

Major Pros: Remove overhead from the application servers and keeping traffic as small as possible.
Major Cons: Another layer in the system.
Bottom Line: Use a stack of Apache+ModSSL servers to encrypt/decrypt and compress/decompress behind the HW load balancer keeping cost low and performance high.

Software Load Balancers:

Major Pros: Low cost.
Major Cons: Slow and low throughput.
Bottom Line: Use it when load balancing is almost in the application level (Layer 7) like HTTP Redirect. HAProxy and Apache mod_proxy are good examples.

CDN (Content Delivery Network or Static Files):

Major Pros: Reduce the number of servers in the system
Major Cons: Another layer in the system.
Bottom Line: Use commercial CDN (Amazon S3 and CloudFront are good examples) or dedicated lightweight HTTP like lighttpd to serve static content. It is also recommended to convert dynamic content to static one if possible.

Application Server:

Major Pros: Well we hope at least part of your system is dynamic...
Major Cons: You know, performance boosting is not a joke after all...
Bottom Line: Use optimized system from system to application level, including caching and in memory database to server the dynamic part of the application.

Database:

Major Pros: Well we hope at least part of your system is dynamic... (X2).
Major Cons: You know, performance boosting is not a joke after all... (X2).
Bottom Line: If you plan a very large database, we recommend planning for HA and Sharding from day 1. It's not so difficult and it'll save you a lot work and sleepless nights in the future. And yes, you can use commodity databases such as MySQL.

The bottom line: a wise use of each component can lead you to highly available system while keeping your budget low.

Keep Performing,
Moshe Kaplan. RockeTier. The Performance Experts.

Jun 22, 2009

Billion Events per Day, Israel 3rd Java Technology Day, June 22, 2009

We presented today at the Israeli 3rd Java Technology Day, the largest SUN Microsystems/MySQL event in Israel. We presented here the essentials parts of building a real life web/enterprise system that needs to handle the performance needs of 1 billion events per day (a case study from the ad networks billing systems). We presented the adoption rate in the internet, Load Balancers (HAProxy, Apache, Radware, F5, Cisco), Web Servers, In Memory Database (IMDB inc. Memcached, Gigaspaces, Teracotta and Oracle Coherence) and finally Sharding (inc. Veritical, Static Horizontal and dynamic). A great example for a performance boosting architecture.

Feel free to take a look at the presentation:


ShareThis

Intense Debate Comments

Ratings and Recommendations