Showing posts with label Cloud Auto Scaling. Show all posts
Showing posts with label Cloud Auto Scaling. Show all posts

Jul 10, 2014

Scale Out Patterns for OpenStack (and other Cloud) based Systems

It is a common question these days how one should design their next system to support elastic and growth requirements in the cloud era.

As I got this specific query today, I would like to share with you my answer based on various materials I created in the last few years:


1. Avoid File Storage SPOF: Use AWS S3 or its equivalent open source OpenStack Swift as a central file server repository when needed

How to use OpenStack Swift for your business case

2. Avoid Data Store SPOF: Use clustered data store that recovers automatically w/o a DBA or an operator intervention. Fine examples are AWS MySQL RDS, MongoDB and Cassandra

MongoDB HA concepts

3. Avoid Static Servers: Leverage Autoscale features or customize it for your needs

How to implement your application logic to auto scale your app

4. Avoid Messing with Cache: Use a central sharded cache to avoid cache revocation and reduce data stores load using MongoDB, CouchBase or Redis.




5. Offload your servers sessions: in order to avoid users log off and lost transactions:

How to implement a session offloading using a central store

6. Avoid Service Downtime due to Servers Downtime: More issues that can be found in my extensive presentation:

 - Use DNS Load Balancing for Geo LB and DRP (Slide 10)
 - Use CDN to offload network traffic (Slides 16-19)
 - Perform Session Offloading by cookies or a central store (Slides 62-65)



Bottom Line
You can scale your app! just follow the right recommendations

Keep Performing,

Moshe Kaplan

Jun 15, 2014

Auto Scaling your Workers based on Queue Length

Having an image or video processing engine?
Having a web crawling solution?
Doing OCR on media?
If so, this post is probably for you.

The Producer-Consumer Auto Scaling Problem
Back at school, you probably learnt how to manage the queue of waiting tasks in a producer-consumer problem. You may learnt how to avoid expired tasks and double processing of same tasks.
However, back then we all assumed the number of workers and producers is fixed.

Well, it is not true any more...
If you system includes a backend processor that its load may vary based on clients demand, you may need to auto scale it. Back then, you needed to purchase new servers. These days it just about launching new instances in your favorite cloud.

Adjusting AWS Auto Scaling to Support this Pattern
AWS Auto Scaling solution is known as the best of its kind in the industry. It supports:
  1. Dynamic scaling based on instances load (e.g CPU).
  2. Automatically launching and terminating instances.
  3. Supporting both on demand (expensive) instances and Spot (marginal cost) instances.
However, it does not support complex decisions such as queue length/instances length ratio out of the box.

Fortunately, AWS provides a complete set of API that let us define a solution that its general flow is described here and here:
  1. Create a base AMI to a worker/consumer machine
    1. If your code tends to change you may define a method that will update to the latest code when machine is being launched
    2. A better solution may be using a DevOps platform such as Puppet or Chef.
  2. Define auto scaling initial configuration with an initial size of minimum and maximum number of instances (N = 1).
  3. Define a spot instance price: make it reasonable (not too high to avoid extra cost, and not too low to actually launch instances when needed).
  4. Create a cron that will run every 1 min and will check the queue length. This cron will terminate or launch instances by adjusting the auto scaling configuration minimum and maximum number of instances (N).
The Algorithm
You need to take several decisions in order to decide how many instances you want on air:
  1. What is the ratio of tasks in queue (Q) to running instances (for example R = 2)
  2. How quick do you want to reach this ratio. The quicker you do, the higher it will cost (instances that are billed hourly, may be terminated and relaunched again only few minutes after that).
Two examples for a possible algorithms:
  1. Keeping the ratio of queue length and running instances constant: for example: N = max[1, Q/R]
  2. Soften the ratio by using previous calculated number N': N = (N' + max[1, Q/R])/2
Bottom Line
Auto Scaling is an amazing tool, and by adjusting it we can easily solve complex issues.

Keep Performing,

Apr 28, 2009

Managing Your Resources in the Cloud

One of the major issues while working in the cloud is managing your resources.

Why?
When you have your servers in your own data center, you can touch them, see them and count them. And of course you pay a lot of equity for them.
When migrating to the cloud your CapEx is finally zero, however, if you'll not manage your resources wisely, your OpEx will soon right and be larger than your old CapEx budget. Moreover, when working with a contractor (yes, your cloud computing provider is now your contractor), you should manage it as a contrator. meanning that you should monitor your resources and verify the service level.
Moreover, many cloud clients are soon will find themselves runing dozens of servers, which in enterprises usually require a significant effort including command and control systems, NOC, helpdesk and so on.

What should be my requirements?

So what can I do?
Well you have several options:
A. Develop your own managment consule based on the chosen cloud computing provider API, monitoring each server resources.
B. Use enterprise world command and control systems such as CA Unicenter, IBM Tivoli suite, HP OpenView suite, BMC or Microsoft.
C. Use cloud monitoring niche systems. The largest and most significant player in this field is RightScale which provides both monitoring and auto scaling service. Another new player in this market is cloudkick. This player backed by the VC botique Y Combinator, provides its monitoring, graphs and alerting system free of charge.

Bottom line:
Option A (Develop your own solution) is too tedious and will walk you out of your main course of business. Option B is nice, but most of these players are fouces in an on premise market, and it will take them time to get into the cloud market. Moreover, most cloud computing clients are start ups and SaaS providers, that these massive C&M systems will not fits their needs.
Option C in the current time seems to be the right one if you established your operations in the cloud. From our analysis cloudkick is a nice start for a free service, but there is way to go. For exmpale, it still missing the auto scalling feature (if my CPU is over 80%, provide me ASAP new EC2 instance). So our recommandations in the current is usign RightScale.

Keep Performing
Moshe Kaplan. RockeTier. The Performance and Cloud Experts.

ShareThis

Intense Debate Comments

Ratings and Recommendations