Showing posts with label amazon aws. Show all posts
Showing posts with label amazon aws. Show all posts

Jun 27, 2016

Neo4j Cluster Performance Tuning

Neo4j is one of the leading graph database these days, and it is very popular in recommendation systems, fraud detetion and social networks scenarios.

While the single instance (that is included in the community edition) performs very well (usually w/ under 10ms response time), You may face challenges in cluster mode

Why Should You Expect for Performance Degradation in Neo4j Cluster?
Two simple reasons:
  • Neo4j cluster is a Master-Slave cluster w/ an auto failover method (much like MongoDB). However, unlike MongoDB, primary node deteciton by client is done by a server side load balancer and not by the client's driver. 
  • Cluster replication is syncronious by default, unlike MongoDB async default behviour.
How Much will it Cost us?
  • The various nodes of the cluster should be behind a LB. If you select AWS ELB, it will cost you 7 to 30ms according to our measures below. The ELB latency is increased as request and response become larger (see details on the bottom). Note: impelementing a MongoDB like driver could be a great improvment and will help saving this latency and minimize system cost. and it's a great idea for a side project!
  • The nodes behind the ELB replicate changes.from master to clients. The level of syncronization is controlled by the ha.tx_push_factor parameter w/ a default value of 1. This parameter controls the number of slaves that should recieve the commit before answering the client. By setting it to 0, you avoid syncronization and get a similar result to a single node. Changing the factor wil save 70ms at average (and much more at peak time), and will leave us w/ an average 40ms per query (inc. ELB cost).
You can find the diffrences below, where in the tested environment a community edition instance was replaces by a 3 nodes cluster behind an ELB:
  • In the left section you can see an average of 9ms in the initial state (single community edition instance)
  • In the middle you can see a flactuating response time of 40ms (reads) to 300ms (writes) for a 3 nodes cluster behind ELB w./  ha.tx_push_factor parameter w/ default value 1.
  • In the right section you can see a steady 40ms for both reads and writes for or a 3 nodes cluster behind ELB w./  ha.tx_push_factor parameter w/ value set to 0 (async replication).
Neo4j performance as measured by DataDog client side metrics
Bottom Line
HA have some cost by its side. Better implmementation of the load balancing and right seleciton of syncronization model, can help you gain the needed performance

Keep Performing,


Measure from Inside the Server:
> mytime="$(time ( curl http://localhost:7474/db/manage/server/ha/master ) 2>&1 1>/dev/null )"
> echo "$mytime"
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100     4    0     4    0     0    581      0 --:--:-- --:--:-- --:--:--   666

real    0m0.006s
user    0m0.005s
sys     0m0.000s

Measure from the Application Server Direct to Master:

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100     4    0     4    0     0    988      0 --:--:-- --:--:-- --:--:--  1333

real    0m0.006s
user    0m0.000s
sys     0m0.000s

Measure from the Application Server to Master through the ELB:

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100     4    0     4    0     0    417      0 --:--:-- --:--:-- --:--:--   444

real    0m0.015s
user    0m0.000s
sys     0m0.000s

Some More Measures to Gain Data to Explore the Neo4j Performance:
  • Enable slow log query and filter server time
  • Install monitoring like Datadog in the application level:

Sep 2, 2015

Prepare for Failure in Your AWS Environment

In the cloud everything can happen. 
Actually everything will happen.

Therefore, in your design, you should be ready for failures: even if you expect your disk mounts to be there for you, they might not be. And you are doing auto scaling, it is most likely that one in a time they won't be there for you.

Therefore, to avoid hanging servers due to failure to mount disks and bad messages as the follow: "The disk drive for /tmp is not ready yet or not present", make sure your servers are not bound by your disks (otherwise you will not be able to contact your servers, or your OpsWorks will notify you that the server is booting forever).

Avoiding Waiting for Your Mount
The secret is a small option: nobootwait that will make sure your server is not waiting for the mount to be ready. You can configure it in your /etc/fstab, or even better in your Chef recipe:
mount "/tmp" do
  device "172.32.17.48:/tmp"
  fstype "nfs"
  options "rw,nobootwait"
  action [:mount, :enable]
end

Bottom Line
The right design will help you keep you system running in a cloud based environment

Keep Performing,
Moshe Kaplan

Jan 16, 2015

Offloading SSL using AWS ELB

If you are using AWS elastic load balancer to scale your system, you may find that it is a good solution to offload SSL termination from your servers.

Why Should You Offload SSL Termination?
HTTPS is an encrypted protocol, and encryption required high CPU utilization to perform the needed mathematical computations.
Since most web applications are CPU bounded, you should avoid processing SSL at your servers.

Why AWS Elastic Load Balancer (or Any other LB) Is a Great Candidate?
In order to perform load balancing, the load balancer must decrypt the traffic and read its content. This is done by placing your certificate on the load balancer.
If you consider the network between your LB and your servers to be secure, you should prefer to avoid re-encryption of the traffic, and keep it clear.

How Can I Make Sure Traffic is Actually Secured?
In some cases, you want all your users to use HTTPS as an encrypted channel in order to keep your users privacy and avoid eavesdropping and injections.
In these cases you want to catch traffic that did not use HTTPS before being terminated in the LB and redirect it to HTTPS. This can be done by evaluating by the X-Forwarded-Proto server field in your .htaccess or Apache configuration:
RewriteEngine On
RewriteCond %{HTTP:X-Forwarded-Proto} !https [NC]

RewriteRule ^ https://%{HTTP_HOST}%{REQUEST_URI} [L,R=301]

Bottom Line
A careful design can help you get more out of your web servers

Keep Performing,
Moshe Kaplan

Jun 15, 2014

Auto Scaling your Workers based on Queue Length

Having an image or video processing engine?
Having a web crawling solution?
Doing OCR on media?
If so, this post is probably for you.

The Producer-Consumer Auto Scaling Problem
Back at school, you probably learnt how to manage the queue of waiting tasks in a producer-consumer problem. You may learnt how to avoid expired tasks and double processing of same tasks.
However, back then we all assumed the number of workers and producers is fixed.

Well, it is not true any more...
If you system includes a backend processor that its load may vary based on clients demand, you may need to auto scale it. Back then, you needed to purchase new servers. These days it just about launching new instances in your favorite cloud.

Adjusting AWS Auto Scaling to Support this Pattern
AWS Auto Scaling solution is known as the best of its kind in the industry. It supports:
  1. Dynamic scaling based on instances load (e.g CPU).
  2. Automatically launching and terminating instances.
  3. Supporting both on demand (expensive) instances and Spot (marginal cost) instances.
However, it does not support complex decisions such as queue length/instances length ratio out of the box.

Fortunately, AWS provides a complete set of API that let us define a solution that its general flow is described here and here:
  1. Create a base AMI to a worker/consumer machine
    1. If your code tends to change you may define a method that will update to the latest code when machine is being launched
    2. A better solution may be using a DevOps platform such as Puppet or Chef.
  2. Define auto scaling initial configuration with an initial size of minimum and maximum number of instances (N = 1).
  3. Define a spot instance price: make it reasonable (not too high to avoid extra cost, and not too low to actually launch instances when needed).
  4. Create a cron that will run every 1 min and will check the queue length. This cron will terminate or launch instances by adjusting the auto scaling configuration minimum and maximum number of instances (N).
The Algorithm
You need to take several decisions in order to decide how many instances you want on air:
  1. What is the ratio of tasks in queue (Q) to running instances (for example R = 2)
  2. How quick do you want to reach this ratio. The quicker you do, the higher it will cost (instances that are billed hourly, may be terminated and relaunched again only few minutes after that).
Two examples for a possible algorithms:
  1. Keeping the ratio of queue length and running instances constant: for example: N = max[1, Q/R]
  2. Soften the ratio by using previous calculated number N': N = (N' + max[1, Q/R])/2
Bottom Line
Auto Scaling is an amazing tool, and by adjusting it we can easily solve complex issues.

Keep Performing,

Mar 6, 2012

Amazon Latest Price Reduction: News for Large Accounts and Linux based Users

Amazon announced today a price reduction on the EC2 instances prices. The price reduction does not include storage or traffic so you may expect that effective cost recduction numbers will be half on your montly bill.


On Demand Pricing Effects
Minor price reduction on the on demand instances: 5% on small instances, 10% on larger instances. Price Reduction Target: Traditional hosting offers.


Reserved Instances Pricing Effects
Major price reduction on the reserved instances: 20% effective reduction. Price Reduction Target: Traditional hosting offers.


Volume discounts
Finally, Amazon offers volume discounts. The volume discounts are focused on large accounts that spent at least $1M annually at AWS services. The volume discount are focused on the the reserved instances (the equivalent of traditional hosting). You can expect 10% reduction (5% off bill's bottom line) for $250K upfront bill ($1M annual AWS bill) and 20% reduction (10% of bill's bottom line) for $2M upfront bill ($8M annual AWS bill). Price Reduction Target: Traditional hosting offers.


Linux or Windows?
The price reduction increases the price differences between Windows based instances and Linux based. Price Reduction Target: Avoid price wars with Microsoft.


Why Amazon Made These Changes?
The price reduction signals that Amazon has a larger price pressure from Linux cloud providers rather than from Microsoft. It also signals that traditional hosting providers offers focus on large customers and larger servers accounts. These price reductions will help Amazon reduce larger customers migration out of its services.

Keep Performing,

Oct 30, 2011

Adding One More MySQL Slave to Cloud Based System (Amazon AWS)

MySQL Replication tends to be complex.

Creating a new slave to support your system expansion is not easier: coping large data files over the LAN, recovering replication, etc.

Fortunately, virtualization and SAN help us do it easier and faster.

Lucky us, Cloud Computing make it even easier.

Adding a New Slave is Easier than Ever
  1. Select an existing slave that can be reboot w/o affecting the service.
    1. Take the slave out of the production environment.
    2. Stop the MySQL slave and perform a read lock: SLAVE STOP;FLUSH TABLES WITH READ LOCK;
    3. Create an AMI based on this instance using AWS console or API.
    4. Recover the instance upon AMI creation completion and get the slave back to production: UNLOCK TABLES;SLAVE START;SHOW SLAVE STATUS\G
  2. Create a new slave based on the AMI
    1. Launch a new slave using the AWS console or API.
    2. Change the server's /etc/my.cnf file to fit a new slave instance:
      1. Change the server-id=N value to a new numeric value.
      2. Set relay-log='server-relay-bin' to the original slave server relay log name. (if you want to make it even easier, configure in the first slave's my.cnf relay-log=mysqld-relay-bin, and you'll be able to avoid this step).
    3. Restart the MySQL instance: sudo /etc/init.d/mysql restart
    4. Recover the MySQL: UNLOCK TABLES;SLAVE START;SHOW SLAVE STATUS\G
    5. Add this new slave to the production environment.
Bottom Line
Expanding your business today is easier w/o the need for file copy or any other tedious action.

Keep Performing,

Oct 3, 2011

MySQL CentOS and Amazon Web Services (AWS)

Many Linux experts prefer CentOS. The reason is clear, Redhat's RHEL is probably the best Linux server distribution and CentOS is very much a like (except for the yearly fees).
This distribution (and Fedora as well) is very common for MySQL installation, since there are many predefined packages for it. This is unlike other distributions such as Ubuntu.

Therefore, when you will perform your database migration to the cloud, I recommend that you will first verify your needs. The following items will help you implement your systems right:

  1. Select the Linux flavor
    1. CentOS: Detailed options are available bellow. Don't forget to verify the recent CentOS version: Usually there are two (an early major version such as 6.0 and an old advanced minor version such as 5.7) and you should select the right version for you.
    2. Fedora: Some free and good AMIs are available.
    3. RHEL: There is a premium AMI for it
    4. Ubuntu: No formal MySQL distribution exists to this flavor.
    5. Debian.
  2. Select the Storage Type. Amazon AMIs can store their information on one of two storage type:
    1. Instance storage: this is a server attached disks storage that is great from plain performance issues (no network traffic and latency for storage). Yet, it lacks the option to extend the disk seamlessly to a larger size and backup must be done through the network instead on an instant disk duplication (shorter downtime).
    2. EBS: this is Storage Area Network based disks that easily can be extended and backed up with minimal server downtime. Yet, this storage costs both in money (~$0.15/GB per month) and network latency.
  3. Select the right image for you:
    1. RightScale Image: (last update Jan 2009. If you don't use RightScale, don't forget to remove their management tools and don't forget to update your version to a recent one. RightScale has the two types of images.
    2. Amazon Linux AMI: The AWS image is based on CentOS image, yet it includes some tweaks. The plus is having an Out of the Box solution with Amazon support. The minus is that it's not a 100% RHEL compatible anymore.
  4. Install your database (MySQL probably) and web server (Apache probably) on the machine.
    1. The recommended MySQL version is MySQL 5.5 with major performance enhancement since MySQL 5.1.
Bottom Line
Now, you just need is to choose the configuration that most fit your needs.

Keep Performing,
Moshe Kaplan

Moshe Kaplan is an entrepreneur and a cloud systems consultant

Mar 22, 2011

MySQL DRP Done Right

When your systems are up and running and money is getting into the bank, it is the best time to ask yourself: What if my system fails? What if my data center (yes, even your cloud one) goes down?


The Instant DRP Solution
The instant solution is preparing a backup data center. Then you should make sure that the web and application servers will be ready to take over when needed. Yet, what about taking care of your data? How will the database be managed across two different data centers?


Many cloud, web, mobile and service providers base their systems on MySQL. Therefore, this post is dedicated to best MySQL DRP strategies..


Fail and Then Manually Recover
The most simple and easy to implement method is the Master-Slave schema. This method supports:
  1. "Master": A single database server that serves for write and read.
  2. "Slaves" One or more servers that serves for read only.
When the master fails, you can promote one of the slaves to become the master.
This method can also be used to reduce stress on the master and enable various partitioning modes.


Fail and Recover Automatically
A more complex way to implement, but faster to recover (if nothing goes wrong) is implementing the Master-Master schema. In this schema both servers serve as masters and recover when one fails. Please notice that this method is a little bit tricky and you should avoid conflicts since the conflict resolution is poor:

  1. You should need to either avoid auto increment fields or make sure that only one server is an active master at a time.
  2. You should avoid same record modified by both parties.

Do It the Cloud Way
Cloud providers propose these days cloud based database (PaaS). These solutions remove the database management headache from your task list. Amazon for example provides the Rational Database Service (Amazon RDS). RDS provides a high available managed MySQL instance that supports instant recovery to any point in time. This service is more expensive than having your own MySQL instance, but if you are a medium shop, it may be the best offer for you.


Bottom Line
Select your MySQL DRP method based on your resources, recovery needs and the effort you willing to allocate. Choose Master-Slave for novice, Master-Master for experts and the cloud way to avoid the headache.


Keep Performing,
Moshe Kaplan  Follow MosheKaplan on Twitter


P.S feel free to contact for DRP and MySQL consulting

Jul 25, 2010

Open is the New Black (hmm.... Standard)

Click on the post title to read the full post and the comments.

Only few years ago when a technological company was lagging behind in a new niche that was turned into a mainstream, it had a magic move: creating a new standard.

Standard: The Magic Touch
Many times the standard technology was less efficient, less robust and with less users than the technology that was advanced by the market leader. Yet, it had a magic charm: it was open, and most the players in the market could use it without royalties or other fees. That was including #2, #3 in the market as well as many other giants that were not focused in that market, but had to integrate and expand their product support to that technology.
It worked for DEC with Ethernet (vs. IBM's Token Ring); it was great for NetApp that chose iSCSI with 3% market share vs. EMC leading Fiber Channel protocol and it was like a charm for a small silicon valley firm named CISCO that chosen IP.

How do you Standardize Software?
Let say you are a software company that is lagging behind. Anyone said Yahoo! vs. Google in the internet market? Yahoo! had a great idea to scale out using a BigTable like product, but had no deep pockets for that.
Another example? Facebook? What if you had the largest image store publicly available, yet you have only few hundred developers and you need a new scalable database. What do you do?

Open: The New Magic Touch
Well, both Yahoo! and Facebook chose Open Source their infrastructure products creating two of the leading infrastructure products in todays world: Apache Hadoop and Apache Cassandra.

And Cloud, What about the Cloud?
Well it seems that last week Rackspace move: establishing OpenStack.org and contributing major parts of its propriety cloud infrastructure code to this project, is a clear say that being #2 in the IaaS market (see Jack of all Clouds latest report) that is being invaded by major players such as VMware, Google, Salesforce.com and Microsoft is not an easy task even to such a major player in the old hosting business. (If you want to know more regarding this move, see Geva Perry's analysis).

Bottom Line
It seems that the cloud market and mostly the IaaS one is getting to their mainstream phase, where scale is a top factor and community is a major factor in survival.

Would you like to share you opinion? Can you add more information? Feel free to comment!

Keep Performing,
Moshe Kaplan
Follow MosheKaplan on Twitter

Jun 2, 2010

How Much Money did Amazon Make from that Post?

Click on the title to view the full post
My mate, Dima Shestak, just noticed me regarding a new phenomena in Amazon AWS spot prices in the last few days. The prices are going up...
Is there any connection to my last post: "Solving the Spot Instances Pricing Paradox"? Only time will tell...

Update: a recent graph (June 19, 2010) shows that the trend continues and prices are even touching the on demand prices:


Keep Performing,
Moshe Kaplan Follow MosheKaplan on Twitter

May 26, 2010

Solving the Spot Instances Pricing Paradox

Click on the title to view the full post
Geva Perry, from Thinking out Cloud, had a great post last December regarding AWS spot instances prices (bidding prices where charge is done based on market current pricing) and why the spot instances pricing should not be higher than on demand instances. The post had a comment thread by James Watters, Guy Rosen, Shlomo Swidler and me discussing why spot instances pricing should not surpass the on demand instances pricing, and why this assumption was broken (for relatively short spikes) in the reality.

NOTE: Please find my previous post regarding spot instances for more information.

May 21, 2010

When Agile Met Cloud Computing

My mate Alon Nativ just referred me to JetBrains newly introduced TeamCity feature: integration of their Continues Integration (CI) platform with Amazon EC2. This new feature enables on demand instances during the CI process.
What is it good for?
CI tends to have non linear usage behavior: Minor use in the beginning of the project, and massive use in the end of the day and just before releasing the new version.
During these peak hours TeamCity can launch on demand instances, running the build and the tests and shut down the instances when it ends.
 
Bottom Line
It is a fine example of how development teams can keep their infrastructure costs as low as possible, while maintaining maximum flexibility.

Keep Performing,
Moshe Kaplan

Apr 30, 2010

Issuu: Erlang, Hadoop and AWS Become Mainstream

There is a buzz in the industry regarding how the Erlang, Hadoop and AWS stack enables sites scalability and how it lets them handling the demand when you hit the buzz (TechCrunch, Dig...).
An example for that is Issuu.com, a content publishing platform that is using this stack to enable millions of publications in the web and the mobiles (Android and iPhone). Tania Anderson describes this case study in a Danish article (use Google Translate for it).

In that article Issuu's CEO Michael Hansen reveal several key issues in their strategy:
  1. How they managed to recruit Erlang programmers (hired talented programmers and learned them from scratch)
  2. Why they favored Amazon Web Services (Microsoft Azure hardly fits the internet mainstream stacks and Google App Engine is focused on the front end while Issue needs back end processing as well), 
  3. How could they avoid AWS down times (They did not. However, most of the industry was effected, so it was a non issue).

Keep Performing,
Moshe Kaplan 

Apr 7, 2010

Amazon Pushed the Bar. Again!

Everybody is talking these days about iPad, Windows Phone 7, iPhone OS4 and Nexus One and how they gonna change the world. Well if everybody is going to change the world, they need reliable servers and more important traffic efficient communications that will enable mobile application keep updated while consuming as few as possible battery and network resources. If you look for the right buzz Google for COMET, Push and Long Polling.
Current State
Well to be honest, the COMET Servers market state is not so good. Most of the products in the market are either young or luck important features such as high availability. However, I'll keep the full details of the market state to survey that will be exposed in the next few days.
The Amazon Move
Amazon detected this new hot market and exposed today SNS: Simple Notification Service. This service provides you the ability to push data to your clients without taking care of establishing the server infrastructure and taking care of HA.
As a first impression I can say the pricing is very attractive (100K first messages are free) and even the beta limit of 100 topics (channels) is valid for many application (yes I know, there are many of us who needs XXXK :-)
UPDATE: at a second impression this product (SNS) still do not support the need for a push.
The bottom line
If you are a mobile developer, take a look at this new service. It can save you a lot of TTM.

UPDATE and CLARIFICATION
The Amazon SNS API includes the following API:
  1. Topics: Create, Delete and List channels and Set and Get their attributes.
  2. Topics Permissions: Add and Remove permissions on the channel to AWS accounts
  3. Subscription: Subscribe and Unsubscribe to a channel and subscription confirmation. Subscription is done to URL (http and https), Email account and SQS. Subscription cannot be done using COMET method or other methods that are feasible for mobile subscriber (unless you gonna poll your email account)
  4. Publish: Publish a new info item into the channel.
What is missing? the push mechanism that can support mobile and web clients and notify them regarding new info items.

The Real Bottom Line
Read the full Spec and the API before posting to your blog :-)

Keep Performing,
Moshe Kaplan

Feb 9, 2010

Blocked Sessions In the Cloud

When you test you new software (or feature) in a new environment (e.g installing your system in cloud environment) you may face errors when you'll try to connect your newly deployed service. What happened?
There are two options:
  1. You did not install correctly your system. You can verify it by connecting the service from within the server (using Terminal Services in Windows case). If it fails you should start exploring the event log and your application log.
  2. Someone is blocking your sessions (probably it is a firewall). You can verify it by running netstat -na from you client command line, and check for SYN_SENT lines in the output. If it's attached to the server IP and port that your service uses, you definitely have firewall in you way. There are several options who is the blocker and how to solve it:
    1. Your computer personal firewall. Probability: Low; Verification: try connecting from another computer.
    2. Your company firewall. Probability: Medium-Low; Verification: try connecting from another computer which is outside your company network (your favorite neighborhood cafe can be great).
    3. Your cloud provider firewall. Probability: High; Verification: if it's Amazon AWS, login to the AWS Management Console, verify the Security Group that your instance is linked to, and verify the Security Group rules.
    4. Your server firewall: Probability: High (if you are using Windows); Verification: check the server firewall configuration that it allows incoming connections in the relevant ports.
Keep Performing
Moshe Kaplan

Dec 19, 2009

Load Balacing Support in Dynamic Environments

Hi,

The Mission
These days we face a challenging task: designing a very large system of scalable instances. Each of these instances may be in a different geographic location, and many of them are on demand instances that are  being started and shutdown instantly.
Another requirement in this system that a given client will be direct to a defined instance, due to system restriction (a round robin is not an option is this case).

One Step Further
Since the number of IP addresses in the internet is limited, we would like to use as few as possible Public IP addresses. This can be done using a load balancer or a proxy.
In the current state we would like to avoid using hardware load balancers in order to keep initial fixed costs minimal, but we may consider to use them in the future.

Is Amazon Cloud Load Balancer Service (AWS) is an Option?

AWS EC2 instances is a feasible option for on demand instances hosing. However, AWS charges $0.025 per a single load balancing rule per hour (+traffic). Therefore, it can be used, but for a large number of rules (>7) or high traffic, better solutions can be found in the market.

So What Can Be Done?
We left with software load balancers. The major ones Apache mod_proxy and HAProxy. Supporting large number of instances behind the load balancer can be done in one of the following two options:
  1. Pre register a large number of DNS addresses (sub domain) and associate them with the load balacner IP. The load balancer will simply redirect the request to the defined instance, based on the a simple rule in the load balancer. For example: http://instN.foofarm.com  Pros: simple. Cons: not fully dynamic, requires additional DNS registration once in a while to keep up with the application growth.
  2. Performing ProxyPass in the Load balancer: Every request will include in its path an instance identification for example: http://foofarm.com/instN/. This method does not require mass DNS declarations, but it requires specific definitions in the load balancers that may be more CPU consuming. In Apache the definition is pretty trivial, however, this product is less scalable from HAProxy. In HAProxy the task can be done as well based on a two phases: switching to the server and rewrite the URI:
    In order to switch to the server, you have to use ACLs to match the path,
    then a use_backend directive to select a server farm ("backend"). Your
    farm may very well support only one server if you want.

    Then in this "backend", you can use a rewrite rule ("reqrep") to replace
    the request line.

    This would basically look like this :

    frontend xxx
           acl path_mirror_foo path_beg /mirror/foo/
           use_backend bk_66 if path_mirror_foo

    backend bk_66
           reqrep ^([^: ]*\ )/instN/\(.*\)  \1/\2
           balance roundrobin
           server srv66 192.88.77.66:80

However, Willy Tarreau, HAProxy author who kindly provided this hint for me, recommends that you avoid the second part (rewriting) because :

 1) it requires good regex skills which sometimes makes the configs hard
    to maintain for other people

 2) rewriting URIs in applications is the worst ever thing to do, because
    they never know where they are mapped, and regularly emit wrong links
    and wrong Location headers during redirections.

Willy Tarreau also advices that the best thing to do clearly is to correctly configure your application to be able to respond with the real, original URI. Remapping it can be used
as a transitional setup in order to ease a graceful switchover, though. Bottom line: Pros: No DNS configuration and fully scalable solution, with no dependence on DNS replication. Cons: CPU Consuming and error prone declarations.

So, What to Choose?
The answer is based on your needs, and your believe in your people regex capabilities. We made our choice.

Keep Performing,
Moshe Kaplan. Performance Expert.

Nov 26, 2009

What if I have been in NYC, London and SF in the same day?

When you analyzing a website performance, it's always a mystery what is the performance issue root cause (well at least till we dive in and reveal it :-).

A common issue in this analysis is what is the network round trip time (RTT) in the play, and who is to blame for. Usually we start with following measures:
  1. Analyze the website from our offices. This test includes both Wireshark analysis to analyze the network RTT and FireBug to understand website behavior.
  2. Analyze the website performance in the hosting environment by running a terminal services on a server (or the appropriate measure in Linux or UNIX based environment), were RTT should be zero.
  3. Finally, we usually connect the site from another site around the world. This is done usually by connecting a server in another hosting environment (booting a server for a hour using Amazon AWS is a great solution for that).


    These tests are usually a good method in order to evaluate the communication layer play in the performance issue. However, testing from remote sites is not always accurate and sufficient. In these cases we recommend using a service name browsermob.com, that provides you a FireBug like analysis from several sites around the world.  

    Keep Performing!
    Moshe Kaplan. 9AFW6C8B3GKN

    Oct 23, 2009

    Service: Hoopoe - GPU Cloud


    I posted several notes in the last year regarding the niche market of GPU cloud services and why current cloud computing providers such as Amazon AWS that are based on the XEN and VMware virtual instances (hypervisor based) cannot fit them.

    Hoopoe is a new service that is focused on this niche market and provides GPU services in the cloud based on NVIDIA TESLA CUDA devices. This service is integrated with Amazon S3.

    Hoopoe will be presented by Mordechai Butrashvily in the next IGT GPGPU Meeting at Nov 10, 2009 14:30-17:00.

    To reserve your place, send your contact details to info at grid dot org dot il

    Keep Performing,
    Moshe Kaplan. RockeTier. The Performance Experts.

    Mar 9, 2009

    Amazon Session Highlights

    Hi,
    As you may read we participated last week in the AWS session hosted by the IGT and SUN.

    Simone Brunozzi, Amazon Web Services Evangelist, was great and he gave several interesting notes including:
    1. Amazon will provide load balancing capability in the near future (you can avoid this issue right now using HAProxy, but it always great to get things out of the box).
    2. Amazon support availability zones: Amazon has several DCs in the US. Each availability zone is in another DC, so can provide multi site architecture in day 1.
    3. Amazon suffered for 12 hours downtime in the last 3 years. The longest was about 2 hours. This numbers are for intended and unintended downtime. It gives us about 99.95%, which a very remarkable number for such a scale and the given cost.


    Simone presented the Animoto case study: "from 80 servers to 3500 in 3 days" as well.

    Keep Performing,
    Moshe Kaplan. RockeTier. The Performance Experts.

    Feb 23, 2009

    UPDATE 2: IGT Hosting Amazon AWS Hands-on workshop will start at 18:00

    Hi,

    Great News,
    After all the Amazon meetups, the IGT is going to host Amazon AWS Hands-on workshop,
    It's a great opportunity to meet Simone Brunozzi, Amazon Web Services Evangelist - Europe, and have real life hands on experience as well as asking questions.

    Date: Mar 3, 2009 10:00 13:00
    Location: SUN Offices, Manofim 9, 8th Floor, Hertzelia, Israel
    Organizer: IGT

    The preliminary agenda:
    18:00- 18:30 Reception
    18:30 - 19:00 Intro to Amazon AWS
    19:00 - 20:30 Hands-on AWS Workshop
    Including:
    Account management
    S3 – details and examples, using Firefox S3 organizer
    EC2 – details and examples, Linux and Windows, using the AWS Console
    Cloudfront – details and examples, using Firefox S3 organizer and/or other tools
    20:30 - 21:00 AWS Q&A

    Best,
    Moshe Kaplan. RockeTier. The Performance Experts.

    ShareThis

    Intense Debate Comments

    Ratings and Recommendations