Hi,
I received from Shachar Zehavi, our new director of R&D, a link to Chris Kasten's presentation regarding eBay
In this presentation Chris presents a simple but very clever solution to support 4 billion events per day on 25 commodity servers. Why instead of using complex in memory databases solutions? not using MySQL as the ultimate grid solution?
What are the key components in eBay solution:
1. In memory database: implementation using MySQL in memory engine
2. HA: implemented using MySQL replication between two different MySQL in memory databases
3. Persistence: implementation using batch process once in 5min. In this case the store is done using InnoDB one.
4. Scalability: can b achieved using horizontal sharding
The number described in this presentation (4 billion requests per day using 25 machines) remind me another presenation of Paul Strong, distinguished research scientist from eBay. This presensation at the IGT2008 - The World Summit of Cloud Computing described eBay numbers and challenges including: 150 Billion request per day which is about 200K requests per second, which is a remarkable number.
Keep Performing,
Moshe Kaplan. RockeTier. The Performance Experts.
I received from Shachar Zehavi, our new director of R&D, a link to Chris Kasten's presentation regarding eBay
In this presentation Chris presents a simple but very clever solution to support 4 billion events per day on 25 commodity servers. Why instead of using complex in memory databases solutions? not using MySQL as the ultimate grid solution?
What are the key components in eBay solution:
1. In memory database: implementation using MySQL in memory engine
2. HA: implemented using MySQL replication between two different MySQL in memory databases
3. Persistence: implementation using batch process once in 5min. In this case the store is done using InnoDB one.
4. Scalability: can b achieved using horizontal sharding
The number described in this presentation (4 billion requests per day using 25 machines) remind me another presenation of Paul Strong, distinguished research scientist from eBay. This presensation at the IGT2008 - The World Summit of Cloud Computing described eBay numbers and challenges including: 150 Billion request per day which is about 200K requests per second, which is a remarkable number.
Keep Performing,
Moshe Kaplan. RockeTier. The Performance Experts.
2 comments:
This solution seems to heavily relay on caching policy.
A good caching policy will keep most of the DB requests in memory.
Where as a bad one will have to read from persisted data all the time.
Do you know what caching policy they use?
Is a MRU cache enough or something more clever is needed?
Also - do you have any info on what response times do they get?
Hi Yuval,
To be honest, most of the cache and in memory databases solution are based somehow on a some kind of caching policy. The only exception are limited size tables which can be loaded to memory.
I sent Chris a private email and asked him to place his comments, but I will try to bring my thoughts:
1. Caching policy: probably they used LRU, since usually users that just log in to the system will be most active.
2. Response Times: we can estimate them from eBay performance as regular users
Keep Performing,
Moshe Kaplan. RockeTier. The Performance Experts
Post a Comment