Microsoft is Catching Up with the Internet Industry

Click on the title to view the full post, comments and add your own comment
For a long time I blame Microsoft for not really understanding the Internet and cloud Industry basic need: very large systems that can handle billions of daily transactions with a very low revenue per transaction. Now, after admitting its failure in the mobile industry (see Kin's death earlier this week), Microsoft finally reveals AppFabric after a very long time (I would like to thank my colleague Alon Biran for notifying me regarding it).

What is AppFabric?
The roots of AppFabric are located in the "Velocity" project that was developed in the SQL Server group (a year and a half ago I wrote a Knol about). This project was aimed to provide a key-value store (super hash) that will be used for two main objectives:

1. Instant in memory storage data store (database replacement).
2. Shared web session repository that will enable instant failure between web servers (the only Microsoft in house solution till now was using SQL Server as a shared repository)

After being in various CTP and beta versions for almost two years, AppFabric is finally out as part of the Windows platform rather than being a part of the SQL Server product. You may take a look at my colleague David Chappell's AppFabric white paper for complete set of features and scenarios.

Who are the Current Players?
The major players in the market these days are open source (Every each of the products has its own unique features, please feel free to comment on this post regarding it):

1. Memcached: the #1 product in the market. Its roots are in 2003. It is an open source and Linux based product. Yet, Memcached has .Net client API as well as windows porting of the product itself.
2. SharedCache: a native C# open source product that is a Memcached like.
3. StateServer: commercial native windows product by ScaleOut Software.
4. NCache Express: another commercial native windows product by Alachisoft.

Since AppFabric is provided for "free" as part of the Windows platform, the commercial native .Net products may be hurt from the Microsoft move. However, they may gain more users due to better publicity for this kind of solutions in the Microsoft development community,

Why Did It Take So Long?
It seems that Microsoft had a major cannibalization issue: If you provide a good key-value store then you may reduce the databases size and usage intensity. Thus, the number of high cost database licenses will be significant lower and the bottom line will be smaller. Oracle has the same problem with Coherence that is now part of the Fusion platform.
Taking this product into RTM is a true Microsoft understanding that if it really want to take a significant role in the Internet and cloud industry, it will have to sacrifice more than just few cents in its financial reports bottom line.

Bottom Line
These are very good news that Microsoft is finally provide a solution for a basic need of the Internet and cloud industry. However, while Microsoft was catching up with the Industry 2003 news, new requirements were raised including MapReduce, BigTable and other NoSQL advanced features. How much time will it take this time? Probably only Steve Ballmer has the answer.

Keep Performing,
Moshe Kaplan

Shared Cache: The Windows and .Net memcached

Hi again,

In memory databases and cache are must in any large scale system these days (you can ask Facebook architects regarding it). The Linux guys have memcached and there are many other products (with much more features of course) such as Gigaspaces XAP, Oracle Coherence and Scaleout. However, since Microsoft did not release yet the long expected Velocity product (will it ever be in production?), it seems there is a clear winner to the Windows environment: SharedCache. This product is some kind of a memcached porting to native managed .Net code and it is licensed under LGPL!.
SharedCached is already in production in several major sites including theport.com and it is fully documented. The product supports all major requirement from a caching product including:

1. Partitioning - just like sharding, every server is taking care of part of the data, and the application server are talking with every cache server (unless of course you were wise enough to shard the application servers as well).
2. Replicated Caching - keeping data in multiple instances, making sure data will always be available and lots of reads will be handled correctly.

I recommend downloading and testing this product and see if it fits your product/service needs,

Keep Performing,
Moshe Kaplan. RockeTier. The Performance Experts.