(2007-10-03) Amazon Dynamo

Werner Vogels on Amazon's internal Dynamo system, internal technology developed at Amazon to address the need for an incrementally scalable (Scaling Out), highly-available Key-Value storage system... We submitted the technology for publication in SOSP because many of the techniques used in Dynamo originate in the operating systems and distributed systems research of the past years; DHTs (Distributed Hashtable), consistent hashing, versioning, vector clocks, quorum, anti-entropy based recovery, etc. As far as I know Dynamo is the first production system to use the synthesis of all these techniques, and there are quite a few lessons learned from doing so. The paper is mainly about these lessons... There are many services on Amazon's platform that only need Primary Key access to a Data Store. For many services, such as those that provide best seller lists, shopping carts, customer preferences, session management, sales rank, and product catalog, the common pattern of using a Relational Data Base would lead to inefficiencies and limit scale and availability.

This is used behind Amazon Web Services.

Oct19 update

• Dare Obasanjo [made](http://www.25hoursaday.com/weblog/2007/10/06/Thoughts On Amazons Internal Storage System Dynamo.aspx) some interesting comments, including that BigTable also takes a Primary Key model.

• Sam Ruby confirms What do dynamo, MemCached, BerkeleyDB, and CouchDB have in common with each other, and in many ways with other structures like my hard drive or your mail or the www? Namely that everything is accessed by a Primary Key, and that Meta Data is either attached to, or embedded within, that data.