View source for Dynamo (storage system)

{{Short description|Cloud-based service}}
'''Dynamo''' is a set of techniques that together can form a [[High availability|highly available]] key-value [[Structured storage|structured storage]] system<ref name="decandia-2007" /> or a [[Distributed data store|distributed data store]].<ref name="decandia-2007">{{Cite book | last1 = Decandia | first1 = G. | last2 = Hastorun | first2 = D. | last3 = Jampani | first3 = M. | last4 = Kakulapati | first4 = G. | last5 = Lakshman | first5 = A. | last6 = Pilchin | first6 = A. | last7 = Sivasubramanian | first7 = S. | last8 = Vosshall | first8 = P. | last9 = Vogels | first9 = W. | chapter = Dynamo: Amazon's Highly Available Key-value Store   | doi = 10.1145/1294261.1294281 | title = Proceedings of twenty-first ACM SIGOPS symposium on Operating systems principles - SOSP '07 | pages = 205 | year = 2007 | isbn = 9781595935915 | s2cid = 221033483 }}</ref> It has properties of both [[Database|database]]s and [[Distributed hash table|distributed hash table]]s (DHTs). It was created to help address some scalability issues that [[Company:Amazon|Amazon]] experienced during the holiday season of 2004.<ref name="readwrite-2012">[http://readwrite.com/2012/01/18/amazon-enters-the-nosql-market Amazon Takes Another Pass at NoSQL with DynamoDB]</ref>  By 2007, it was used in [[Company:Amazon Web Services|Amazon Web Services]], such as its [[Amazon S3|Simple Storage Service]] (S3).<ref name="decandia-2007" />

== Relationship to DynamoDB ==

[[Software:Amazon DynamoDB|Amazon DynamoDB]] is "built on the principles of Dynamo"<ref>[http://www.allthingsdistributed.com/2012/01/amazon-dynamodb.html Amazon DynamoDB – a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications]</ref> and is a hosted service within the AWS infrastructure. However, while Dynamo is based on leaderless replication, DynamoDB uses single-leader replication.<ref>{{cite book |last1=Kleppmann |first1=Martin |title=Designing Data-Intensive Applications |date=April 2, 2017 |publisher=O'Reilly Media |isbn=978-1449373320 |page=177 |edition=1 |quote=Dynamo is not available to users outside of Amazon. Confusingly, AWS offers a hosted database product called DynamoDB, which uses a completely different architecture: it is based on single-leader replication.}}</ref>

== Principles  ==

* Incremental scalability: Dynamo should be able to scale out one storage host (or “node”) at a time, with minimal impact on both operators of the system and the system itself.
* Symmetry: Every node in Dynamo should have the same set of responsibilities as its peers; there should be no distinguished node or nodes that take special roles or extra set of responsibilities.
* Decentralization: An extension of symmetry, the design should favor decentralized peer-to-peer techniques over centralized control.
* Heterogeneity: The system should be able to exploit heterogeneity in the infrastructure it runs on. For example, the work distribution must be proportional to the capabilities of the individual servers. This is essential in adding new nodes with higher capacity without having to upgrade all hosts at once.

== Techniques ==

{| class="wikitable"
|-
! Problem !! Technique !! Advantage
|-
| Dataset partitioning || [[Consistent hashing|Consistent Hashing]] || Incremental, possibly linear scalability in proportion to the number of collaborating nodes.
|-
| Highly available writes || Vector Clock or Dotted-Version-Vector Sets, reconciliation during reads || Version size is decoupled from update rates.
|-
| Handling temporary failures || Sloppy Quorum and Hinted Handoff || Provides high availability and durability guarantee when some of the replicas are not available.
|-
| Recovering from permanent failures || Anti-entropy using Merkle tree || Can be used to identify differences between replica owners and synchronize divergent replicas pro-actively.
|-
| Membership and failure detection || [[Gossip protocol|Gossip-based membership protocol]] and failure detection || Avoids having a centralized registry for storing membership and node liveness information, preserving symmetry.
|}

== Implementations ==

Amazon published the paper on Dynamo, but never released its implementation.  The index layer of [[Amazon S3]] implements and extends many core features of Dynamo.  Since then, several implementations have been created based on the paper. The paper also inspired many other [[NoSQL]] database implementations, such as [[Software:Apache Cassandra|Apache Cassandra]], Project Voldemort and [[Software:Riak|Riak]].<ref name="readwrite-2012" />

==See also==
* [[Distributed data store]]
* [[NoSQL]]
* [[Structured storage]]

==References==
<references/>

== External links==
* [http://www.allthingsdistributed.com/2007/10/amazons_dynamo.html Amazon's Dynamo] (2007)
* [https://arstechnica.com/old/content/2007/10/amazon-reveals-its-distributed-storage-dynamo.ars Amazon reveals its distributed storage: Dynamo] (2007)


{{DEFAULTSORT:Dynamo (Storage System)}}
[[Category:Distributed data stores]]
[[Category:NoSQL]]

{{Sourceattribution|Dynamo (storage system)}}