BOOL

Dynamo DB's Developer Experience

Dynamo DB "is a key-value and document database" that has a lot going for it. It's cheap and it has autoscaling. It works well...except when it doesn't. At first, I thought it was super powerful, but after 2 years of using it, I will be avoiding it for everything but the most obvious use-case.

Dynamo DB was released to the public in 2012. It was created to do quick lookups by key. Say you load a product page on Amazon, you want to fetch the product record as quickly as possible. Fetching a single item is something that DynamoDB does extremely efficiently. Especially when compared to constructing the same entity from a bunch of SQL tables via a handful of joins.

If you're focusing on this type of behavior, then you might want to consider using DynamoDB. However, the further you get from this core use-case the more issues you are going to run into.

AWS Points Beginners Towards Dynamo DB

Before discussing the issues I want to point out how DynamoDB is marketed towards new AWS users. Several of AWS's beginner resources point toward Dynamo DB. On the Getting Started Resource Center, there are several references to Dynamo DB or tools that use Dynamo DB (DB tutorial, Deploying React tutorial, Lambda Tutorial). I think they see Dynamo as a good entry point for beginners because it is cheap and highly available. However, beginners are less likely to understand the limitations of DynamoDB and more likely to make mistakes using it.

The Query Language

DynamoDB's query language is written in JSON. The thought was that JSON is such a common format on the web that it would be a good choice. However a static object notation is cumbersome compared to the full language that is SQL.

Let's start with an example derived from DynamoDB's documentation, here is a basic SQL query.

SELECT Artist, Title FROM Music
WHERE Artist='No One You Know' AND SongTitle = 'Call Me Today';

This is super understandable. It even reads like English. Select a couple columns from the Music table where an artist and song title match the passed values. Excellent!

Here is the same query in DynamoDB

{
"TableName": "Music",
"KeyConditionExpression": "Artist = :a and SongTitle = :t",
"ExpressionAttributeValues": {
":a": "No One You Know",
":t": "Call Me Today"
},
"ProjectionExpression": "Artist, Title"
}

Lets set aside the fact that there are over two times the characters used in the DynamoDB query as the SQL query.

To me, this query takes longer to read. It definitely does not read like English. The "where" clause is broken into two objects: The KeyConditionExpression which is a template string and ExpressionAttributeValues which are the values that get passed into the template string. Finally, we have the ProjectionExpression which is akin to the SELECT columns statements in SQL. Projections are a part of most database queries, but it feels like a pretentious term to use in your beginner's database.

This query is hard to read, everything is labeled an "Expression", and it is intimidating to new users but that's not the only way this query falls short.

Forced Pagination

When querying a Dynamo DB the limit for the payload is 1 MB. Higher than that and you will have to deal with pagination. In scenarios where you want to fetch large amounts of data, this can be inconvenient. You will need to fetch each page synchronously, then check if there is another page and fetch it. Repeat this until some condition is passed as seen in this example.

async function loopQuery(params) {
let keepGoing = true;
let result = null;
while (keepGoing) {
let newParams = params;
if (result && result.LastEvaluatedKey) {
newParams = {
...params,
ExclusiveStartKey: result.LastEvaluatedKey,
};
}
result = await AWS.query(newParams).promise();
if (result.count > 0 || !result.LastEvaluatedKey) {
keepGoing = false;
}
}
return result;
}

const params = {
TableName: user,
IndexName: 'userOrder',
KeyConditionExpression: 'un=:n',
ExpressionAttributeValues: {
':n': {
S: name,
},
},
ConsistentRead: false,
ReturnConsumedCapacity: 'NONE',
ProjectionExpression: ALL,
};

const result = await loopQuery(params);

This is simply not an issue in a SQL database.

SELECT * FROM table;

And if you do not want all results, you can just add a limit.

SELECT * FROM table LIMIT 20;

The overly verbose queries and the forced pagination aren't the only things that make Dynamo DB a bad choice for beginners.

Limited Community

When learning a new database technology, it's useful to have a community of experts who are active and willing to help. Compared to mature languages like SQL, Dynamo DB's community is lacking. From what I've seen, Dynamo DB's bad developer experience pushes experts towards more accessible technology.

Related to this is the infamous AWS documentation. A beginner will have to grapple with unwieldy documentation of unwieldy queries, a bad combination.

So should I use it?

Dynamo DB is a great place to store data on the edge of a large infrastructure. It has low latency, it auto-scales, its cheap, and you don't have to manage any database servers. However, as a core database for a small project, it is more trouble than its worth. For the beginner database administrators to whom AWS is marketing Dynamo DB, it is a particularly bad choice.

Sources

https://www.allthingsdistributed.com/2012/01/amazon-dynamodb.html

https://en.wikipedia.org/wiki/Amazon_DynamoDB#Background

https://stackoverflow.com/questions/48823517/query-size-limits-in-dynamodb/57445396#57445396


Tags

Subscribe

* indicates required
/ ( mm / dd )