Simple item versioning with DynamoDB

2018-03-26 Development No Comments 4 minute read

Occasionally you want to store information in a database with keeping versioning, so you will be able to retrieve previous versions of the record.
In this example, we will use AWS DynamoDB and take advantage of some of its features.

For the example, lets take a simple record that contains Id, Name and Email. For keeping the versions we need to add two more items to each record: version and creationDate. The version will be an auto-generated uuid and the creation date will be the date/time of the entry creation.
For the database schema we will use the Id as the HASH key and the creationDate as the RANGE key.
In order to be able to get the latest version, we should create an LSI (Local Secondary Index) with the creationDate as the RANGE key (LSI must have the same HASH key as the table schema).

The table creation should look like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

public void createTable() {
final CreateTableRequest createTableRequest = new CreateTableRequest()
.withTableName("ExampleTable")
.withAttributeDefinitions(Arrays.asList(
new AttributeDefinition("id", "S"),
new AttributeDefinition("version", "S"),
new AttributeDefinition("creationDate", "S")))
.withKeySchema(Arrays.asList(
new KeySchemaElement("id", KeyType.HASH),
new KeySchemaElement("version", KeyType.RANGE)))
.withLocalSecondaryIndexes(new LocalSecondaryIndex()
.withIndexName("SortedByCreationDate")
.withKeySchema(Arrays.asList(
new KeySchemaElement("id", KeyType.HASH),
new KeySchemaElement("creationDate", KeyType.RANGE)))
.withProjection(new Projection().withProjectionType(ProjectionType.ALL)))
.withProvisionedThroughput(new ProvisionedThroughput()
.withReadCapacityUnits(10L)
.withWriteCapacityUnits(10L));

try {
client.createTable(createTableRequest);
}
catch(final ResourceInUseException e) {
// Table already exists. Ignore this exception.
}
catch(final Exception e) {
throw new RuntimeException("Failed creating table ExampleTable: "+ e.getMessage(), e);
}
}

As a good practice, I would like to start with writing some simple tests before making the actual implementation. This technique is called TDD and can help us finalizing the requirements before writing the actual code.

For readability, we’ll create a descriptor object:

1
2
3
4
5
6
7
8
9
10

@Value
@Builder
@EqualsAndHashCode
public class Descriptor {
String id;
String name;
String email;
String version;
Instant creationDate;
}

Let’s start with defining our interface:

1
2
3
4
5
6
7
8
9
10
11
12
13
14

// Returns the created object
public Descriptor addEntry(String id, String name, String email) {
throw new NotImplementedException();
}

// Returns null if record does not exists
public Descriptor getSpecific(String id, String version) {
throw new NotImplementedException();
}

// Returns null if record does not exists
public Descriptor getLatest(String id) {
throw new NotImplementedException();
}

For simulating a DynamoDB for unit tests usage, we can simply use the Embedded client that is provided as part of the SDK.
Now we can start writing some unit tests:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73

public class ExampleTest {
private AmazonDynamoDB client;
private Example example;

@Before
public void setup() {
// Create DynamoDB client
client = DynamoDBEmbedded.create().amazonDynamoDB();
// Create the Example instance
example = new Example(client);
// Create the table in the database
example.createTable();
}

@Test
public void addNewEntry_ValidateReturnValue() {
Instant startTime = Instant.now();

String id = randomString();
String name = randomString();
String email = randomString();

Descriptor entry = example.addEntry(id, name, email);
assertEquals(id, entry.getId());
assertEquals(name, entry.getName());
assertEquals(email, entry.getEmail());

assertNotNull(entry.getVersion());
assertTrue(entry.getCreationDate().isAfter(startTime));
}

@Test
public void addNewEntry_getLatest() {
Descriptor expected = example.addEntry(randomString(), randomString(), randomString());
Descriptor actual = example.getLatest(expected.getId());
assertEquals(expected, actual);
}

@Test
public void addNewEntry_getSpecific() {
Descriptor expected = example.addEntry(randomString(), randomString(), randomString());
Descriptor actual = example.getSpecific(expected.getId(), expected.getVersion());
assertEquals(expected, actual);
}

@Test
public void addNumberOfEntries_getLatest() {
Descriptor expected = null;
for(int i = 0; i < 3; ++i) {
expected = example.addEntry("my-id", randomString(), randomString());
}

Descriptor actual = example.getLatest(expected.getId());
assertEquals(expected, actual);
}

@Test
public void addNumberOfEntries_getSpecific() {
List entries = new ArrayList();
for(int i = 0; i < 3; ++i) {
entries.add(example.addEntry("my-id", randomString(), randomString()));
}

for(Descriptor expected : entries) {
Descriptor actual = example.getSpecific("my-id", expected.getVersion());
assertEquals(expected, actual);
}
}

private static String randomString() {
return UUID.randomUUID().toString();
}
}

After defining the interface, thinking about the test cases and writing the unit tests, we can start coding the real thing.
The addEntry and getSpecific are really simple and are implemented by PutItem request and GetItem request:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68

public Descriptor addEntry(String id, String name, String email) {
// Convert the arguments into a descriptor object
final Descriptor entry = Descriptor.builder()
.id(id)
.name(name)
.email(email)
.version(UUID.randomUUID().toString()) // Generate a uuid
.creationDate(Instant.now())
.build();

// Build the put item request
final PutItemRequest request = new PutItemRequest()
.withTableName("ExampleTable")
.withItem(toDDB(entry));

// Add the item to the database
try {
client.putItem(request);
return entry;
}
catch(final Exception e) {
throw new RuntimeException("Failed adding new entry to the database: "+ e.getMessage(), e);
}
}

// Returns null if record does not exists
public Descriptor getSpecific(String id, String version) {
// Build the get item request
final GetItemRequest request = new GetItemRequest()
.withTableName("ExampleTable")
.withKey(toKey(id, version));

// Try to get the item from the database
try {
final GetItemResult result = client.getItem(request);
return result.getItem() == null ? null : fromDDB(result.getItem());
}
catch(final Exception e) {
throw new RuntimeException("Failed reading entry (id: "+ id +") from the database: "+ e.getMessage(), e);
}
}

private static Map<string, AttributeValue> toDDB(final Descriptor entry) {
final Map<string, AttributeValue> items = new HashMap<>();
items.put("id", new AttributeValue(entry.getId()));
items.put("name", new AttributeValue(entry.getName()));
items.put("email", new AttributeValue(entry.getEmail()));
items.put("version", new AttributeValue(entry.getVersion()));
items.put("creationDate", new AttributeValue(Objects.toString(entry.getCreationDate())));
return items;
}

private static Descriptor fromDDB(final Map<string, AttributeValue> item) {
return Descriptor.builder()
.id(item.get("id").getS())
.name(item.get("name").getS())
.email(item.get("email").getS())
.version(item.get("version").getS())
.creationDate(Instant.parse(item.get("creationDate").getS()))
.build();
}

private static Map<string, AttributeValue> toKey(String id, String version) {
Map<string, AttributeValue> key = new HashMap<>();
key.put("id", new AttributeValue(id));
key.put("version", new AttributeValue(version));
return key;
}

The trick to getting the latest version is to making a query request on the LSI with reversed lookup and limit=1:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

// Returns null if record does not exists
public Descriptor getLatest(String id) {
// Build the query request
final QueryRequest request = new QueryRequest()
.withTableName("ExampleTable")
.withIndexName("SortedByCreationDate")
.withKeyConditionExpression("id=:id")
.withExpressionAttributeValues(Collections.singletonMap(":id", new AttributeValue(id)))
.withConsistentRead(true)
.withScanIndexForward(false)
.withLimit(1);

// Perform table query in order to get the latest item
try {
final QueryResult result = client.query(request);
return result.getCount() == 0 ? null : fromDDB(result.getItems().get(0));
}
catch(final Exception e) {
throw new RuntimeException("Failed reading latest entry (id: "+ id +") from database: "+ e.getMessage(), e);
}
}

Now, we can go and run our unit tests:

– Alexander.

At a Glance of a Key

Life got much easier - Using Lombok with Java

Marking field as required when using Lombok Builder

Leave a Reply Cancel reply

Simple item versioning with DynamoDB

Oh hi there 👋It’s nice to meet you.

Sign up to receive a notification when new posts are published!

Life got much easier - Using Lombok with Java

Marking field as required when using Lombok Builder

Leave a Reply Cancel reply

Oh hi there 👋
It’s nice to meet you.