Amazon Dynamodb

Amazon DynamoDB is a fully managed NoSQL database service that provides fast and predictable performance with seamless scalability. - docs.aws.amazon.com

Programming Interfaces

There are are 3 programming interfaces each have their own use case.

Low-Level Interfaces

The low-level programming model wraps direct calls to the DynamoDB service. You access this model through the Amazon.DynamoDBv2 namespace.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
// ---------- client
var config = new AmazonDynamoDBConfig()
{
    ServiceURL = "http://localhost:4001"
};

// This expects `access key` and `secret access key` to either be set in:
// the system file .aws\credentials
// OR
// as environment variables `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY`
_client = AmazonDynamoDBClient(config);


// ---------- GetItemAsync
var key = new Dictionary<string,AttributeValue>() { { "Id", new AttributeValue { N = "42" } } };
var request = new GetItemRequest
{
    TableName = _dynamoDbOptions.TableName,
    Key = key
};

// Returns the item attributes for the given key
_client.GetItemAsync(request);

Document Interfaces

The document programming model provides an easier way to work with data in DynamoDB. This model is specifically intended for accessing tables and items in tables. You access this model through the Amazon.DynamoDBv2.DocumentModel namespace.

Object Persistence Interface

The object persistence programming model is specifically designed for storing, loading, and querying .NET objects in DynamoDB. You access this model through the Amazon.DynamoDBv2.DataModel namespace.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
// ---------- client and di
using Amazon.DynamoDBv2;
using Amazon.DynamoDBv2.DataModel;
using Amazon.Extensions.NETCore.Setup;
using Microsoft.Extensions.DependencyInjection;

// creating the client as part of the .net application startup using IServiceCollection
var options = new AWSOptions();
options.DefaultClientConfig.ServiceURL = "http://localhost:4001";

var config = new DynamoDBOperationConfig
{
    TableNamePrefix = "Porky-"
};

services.AddAWSService<IAmazonDynamoDB>(options);
services.AddSingleton<IDynamoDBContext>(p => ActivatorUtilities.CreateInstance<DynamoDBContext>(p, config));


// ---------- creating the client manually could be used with integration tests
var dynamoDBContextConfig = new DynamoDBContextConfig()
{
    TableNamePrefix = "Porky-"
};

var amazonDynamoDBConfig = new AmazonDynamoDBConfig()
{
    ServiceURL = "http://localhost:4001"
};

var client = new AmazonDynamoDBClient(amazonDynamoDBConfig);

// This expects `access key` and `secret access key` to either be set in:
// the system file .aws\credentials
// OR
// as environment variables `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY`
return new DynamoDBContext(client, dynamoDBContextConfig);


// ---------- LoadAsync
return await _dynamoDbContext.LoadAsync<MyTableSchema>(keyId);

Where the class MyTableSchema is defined below. Note that DynamoDBHashKey is the partition key.

Optionally we can add DynamoDBRangeKey as a sort key.

1
2
3
4
5
6
7
8
9
10
11
12
[DynamoDBTable("Audit")]
public class MyTableSchema
{
    [DynamoDBHashKey] 
    public Guid KeyId { get; set; }

    [DynamoDBProperty]
    public string SomeString { get; set; }
    
    [DynamoDBProperty]
    public Guid SomeGuid { get; set; }
}

DynamoDBHashKey,DynamoDBRangeKey and DynamoDBProperty can also include enums using IPropertyConverter implemented in MyEnumConverter

See Mapping Arbitrary Data with DynamoDB Using the AWS SDK for .NET Object Persistence Model

1
2
[DynamoDBProperty(typeof(MyEnumConverter))]
public MyEnum MyEnum { get; set; }

Keys

Two types of Primary Keys and their valid data types are String, Number and Binary. All the examples below use the table Music

Partition Key

Artists Partition/Primary/Hash key by itself, it must be unique in the table. 

1
2
3
{
    "Artists" : "Celine Dion"
}

Composite primary key

Combination of partition Key and a sort/range key which makes it a composite key. Individually, the partition key and the sort/range key does not need to be unique, but the combination of both which makes it into a primary key needs to be unique.

Artists Partition/Primary/Hash key and Songtitle Sort key. This allows for a query like Artists = 'Celine Dion' & Songtitle = 'My Heart Will Go On'

1
2
3
4
{
    "Artists" : "Celine Dion",
    "Songtitle" : "My Heart Will Go On"
}

Indexes

When you need to query data and the current Partition/Primary/Hash key and/or sort/range keys are not enough - and you dont want to perform a db scan (ie: loop though all the data in the table) the you can add additional indexes.

Global secondary index (GSI)

  • Simple or composite
  • Can be created after the table is created
  • Max GSI per table is 20
  • Results in additional writes (cost)
  • Keep WCU (write capacity units) on GSI >= WCU on main table

An index with a partition key and sort key that can be different from those on the table. If you need to query on Genre you would create a GenreIndex and then query on Genre=Pop.

This means Genre is now a partition key for GenreIndex (its a new table that is automagically kept in sync) - anything needed to be returned when querying on the index is defined with Projection. If you ask for attributes not in the projection dynamo will do a Fetch, the default Projection is ALL.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
// Music Table
{
    "Artists" : "Celine Dion", // Partition key
    "Songtitle" : "My Heart Will Go On",
    "Genre" : "Pop",
    "AlbumTitle" : "Let's Talk About Love",
    "Year" : 2007
}

// GenreIndex (GSI)
{
    "Genre" : "Pop", // Partition key
    // and any projected attributes
}

Local secondary index (LSI)

  • Only composite
  • Must be created with the table
  • Esentially extends the functionality of the sort/range key to other attributes so you can perform more optimal queries
  • You need to know the Partition key value
  • Soft limit of 5 LSI
  • There is no additional cost for LSI’s

An index that has the same partition key as the table, but a different sort key. So when the LSI has Artists as partition key and Year as sort key, you can get a list of records with Artists = 'Celine Dion' & Year > 2000

1
2
3
4
5
6
7
8
// Music Table
{
    "Artists" : "Celine Dion", // Partition key
    "Songtitle" : "My Heart Will Go On",
    "Genre" : "Pop",
    "AlbumTitle" : "Let's Talk About Love",
    "Year" : 2007 // LSI
}

Locking

The SDK provides some settings that we can simply turn on/off. Consistent Read is one of them.

Consistent Read

Eventually consistent reads - By default is when you read data from a table, the response might not reflect the results of a recently completed write operation. The response might include some stale data. If you repeat your read request after a short time, the response should return the latest data.

Strongly consistent reads - When you request a strongly consistent read, DynamoDB returns a response with the most up-to-date data, reflecting the updates from all prior write operations that were successful.
but it has a higher latency and it uses more throughput capacity

Optimistic Locking

This can be easily enabled/disabled in the table.

We can add a version number column to our table to enable optimistic locking. When updating the item, it checks whether the version of the item matches the one stored in DynamoDB. If not, it throws exception. If version is the same, DynamoDB updates the record and automatically increments the version number.

1
2
[DynamoDBVersion]
public int? VersionNumber { get; set; }

Data Types

Scalar Types

Scalar Types represent exactly one value

1
2
3
4
5
Number
String
Binary             // base 64 encoded
Boolean
Null

Document Types

Can represent a complex structure with nested attributes such as a JSON document.

1
2
// List, can be a combination of data types
MyCoolList ["Foo", "Bar", 42]
1
2
3
4
5
6
7
8
9
10
11
12
13
// Map, complext object
{
    Pineapple: "Pen"
    Notes: [
        "Do": "Better",
        {
            Thing : {
                Id : 42
                Message : "The answer"
            }
        }
    ]
}

Set Types

Can represent multiple scalar values, the types in the set need to be the same, else it needs to be a List type

1
2
3
4
5
6
7
8
// String Set
["Foo", "Bar", "Biz", "Bat"]

// Number Set
[-42, 42, 142]

// Binary Set
["NDI=", "bGlmZQ=="]

Thew binary sets are base64 encoded.

  • NDI= = 42
  • bGlmZQ== = life