Search for a command to run...
I'm an old school developer, I started 20 years ago with Notepad as an IDE (?), then I worked as DevOps and for a few years now I've been working as a Principal Technical Architect. I'm interested in cloud computing and serverless architectures and love to dive deep into problems and find smart solutions.
No comments yet. Be the first to comment.
In this series, I'll cover different atomic counter pattern implementations from distributed databases. We want to investigate how popular distributed databases deal with CAP theorem dilemma.
A Hands-on Guide
Specs, just enough
My experience and thoughts with coding vibes
A Hands-on guide
Scaling SQL Across the Universe
A Hands-on Guide
DynamoDB, a serverless NoSQL database, is a go-to choice for implementing atomic counters due to its built-in support for atomic operations and managed scalability. This article will guide you through how DynamoDB ensures consistency and replication, a refresher on the atomic counter pattern, and a hands-on walkthrough of a deployable example from this GitHub repository.
By the end, you’ll understand how to leverage DynamoDB for atomic counters and know the trade-offs involved.
Before we can dive deep into code, we need to recall few concepts (please refer to the first article of this series for a detailed explanation)
Serializability: Operations appear in a consistent sequential order, ensuring correctness.
Linearizability: Writes are immediately visible for subsequent reads, ensuring real-time consistency.
DynamoDB achieves linearizable writes through its single-leader replication model:
Write operations are directed to the leader node for the partition key, and changes are propagated to replicas.
Strongly consistent reads (optional) ensure the latest value is returned immediately after a write.
This guarantees DynamoDB can safely implement atomic operations like counters.
DynamoDB automatically manages replication across multiple availability zones to ensure durability and availability. Key mechanisms include:
• Single-leader replication: A leader node handles writes, maintaining consistency while replicas handle reads.
• Leader election: If a leader fails, DynamoDB promotes another replica seamlessly, ensuring high availability without manual intervention.
This replication strategy enables DynamoDB to handle distributed workloads while maintaining data consistency.
Synchronized timestamps play a critical role in distributed databases, especially for ensuring consistency across geographically dispersed replicas.
Without synchronized clocks, it becomes challenging to determine the order of operations accurately, leading to potential consistency issues in global-scale applications.
In the AWS ecosystem, the AWS Time Sync Service provides a highly accurate and reliable time source synchronized across all AWS Regions.
Announced last year, this service offers nanosecond-level precision and a consistent view of time, serving as a foundational piece for distributed systems.
Recently, AWS built upon this foundation to announce strong consistency for DynamoDB global tables. This new feature allows applications to perform strongly consistent reads and writes across multiple regions, ensuring the same data is visible no matter where the query originates.
Why is this important? Strong consistency in global tables depends on synchronized timestamps to ensure that write propagation across regions respects causal ordering. This prevents race conditions and ensures data correctness even in high-latency or failure scenarios.
Impact on Atomic Counters: If your atomic counter spans multiple regions via global tables, synchronized timestamps enable accurate propagation of updates, preserving the order and integrity of increments.
This synergy of the AWS Time Sync Service and DynamoDB advancements showcases how synchronized time is more than just an infrastructure detail—it’s a cornerstone of achieving robust distributed consistency.
DynamoDB’s conditional write feature allows you to execute write operations (PutItem, UpdateItem, DeleteItem) only if specific conditions are met.
This capability is crucial for enforcing business rules, ensuring data integrity, and preventing race conditions in distributed systems.
When you perform a conditional write, you include a ConditionExpression in the request.
DynamoDB evaluates this condition against the item’s existing attributes before executing the operation:
If the condition evaluates to true, the write operation proceeds.
If the condition evaluates to false, the operation fails with a ConditionalCheckFailedException.
Few use cases for conditional writes includes:
ConditionExpression: "attribute_not_exists(partitionKey)"
ConditionExpression: "counterValue < :maxValue"
Safely update an item only if its version matches a known value (optimistic locking):
ConditionExpression: "version = :expectedVersion"
enforce rules like “only update if another attribute matches a specific state.”
Atomicity:
Conditional writes ensure atomic operations by evaluating and writing in a single step, reducing the need for complex locking mechanisms.
Data Integrity:
Prevent unintended overwrites or updates by applying conditions based on the item’s current state.
Performance:
Conditional expressions are evaluated directly on the DynamoDB service, minimizing latency and avoiding additional queries to check conditions beforehand.
The atomic counter pattern ensures safe, concurrent updates to a counter without losing increments due to race conditions. In distributed systems:
• Operations must be atomic (all or nothing).
• DynamoDB achieves this with the UpdateItem operation and the ADD attribute update expression, which ensures the counter is incremented atomically.
Let’s explore the inner workings of the example in the GitHub repository, focusing on how DynamoDB is used for atomic counters.
The implementation includes:
1. API Gateway: Provides HTTP endpoints for interacting with the counter.
2. Lambda Functions: Implements the business logic for incrementing the counter and enforcing optional constraints like a maximum value.
3. DynamoDB Table: Stores the counters with atomicity guarantees.
In my example project you can decide wheter to use a maximum value for the counter or not: this determine if use ore not conditional writes.
Let’s focus on this logic,from the dynamoDbAtomicCounter Lambda code:
const id = event.pathParameters?.id;
const writeParams = getWriteParams(useConditionalWrites, id, maxCounterValue);
const dynamoDBClient = await buildDynamoDbClient();
const result = await dynamoDBClient.send(new UpdateItemCommand(writeParams));
const counter = Number(result.Attributes?.atomic_counter.N);
This code snippet simply sends an UpdateItemCommand and gets the new updated counter value, using the AWS SDK DynamoDBClient.
Le’t see how the getWriteParams works:
const getWriteParams = (useConditionalWrites: boolean, id: string, maxCounterValue: string) => {
const TABLE_NAME = process.env.TABLE_NAME || '';
const unconditionalWriteParams = {
TableName: TABLE_NAME,
Key: {
id: { S: id },
},
UpdateExpression: 'ADD atomic_counter :inc',
ExpressionAttributeValues: {
':inc': { N: '1' }
},
ReturnValues: 'UPDATED_NEW' as const,
};
const conditionalWriteParams = {
TableName: TABLE_NAME,
Key: {
id: { S: id },
},
UpdateExpression: 'ADD atomic_counter :inc',
ConditionExpression: 'attribute_not_exists(atomic_counter) or atomic_counter < :max',
ExpressionAttributeValues: {
':inc': { N: '1' },
':max': { N: maxCounterValue },
},
ReturnValues: 'UPDATED_NEW' as const,
};
return useConditionalWrites ? conditionalWriteParams : unconditionalWriteParams;
}
This method checks if conditional writes are required: the update expression is the same in both cases, and it leverages the ADD command.
UpdateExpression: 'ADD atomic_counter :inc'
If conditional writes are required, a Condition Expression is used:
ConditionExpression: 'attribute_not_exists(atomic_counter) or atomic_counter < :max'
DynamoDB’s managed infrastructure handles replication and scaling.
Simple and efficient atomic updates using UpdateItem.
Hot partitions: High traffic to a single counter may cause throttling.
Throughput limits: Monitor RCUs/WCUs to avoid performance degradation.
Eventual consistency: Use strongly consistent reads when precise counter values are critical.
DynamoDB provides a simple and scalable solution for implementing atomic counters in distributed systems.
Its built-in atomicity and managed replication make it a strong candidate for this pattern.
Explore the GitHub repository to deploy the example and experiment with atomic counters in DynamoDB, and stay tuned for the next article, where we’ll explore the ElastiCache Redis implementation of the atomic counter pattern.