Skip to content
← All patterns

compliance

Change & Audit Log

What happened, and when?

An append-only history beside the current state it explains.

Every change to a resource is written as an immutable event, and the resource's current state lives in the same partition with a version number. One Query returns the state and the history that produced it. An index orders each resource's events by time for a range read, and the table's stream carries every change, old and new image, to whatever archives it.

The model

CurrentState

What the resource looks like now, and which version that is.

pk
RESOURCE#<resourceId>
sk
CURRENT

Attributes: status (S), version (N), updatedAt (N)

ChangeEvent

One immutable record of a change, numbered in sequence.

pk
RESOURCE#<resourceId>
sk
EVENT#<sequence>

Attributes: action (S), actor (S), at (N), eventId (S), actorType (S), payload (M), success (BOOL), correlationId (S)

Access patterns

  • QueryRead a resource and its history

    The current state and every event that led to it.

  • UpdateItemApply a change

    Update the current state, only if nobody else has since you read it.

  • GetItemRead the current state

    Just the resource as it stands, without its history.

  • Query · GSI1Events for entity in time range

    A resource's events between two times, read from the index that orders them by when they happened, for reconstructing a period of its history.

  • QueryLatest event for entity

    The most recent event on a resource: its events read in descending sequence, stopped after one. Useful for last-modified and state checks.

  • GetRecords · StreamStream fan-out to audit archive

    Consume the table's stream with GetRecords and ship each record to S3 or Firehose. NEW_AND_OLD_IMAGES gives downstream consumers enough context to reason about state changes.

Design notes

CURRENT sorts before every EVENTwhy

The sort key labels put the current state at the head of the partition, ahead of the log. So a Query that stops after one row is a cheap read of the present, while the same Query run out returns the whole history. One key design, two very different reads.

The version number is the lockwhy

An update carries the version it read and asks DynamoDB to apply the change only if version still matches. Two reviewers acting on version 3 both send the same condition; the first wins and moves it to 4, the second is refused and has to re-read. No row is locked and nothing waits - the loser simply finds out.

The state and its event are one write, or neithertrade-off

Advancing CURRENT and appending its EVENT# are two writes, and a crash between them leaves a resource whose state no event explains - which is the one thing an audit trail exists to prevent. Production binds them in a single TransactWriteItems: the guarded update and the conditional append succeed together or not at all. The access pattern here is only the guarded update, because the engine in this page cannot run a transaction.

The second approval is refusedtrade-off

Apply a change approves policy-9930, moving it from version 1 to 2 on condition that it is still at 1. Run it again and it is refused, as the second reviewer would be: the version it expects has gone.

Events are numbered, not timestamped, for orderingwhy

at records when a change happened, but EVENT#000001 is what orders the log. Two changes in the same millisecond collide on a timestamp key and one silently overwrites the other; a sequence removes that. It does not remove contention on its own - two writers can still reach for the same number - so the number comes from the version the guarded update just bumped, and the event is written with attribute_not_exists on its sort key so a second claimant is refused rather than overwriting. The padding matters as much as the number, since sort keys compare as strings and EVENT#10 sorts before EVENT#2 without it.

An index for time, beside the sequencewhy

The sequence keeps the log correct, but a question like what happened last week is about time. GSI1 keys on the same partition with at as its sort key, so a resource's events come back in time order and a range is one BETWEEN. An index key does not have to be unique, so two events at the same moment both stay in it. CURRENT carries no at, so it never appears there.

The stream read is not a run heretrade-off

Stream fan-out to audit archive is a GetRecords on the table's stream. It reads through a shard iterator, which only a live table's stream hands out, so the page cannot run it and lists it without a run. On a real table every write has a stream record: Apply a change produces one carrying both the old and the new CURRENT.

Append-only by conventiongotcha

Nothing in DynamoDB enforces append-only. Guarantee it in the application: CURRENT is the one item that is updated, and an EVENT# item takes no DeleteItem, and no UpdateItem beyond setting an archived flag. Hand the stream's ARN to audit consumers so they can replay.

NEW_AND_OLD_IMAGES has cost implicationstrade-off

Storing both images doubles the per-record stream payload compared with NEW_IMAGE alone. For an audit log this is usually worth it: downstream consumers can diff without another round trip to the base table.

correlationId ties cross-system events togetherwhy

Pass a correlationId through from the inbound request id and put it on every event. It makes it simple to reconstruct what happened across services for a single user action.

No TTLalternative

Audit logs are usually kept for a compliance window measured in years. Archive the stream's records to S3 Glacier, and keep the archive for the compliance window rather than the table. If you do want a TTL, set it far enough out that the archive pipeline is certain to have consumed the record first.

Taught in the course

audit-logDynamoDB workbench
Ready to run
Explore an access patternSelect to load & run

The current state and every event that led to it.

Request
Execute against the local Dynoxide engine
ReturnedFiltered outChanged
PK(pk)
SK(sk)
status
updatedAt
version
action
actor
at
actorType
correlationId
eventId
payload
success
RESOURCE#policy-4471
CURRENTS
approvedS
1794787200N
3N
—
—
—
—
—
—
—
—
RESOURCE#policy-4471
EVENT#000001S
—
—
—
createdS
adaS
1791417600N
—
—
—
—
—
RESOURCE#policy-4471
EVENT#000002S
—
—
—
submittedS
rajS
1794096000N
—
—
—
—
—
RESOURCE#policy-4471
EVENT#000003S
—
—
—
approvedS
meiS
1794787200N
—
—
—
—
—
RESOURCE#policy-9930
CURRENTS
draftS
1793836800N
1N
—
—
—
—
—
—
—
—
RESOURCE#policy-9930
EVENT#000001S
—
—
—
createdS
samS
1793836800N
—
—
—
—
—
RESOURCE#order#1001
EVENT#000001S
—
—
—
createdS
user_aliceS
1794870000N
userS
corr_req_001S
evt_aaa1S
{3}M
trueBOOL
RESOURCE#order#1001
EVENT#000002S
—
—
—
payment_succeededS
stripeS
1794870002N
serviceS
corr_req_001S
evt_aaa2S
{3}M
trueBOOL
RESOURCE#order#1002
EVENT#000001S
—
—
—
createdS
user_bobS
1794871871N
userS
corr_req_002S
evt_bbb1S
{2}M
trueBOOL
RESOURCE#order#1002
EVENT#000002S
—
—
—
payment_failedS
stripeS
1794871874N
serviceS
corr_req_002S
evt_bbb2S
{2}M
falseBOOL
RESOURCE#account#alice
EVENT#000001S
—
—
—
updatedS
user_aliceS
1794865320N
userS
—
evt_ccc1S
{3}M
trueBOOL
RESOURCE#subscription#team-nubo
EVENT#000001S
—
—
—
status_changedS
systemS
1794623400N
systemS
—
evt_ddd1S
{2}M
trueBOOL
Awaiting request
Your next query starts here.

Choose an access pattern above, or build your own request. See what comes back and what it costs.

Write transactions, streams, tags and TTL are among the operations this browser build leaves out. dynoxide's native build has them.