Redis Streams and the Unified Log — Brandur Leach


59 bookmarks. First posted by jtrull november 2017.


Years ago an article came out of LinkedIn about theunified log , a useful architectural pattern forservices in a distributed system share state with oneanother.…
from instapaper
11 weeks ago by aviflax
Services emit state changes into an ordered data structure in which each new record gets a unique ID. Consumers track changes in the wider system by reading the log. Each one maintains the ID of the last record it successfully consumed and aims to read every record at least once – nothing should ever be missed. When a consumer is knocked offline, it restarts and looks up the last ID that it saw, and continues reading the log from there. The log is *unified* because it acts as a master ledger of state changes in a wider system. All components that are making critical changes write to it, and all components that need to track distributed state are subscribed.
redis  unified-log  microservices 
april 2018 by iyoti
Redis Streams and the Unified Log — Brandur Leach
from twitter
november 2017 by espencer
Years ago an article came out of LinkedIn about the unified log, a useful architectural pattern for services in a distributed system share state with one another. In the log’s design, services emit state changes into an ordered data structure in which each new record gets a unique ID.
ifttt  tfav  twitter  getpocket 
november 2017 by linkt
Useful look at how to use redis for a unified transaction log
webdev  log_analysis 
november 2017 by amcewen
A Nov 2017 peek at using Redis as the log service in a CQRS architecture.
redis  kafka  logging  architecture  cqrs 
november 2017 by ivar
Redis Streams and the Unified Log — Brandur Leach
from twitter
november 2017 by wschenk
Years ago an article came out of LinkedIn about the unified log, a useful architectural pattern for services in a distributed system share state with one another. In the log’s design, services emit state changes into an ordered data structure in which each new record gets a unique ID.
november 2017 by dagh
ather than emit directly to Redis, a “staged” record is created in Postgres. This indirection is useful so that in case the request’s transaction rolls back due to a serialization error or other problem, no invalid data (i.e. data that was only relevant in a now-aborted transaction) is left in the log. This principle is identical to that of transactionally-staged job drains, which do the same thing for background work.

The staged records relation in Postgres look like:

CREATE TABLE staged_log_records (
id BIGSERIAL PRIMARY KEY,
action TEXT NOT NULL,
data JSONB NOT NULL,
object TEXT NOT NULL
);
architecture  redis  logging  kafka  best 
november 2017 by hellsten
Architecture unifiedlog
november 2017 by amarandon
Years ago an article came out of LinkedIn about the unified log, a useful architectural pattern for services in a distributed system share state with one another. In the log’s design, services emit state changes into an ordered data structure in which each new record gets a unique ID.
IFTTT  Pocket 
november 2017 by deepblue