by Bookcicle

Documentation / Overview

Rippflow architecture: control plane, execution cells and data pipeline

How the Controller, customer-owned AWS execution cells, DynamoDB authority and Valkey acceleration fit together.

Interactive architecture explorer

Explore Rippflow across three progressive levels: the platform control and data planes, the internals of a single execution cell, and the streaming observability and recovery pipeline.

Platform Overview

ENTRY POINT CLI & Console Developer & operator entry Local authoring, deploy & inspect ISOLATED CONTROL PLANE (PUBLIC-EDGE OR PRIVATE-ENTERPRISE) Controller CloudFront+WAF+Function URL OAC or Internal ALB · Valkey 9 sessions · Registry Not in the per-transition execution hot path INGRESS & ROUTING Workflow Admission & Distribution Weighted cell placement, idempotency & callbacks CUSTOMER BYOC DATA PLANE Customer AWS Execution Cells Runs in customer-owned AWS infrastructure Starter · Worker · Orchestrator · Watchdog DynamoDB authority + Valkey acceleration ASYNC TELEMETRY Data & Observability Pipeline DynamoDB Streams → Kinesis Projections & Parquet archive Pinned at admission only
Controller (Control Plane) →Handles authentication, immutable workflow registry, and read projections. Not in the per-transition execution hot path.CLI, Console & Admission →Publish immutable workflow versions and route admitted executions to healthy cells.Customer AWS Execution Cells →Self-contained BYOC cells running in customer-owned AWS accounts with isolated IAM roles.Data & Observability Pipeline →Streams committed state transitions into near-real-time projections and durable archives.

Inside an Execution Cell

SHARE-NOTHING AWS EXECUTION CELL INGRESS Starter Admission & shard dispatch Pins workflow & policy digest Verifies async callbacks DISPATCH LANES SQS Lanes & DLQs Synchronous & queued lanes Primary & overflow queues Rework & DLQ isolation COMPUTE Worker & Handler Runtime Lambda, Batch, Fargate & asynchronous callbacks Rust, Python, Node.js, Java COORDINATOR Orchestrator Barrier fan-in Outcome routing Shard joins & loops SYSTEM OF RECORD DynamoDB (Authoritative State) Fenced task checkpoints, shard receipts, control state, async outbox & commit counters LOW-LATENCY BARRIERS Valkey (Coordination Accelerator) Sub-millisecond fan-in counters Rebuilt automatically from DynamoDB SELF-HEALING Watchdog Lease expiry & timer firing Stuck-attempt & barrier recovery
Starter, SQS Lanes & Orchestrator →Pins policy digests at admission, dispatches sync_lambda and async_sqs lanes, and coordinates barriers.Worker & Handler Runtime →Runs polyglot subprocess handlers or invokes external Lambda, Batch, and Fargate compute.DynamoDB Authority & Valkey Acceleration →DynamoDB commits authoritative checkpoints and receipts; Valkey accelerates Lua fan-in barriers.Watchdog & Durable Waits →Sweeps expired leases, fires due timers, rebuilds flushed Valkey barriers, and isolates poison messages.

Data, Observability & Recovery

CDC CAPTURE DynamoDB Streams Filters heartbeats, redacts secrets & batches committed transitions EVENT STREAM Regional Kinesis Ordered regional transition stream Zero extra writes on hot commit READ VIEWS & METERING Controller Projections Near-real-time execution & accounting 5m/1h/1d rollups & shard watermarks RESILIENCE & FAILOVER BOUNDARY Multi-Cell Routing & Ownership Epoch Fencing Cell drain, failover & regional recovery fenced by monotonic ownershipEpoch Multi-cell & multi-region depend on installed topology and edition entitlements LONG-TERM RETENTION Parquet & S3 / Glacier Archival Verified Apache Parquet v2 warm cohorts Automated S3 Glacier lifecycle retention
DynamoDB Streams & Regional Kinesis →Filters heartbeats, redacts sensitive payloads, and streams committed state transitions into Regional Kinesis.Controller Projections & Parquet Archival →Materializes execution views and accounting while writing verified Parquet v2 cohorts to S3 and Glacier.Multi-Cell Routing & Topologies →Routes traffic across 1 to 6 cells and up to 3 Regions based on installed topology and edition entitlements.Ownership Epoch Fencing →Protects in-flight executions during cell drain, failover, and regional recovery with monotonic epoch fences.

Core architectural invariants

  • The Controller is not a per-transition dependency: workflow and policy digests are pinned at admission so running executions continue even if the control plane is unreachable.
  • Workflow execution occurs in customer-owned AWS infrastructure: handlers, queues, caches, and state tables run inside the customer's BYOC AWS account.
  • DynamoDB is authoritative; Valkey is a coordination accelerator: every checkpoint and shard receipt commits to DynamoDB first, allowing Valkey barriers to be reconstructed at any time.
  • Multi-region capabilities depend on the installed topology: single-cell, multi-cell regional, and tri-region quorum behaviors require the corresponding deployed infrastructure.
  • Installation capabilities and commercial entitlements are distinct: technical topology support and commercial cell, Region, and SSO RBAC entitlements are enforced distinctly.
  • Management, admission, execution, and observation flows are separated: Controller ingress (IngressMode=public-edge CloudFront + WAFv2 + Function URL OAC or IngressMode=private-enterprise internal ALB + private DNS, where Valkey 9 stores only HMAC-keyed rippflow:ctrl:sess:* session validity {"v":1,"active":true} + TTL with zero PII/credentials/claims and DynamoDB verifies identity and permissions on every request), workflow admission, hot-path execution cells, and Kinesis CDC projections operate on independent planes.

Verify your installation's active topology and release version with rippflow installation version --json and rippflow cells list --json.

Advanced implementation details

Related advanced guides

Execution & lanes → Inspect sync_lambda and async_sqs lanes, shard scopes, barriers and outcome routing.Cells & recovery → Review cell isolation, ownership epoch fencing, watchdog sweeps and checkpoint failover.Streaming, archives & accounting → Trace DynamoDB Streams, Regional Kinesis CDC, Controller projections and Parquet/Glacier archives.
Verified sources

Checked 2026-11-05.

Let's talk

Contact Rippflow

Tell us about your workflows, deployment or pricing.

We'll reply to your email.

Search docs

Search workflows, handlers and operations.