Synchronizing Postgres to Redis with Red Courier

Red Courier is an open-source Go application that syncs data from Postgres into Redis on a schedule. It’s built for high-performance environments where you want to populate Redis hashes, streams, or sorted sets from a relational source like Postgres — useful for caching, real-time analytics, or data fan-out systems.

In this article, we’ll walk through the motivation, architecture, and features of Red Courier — and show you how to use it with just a few lines of configuration.

‍

Why Red Courier?

In many applications — particularly in e-commerce, finance, or high-throughput APIs — Redis is used as a real-time cache or system of engagement, while Postgres (or another RDBMS) remains the source of truth.

‍

The common challenge is:

How do you keep Redis updated with new rows or changes in Postgres, without replicating the entire database or writing brittle glue code?

Red Courier solves this by letting you declaratively configure sync tasks:

• Define the source table and target Redis structure (map, list, set, sorted_set, stream)
• Configure how records should be keyed and valued
• Optionally set a tracking column to support incremental syncing
• Schedule each task using cron or interval notation

We open sourced Red Courier for all to enjoy — and we welcome contributions, feature ideas, and improvements from the community.

‍

Features

Sync Postgres data into Redis structures:

• map → Redis hash (HSET)
• list → Redis list (LPUSH)
• set → Redis set (SADD)
• sorted_set → Redis sorted set (ZADD)
• stream → Redis stream (XADD)

Configurable per-task scheduling

Per-task aliases for Redis keys

Column mapping from logical to physical DB names

Tracking support: Fetch only new/updated rows using a checkpoint value.

‍

Example Use Case

Let’s say you have a table orders in Postgres and you want to publish new orders into a Redis stream every 15 seconds. You can configure Red Courier like this:

tasks:
  - name: stream_orders
    table: public.orders
    alias: order_stream
    structure: stream
    fields: [id, status, amount, created_at]
    column_map:
      amount: total_amount
    schedule: "@every 15s"
    tracking:
      column: created_at
      operator: ">"
      last_value_key: checkpoint:order_stream

‍

Red Courier will:

• On first run, fetch all rows from orders
• Capture the max created_at value from those rows
• Save it to Redis under the key checkpoint:order_stream
• On future runs, fetch only rows where created_at > <last checkpoint>

Tracking supports both > and < comparisons and uses Redis to persist the last-seen value.

‍

Tracking in Detail

Tracking is optional but powerful. If you enable it with:

tracking:
  column: updated_at
  operator: ">"
  last_value_key: checkpoint:orders

Then Red Courier:

1. Checks Redis for the last stored value under checkpoint:orders
2. If no value exists, it fetches all rows and computes the max value
3. Stores that max as the new checkpoint
4. On next runs, uses a WHERE updated_at > $1 clause in the query

This lets you run efficient delta syncs for fast-growing tables.

‍

Configuration Overview

Red Courier reads from a single YAML file that configures:

• Postgres connection
• Redis connection
• One or more tasks

You can see a full example in the README.

‍

Running It

Red Courier can be compiled from source, run as a binary, or containerized:

go run ./cmd/courier

Or via Docker:

FROM golang:1.21 AS builder
WORKDIR /app
COPY . .
RUN go build -o red-courier ./cmd/courier

FROM gcr.io/distroless/base-debian11
WORKDIR /red-courier
COPY --from=builder /app/red-courier .
COPY config.yaml .
ENTRYPOINT ["./red-courier"]

Want to Try It?

Check out the repo:
👉 github.com/Checker-Finance/red-courier

And see the LLM-oriented configuration guide to help generate valid YAML quickly using structured prompts.

Whether you’re building an order feed, caching product data, or syncing financial transactions, Red Courier gives you a clean way to keep Redis in sync with Postgres — efficiently and on your terms.

‍