Feature: Configuration Hot-Reload

pg_tide supports hot-reloading pipeline configurations without restarting the relay process. When you add, modify, or disable a pipeline in the PostgreSQL catalog, the relay detects the change and reconciles — starting new pipelines, stopping removed ones, and updating modified configurations in place.

How It Works

The relay discovers configuration changes through two mechanisms:

1. LISTEN/NOTIFY — Immediate notification when catalog tables change
2. Periodic polling — Rediscovers pipelines every discovery_interval_secs (fallback)

When a change is detected, the coordinator compares the new pipeline set against the currently running pipelines:

• New pipeline → Acquire advisory lock, spawn worker task
• Removed pipeline → Signal worker to stop, release advisory lock
• Modified pipeline → Stop old worker, start new one with updated config
• Disabled pipeline → Same as removed (worker stopped, lock released)

Triggering a Reload

Automatic (via LISTEN/NOTIFY)

The relay listens on the tide_relay_config PostgreSQL notification channel. When you call any tide.relay_set_* function, a notification is emitted automatically:

-- This triggers immediate reload
SELECT tide.relay_set_outbox(
    'orders-pipeline',
    'order_events',
    '{"sink_type": "kafka", "brokers": "kafka:9092", "topic": "orders"}'::jsonb
);

Periodic Discovery

Even without NOTIFY (e.g., if the relay reconnects after a network partition), the coordinator polls for changes every discovery_interval_secs:

# Default: 30 seconds
discovery_interval_secs = 30

What Can Be Changed Without Restart

Graceful Pipeline Transitions

When a pipeline configuration changes, the existing worker is drained before the new one starts:

1. Worker receives stop signal
2. Current batch completes (in-flight messages finish)
3. Source acknowledgment completes
4. Worker task exits
5. New worker spawns with updated config
6. New worker begins polling

This ensures no messages are lost or double-processed during reconfiguration.

Configuration

The discovery mechanism itself is configured at the process level:

# How often to poll for pipeline changes (fallback)
discovery_interval_secs = 30

Or via CLI:

pg-tide --discovery-interval 30

Further Reading

• Relay Configuration — Full process-level configuration
• HA Coordination — How multiple relays handle config changes