Federation Reference

This page is the reference for pg_ripple's SPARQL 1.1 federation engine.

Overview

pg_ripple implements SPARQL 1.1 Federated Query (SERVICE keyword) with a cost-based planner, result caching, connection pooling, circuit-breaker protection, and parallel remote execution. Local graphs can be mapped to SERVICE endpoints so the planner transparently rewrites remote-looking queries to local scans.

flowchart TD
    Q[SPARQL query\nwith SERVICE clause] --> P[Cost-based planner\nVoID statistics]
    P --> L[Local VP tables\nPostgreSQL scan]
    P --> R1[Remote endpoint A\nHTTP/SPARQL]
    P --> R2[Remote endpoint B\nHTTP/SPARQL]
    L --> J[Result join\n& DISTINCT]
    R1 --> C1[Cache\n_pg_ripple.federation_cache]
    R2 --> C2[Cache\n_pg_ripple.federation_cache]
    C1 --> J
    C2 --> J
    J --> Out[Final result set]

    CB1[Circuit breaker] -. guards .-> R1
    CB2[Circuit breaker] -. guards .-> R2

Status

SELECT feature_name, status FROM pg_ripple.feature_status()
WHERE feature_name LIKE '%federation%' OR feature_name = 'sparql_federation';

SQL Functions

Function	Description
`pg_ripple.register_endpoint(url TEXT, name TEXT, complexity TEXT) → void`	Register a remote SPARQL endpoint
`pg_ripple.drop_endpoint(name TEXT) → void`	Remove an endpoint registration
`pg_ripple.list_endpoints() → SETOF record`	List all registered endpoints
`pg_ripple.is_endpoint_healthy(url TEXT) → BOOLEAN`	Check circuit-breaker health status
`pg_ripple.federation_stats() → SETOF record`	Per-endpoint query latency and error statistics

SERVICE Clause

Use the standard SPARQL SERVICE clause in queries:

SELECT ?s ?label WHERE {
  ?s a <http://example.org/Person> .
  SERVICE <https://dbpedia.org/sparql> {
    ?s rdfs:label ?label .
    FILTER(LANG(?label) = "en")
  }
}

Cost-Based Planning

The federation planner uses VoID statistics (stored in _pg_ripple.endpoint_stats) to estimate remote result sizes and order SERVICE sub-queries to minimize total network round-trips. The complexity field (fast/normal/slow) adjusts ordering.

Caching

Results from remote endpoints are cached in _pg_ripple.federation_cache keyed by (endpoint_id, query_hash). The cache TTL is controlled by pg_ripple.federation_cache_ttl (default: 60 seconds).

Circuit Breaker

Each endpoint tracks probe history in _pg_ripple.federation_health. When an endpoint fails more than pg_ripple.federation_circuit_threshold times within the probe window, it is marked unhealthy and future SERVICE calls immediately return empty results until a successful probe.

Shard Pruning (Citus)

When Citus is installed, SERVICE clauses that resolve to a local shard are rewritten to direct shard scans, eliminating coordinator overhead. Multi-hop pruning is supported for chained SERVICE patterns.