research-kg to warehouse handoff

For whoever migrates the research-kg knowledge graph into the staging Postgres. Yes, there is a pipeline, a schema, and a SQL dump. This package has all three.

Start with INTEGRATION_GUIDE.md for the wiring (exact loading steps, the map<->KG link, the /kg API contract, frontend wiring, refresh, and a pre-ship risk checklist). This README covers the migration options.

What the pipeline is

research-kg (org repo mapping-global-china/research-kg) extracts a knowledge graph from the MFA press-conference corpus and academic papers: typed, NLI-verified relations; reified claims with stance; dated events; and every node, edge, and claim carries provenance (source document, character span, evidence text, extractor, model, confidence). It runs on SQLite locally; this handoff is for landing it in Postgres.

That repo already contains the pipeline code (mgc_kg/), the full graph as a 21 MB compressed SQLite (data/mgc_kg.sqlite.gz), and a data dictionary (data/README.md) describing every table with counts.

The graph (current run)

• 44,781 nodes, 118,205 edges, 22,341 claims, 5,622 documents, 45,430 mentions, 2,081 aliases, 8,441 map_link rows.
• Node types: Country, Organization, Person, Project, Framework, Sector, Document, Claim, Impact, Event, Theme, Tag. Events are Event nodes plus HAS_EVENT / HAS_PARTICIPANT / OCCURRED_IN edges, not a separate table.
• Edge tiers: structural (deterministic dataset facts), inferred (extracted or computed relations, claims-as-edges, co-mentions), provenance (document to entity mentions).

Files in this package

• INTEGRATION_GUIDE.md: the wiring reference: exact Postgres loading steps (including the dump's table-naming gotcha), the kg.map_link contract, the full /kg API contract, frontend wiring, refresh, and a pre-ship risk checklist.
• sqlite_schema.sql — the pipeline's SQLite table structures, the source of truth (11 tables).
• postgres_kg_schema.sql — a Postgres + pgvector DDL that mirrors the SQLite schema 1:1 into a self-contained kg.* schema, written so the dump can load unchanged.
• mgc_kg_data.sql.gz — a 21 MB gzipped SQL dump of the graph tables (nodes, edges, claims, documents, node_mentions, aliases, place_mentions). It is SQLite-flavored SQL (CREATE + INSERT); the structures match postgres_kg_schema.sql. If you would rather have per-table CSVs or a Postgres COPY-ready dump, say so and I will produce it.
• map_link.sql: the map<->KG bridge table (kg.map_link, 8,441 rows), resolving a /viz map point to the KG node(s) it represents. Same SQLite-flavored dump format as mgc_kg_data.sql.gz; see INTEGRATION_GUIDE.md section C for the resolution rule and section B for the loading gotcha.

Two migration options (your call, you own the warehouse and entity_registry)

Option A, standalone kg.* schema (fastest). Create kg.* from postgres_kg_schema.sql and load the dump as-is. Self-contained, keyed by research-kg's own text node ids. Queryable immediately, but its entities are separate from global.entity_registry.

Option B, integrated, keyed to global.entity_registry (recommended long-term). Add a kg.node_map (research-kg node_id to entity_registry.entity_id UUID) and reconcile our entities onto the registry, so the KG's relations, claims, and events attach to the same canonical entities the rest of the warehouse uses. More work, with three known snags:

1. research-kg's Project and Framework entities (Gwadar, CPEC, BRI) are not in entity_registry and are excluded from its org seed, so expect a lot of propose-new.
2. Two resolvers (research-kg's BGE-M3 cross-lingual clustering vs the registry's trigram resolver) can over or under merge; reconcile at the canonical-component grain and send ambiguous ones to review.
3. research-kg's press-QA and papers are not yet bridged into global.documents, so the provenance foreign keys need those corpora ingested or bridged first.

We have already designed Option B in detail (the kg.node_map plus kg.edges / kg.claims / kg.events keyed to entity_registry, with an 084 alembic migration sketch that matches mgc-hub conventions). Ask and we will send that design doc.

One coordination note

The mgc-api /kg endpoints (already on branch feat/kg-endpoints) were written for Option B, querying entities keyed to entity_registry. If you go with Option A, the endpoint SQL keys on our node ids instead, a small adjustment either way. Let us align on A vs B before the migration so the API and the schema match.

Questions welcome.