Wikidata - landscape for Facility Foundry v1 onboarding

Phase 2 desk landscape, written 2026-05-07. Reference document for the Wikidata fetcher built against the v1 schema + ADR 008/009 facility-side synthesis + ADR 010/011 lineage substrate. Cross-references: docs/adr/008-cross-source-facility-identity-resolution.md, docs/adr/010-lineage-records-share-canonical-pipeline.md, docs/adr/011-cross-source-entity-reconciliation.md, docs/LINEAGE.md, docs/DATASOURCES.md, docs/phases/02-multi-jurisdictional-and-lineage.md.

Live-data observations (verified 2026-05-07 against the Wikidata Query Service at https://query.wikidata.org/sparql). All counts and property identities below were SPARQL-verified live; pre-research assumptions about MRDS / EPA cross-reference properties have been corrected against the actual schema.

1. Facility-class universe is bigger than the README suggests but skewed toward mining + power. Per-class counts (instances plus transitive subclass closure):

   Wikidata class	QID	Total	With coords	Coord rate
   Mine (mine)	Q820477	56,796	54,819	97 %
   Power station	Q159719	30,151	27,622	92 %
   Factory (broad)	Q83405	24,930	15,790	63 %
   Pulp mill	Q1473346	1,025	308	30 %
   Refinery (generic)	Q1867977	777	511	66 %
   Oil refinery	Q12353044	405	258	64 %
   Chemical plant	Q905286	304	122	40 %
   Steel mill	Q2069494	154	96	62 %
   Processing plant	Q759534	116	53	46 %
   Smelter	Q65515162	19	17	89 %
   Total	-	114,677	99,596	87 %

   Mining + power coverage is dense (97 % / 92 % with coords); metallurgical (smelter / steel mill / refinery) coverage is sparse compared to the FF facility footprint. Wikidata's industrial coverage tracks public visibility, not commodity importance.

2. The "free direct join" hope is mostly false for facility-side cross-references. What pre-research framing assumed (Wikidata has MRDS dep_id + EPA REGISTRY ID + MSHA mine_id properties for direct join):

   • MRDS: no property exists on Wikidata. Searched for MRDS / mineral resources data / USGS against the property label index; only USGS earthquake / publication / thesaurus properties exist. Wikidata's facility records cannot be joined to MRDS by a direct ID.
   • EPA Facility Registry Service (P10145): property exists, but only 9 entities populate it. Effectively unused.
   • MSHA / OSHA: no facility-side properties exist.

   So Wikidata facility records are not a drop-in cross-source join key; ADR 008/009-style synthesis (spatial + name + commodity matching) is the real bridge into FF's existing 432K-facility universe. That pattern already handles MSHA × MRDS × EPA - adding Wikidata as a 4th source is mechanically the same.

3. Entity-side cross-references ARE real and material. Across the whole Wikidata universe (not just facilities):

   • LEI (P1278): 52,759 entities populate it.
   • CIK (P5531): 3,555 entities populate it.
   • GeoNames (P1566): 4 M entities (used for spatial cross-ref).
   • OSM relation (P402): 566 K entities.

   Among distinct mine operators on Wikidata (1,201 unique): 9 % carry an LEI, 3 % carry a CIK. The big mining operators (BHP, Glencore, Rio Tinto, First Quantum, Newmont, MP Materials) all have LEIs; the long tail of small mining companies don't. Per ADR 011, every Wikidata operator with LEI / CIK becomes a HIGH-tier same_entity_as bridge into GLEIF L2 (when A.7 lands) and CorpWatch - the matcher we shipped in A.6 picks these up automatically.

4. Per-QID JSON fetch is tractable. Sampled Mountain Pass Rare Earth Mine (Q4285560): 14.9 KB, 17 claims, 10 languages with labels, populated keys include P31 (instance of: mine subclass), P17 (country: Q30 = USA), P625 (coords: 35.4786, -115.5325 - within 30 m of MRDS's Mountain Pass coords), P137 (operator: Q105563992 = MP Materials), P127 (owned by: MP Materials + historic Molycorp Q2325665), P1566 (GeoNames: 5375462), P1056 (commodities: REE QIDs). Multi-operator + multi-owner claims natural; lifecycle qualifiers (start_time / end_time) capture history. Cache size estimate: ~110K admitted QIDs × ~15 KB average = ~1.6 GB on disk after one full ingest (much smaller than CorpWatch's 7.6 GB).

   Multi-language labels solve the alternate-name coverage gap surfaced repeatedly in MSHA + MRDS work. Mountain Pass alone has aliases in 10 languages; per ADR 007 these flow into FacilityAttribute(attribute_name="alternate_names", value=tuple) for free, no extra fetcher logic.

This is the reference for the v1 Wikidata fetcher: facility-side plus entity-side. Per ADR 010, each admitted facility QID emits Facility + FacilityIdentifier (Wikidata QID + GeoNames ID + OSM ID where present) + Classification (industry from P31 class-tree, commodity from P1056 if mining-side) + LifecycleEvent (commissioned / decommissioned from P571 / P576) + OperatorReference (from P137 / P127 / P749). The operator's linked QID emits a lineage Entity record, with lei populated from P1278 when present (HIGH-confidence anchor) and the matcher picking up CIK-anchored bridges into CorpWatch.

---

1. Source overview

Publisher. Wikidata - a collaboratively-edited knowledge base operated by the Wikimedia Foundation. Entity records are CC0-licensed; the SPARQL query service runs at query.wikidata.org with a 60 s query timeout + ~30K-row result cap; the per-entity REST API serves JSON at https://www.wikidata.org/wiki/Special:EntityData/Q<id>.json with HTTP-cache headers that hishel respects.

License posture. Wikidata content is CC0 1.0 Universal (public domain dedication). FF includes Provenance.fetched(source_id="wikidata", source_url="https://query.wikidata.org/sparql", ...) on every emitted record. Per-QID-level provenance can carry the specific entity URL (https://www.wikidata.org/wiki/Q<id>) when useful.

Coverage. Volunteer-edited, so coverage tracks public visibility rather than economic importance:

• Strong: mines (56K), power stations (30K), publicly-known industrial sites in OECD countries.
• Moderate: refineries (1.2K combined), pulp mills (1K), factories with verified location data (16K).
• Sparse: smelters (19), small private metallurgical / chemical / processing facilities, and most facilities in jurisdictions without active Wikidata mining-editor communities (sub-Saharan Africa, Central Asia, parts of South-East Asia).

The gap shapes downstream consumption: Wikidata is best read as a name-+-jurisdictional enrichment overlay for the existing MRDS / MSHA / EPA universe and a mining-coverage extension into countries those US-anchored sources miss, not as a primary canonical-store backbone.

---

2. The artefact - and why we don't ingest the full dump

Wikidata publishes a complete JSON dump (~75 GB compressed, ~1 TB uncompressed) weekly at https://dumps.wikimedia.org/wikidatawiki/entities/. FF does not consume this dump. Two facility slices (above) plus the operator-entity slice add up to ~115K QIDs out of Wikidata's 110M; the SPARQL endpoint enumerates the universe in seconds, and the REST API serves per-QID JSON in a cacheable form.

Two-pass fetch:

1. SPARQL universe enumeration - one query per facility class. wdt:P31/wdt:P279* covers transitive subclass closure. Output per query: (qid, country_code, lat, lon) only - small payloads, well within the 30K-row / 60 s limits when partitioned by class (and by country for the larger classes if needed).
2. Per-QID JSON via REST - fetch https://www.wikidata.org/wiki/Special:EntityData/Q<id>.json for every QID in the universe. ~15 KB average; ~1.6 GB total cached. Hishel's HTTP cache makes re-fetches free unless the entity changed.

---

3. Records emitted per QID (admitted)

For each admitted facility QID, the parser emits:

One Facility record:

• facility_id minted as ULID (fac_<26-char>).
• canonical_name from labels.en (English label); when missing, fall back to the first non-empty label across configured language preference (en → es → fr → de → pt → other).
• latitude + longitude from claims.P625 mainsnak globe-coords. Admission requires both present + in-range.
• country_iso2 from claims.P17 → claims.P297 ISO-3166 alpha-2 on the linked country QID. Defaults to ZZ when neither resolves.
• coordinate_confidence: MEDIUM (Wikidata coordinates are volunteer-asserted; ~10 % have precision worse than 1 km).
• commissioning_year / decommissioning_year from claims.P571 (inception) and claims.P576 (dissolution / discontinued) mainsnak time values when present.
• status: operating if no P576; decommissioned if P576 has a past date; unknown otherwise.

FacilityIdentifier rows (one per cross-ref present):

• wikidata_qid: every admitted facility (the QID itself is the cross-ref).
• geonames_id (P1566): when populated. Strong spatial cross-ref; enables future GeoNames-anchored synthesis matches.
• osm_relation_id (P402): when populated. Spatial cross-ref for OpenStreetMap consumers.
• epa_registry_id (P10145): when populated (rare - only 9 records globally have it; emit anyway since the join key is precious when it does exist).
• Not emitted today: MRDS / MSHA / OSHA - Wikidata has no property for these. Cross-source matching against MRDS / MSHA / EPA happens through ADR 008/009 spatial-+-name synthesis at resolve time, not at fetch time.

Classification rows (multi-dimensional per ADR 007):

• industry: from claims.P452 (industry) or from the claims.P31 instance-of class-tree mapped to NAICS / ISIC. The mapping table lives in _mappings.py (not yet drafted) and covers the load-bearing facility classes - Wikidata mine → NAICS 2122x; Wikidata power station → NAICS 22111x; etc.
• process_role: derived from facility class + commodity for mining (extraction); for power stations, process_role is not set (FF's process-role taxonomy is mining-/manufacturing-centric; power generation is a Phase-3+ extension).
• commodity_or_product: from claims.P1056 (product or material produced) when present. Each product QID maps to a USGS commodity via a separate lookup table (Wikidata commodity QID → USGS commodity code).

LifecycleEvent rows:

• commissioned: when P571 has a date.
• decommissioned: when P576 has a date.

OperatorReference rows:

• From claims.P137 (operator): one row per operator claim, with entity_id = "wikidata_qid:Q<operator-qid>", role = "operator", valid_from / valid_to from claim qualifiers (P580 start / P582 end) when present.
• From claims.P127 (owned by): one row per owner claim, with role = "owner". Multi-owner claims with P1107 (proportion) qualifier produce the stake_percent field.
• From claims.P749 (parent organization on the operator entity): upstream parent-of-operator → OperatorReference(role="parent_guarantor") on the facility, plus an ownership_relationship lineage edge between the operator entity and its parent.

Lineage Entity records (per ADR 010): for every distinct operator / owner / parent QID referenced from an admitted facility, emit one Entity:

• entity_id = "wikidata_qid:Q<id>" (new identifier scheme; registers in models/_registry_ids.py as SCHEME_WIKIDATA_QID).
• canonical_name from labels.en.
• jurisdiction: claims.P17 country → ISO alpha-2; falls back to ZZ.
• lei from claims.P1278 when populated. The matcher's HIGH tier auto-bridges this entity to GLEIF L2 + CorpWatch by LEI string equality.
• confidence: MEDIUM when LEI populated, LOW otherwise. The Wikidata operator entity for BHP Group with LEI 549300C116EOWV835768 lands MEDIUM and bridges HIGH against CorpWatch's BHP entry; an unanchored mining-shell QID lands LOW.
• provenance: Provenance.fetched(source_id="wikidata", ...).

---

4. Quirks

• Multi-claim properties. Wikidata allows multiple values per property (a mine can have several historical owners; a company can have multiple LEIs over time). The parser reads ALL claim values and emits one record per claim, with start/end qualifiers capturing the temporal range. Live spot-check: BHP Group (Q625713) has two LEI claims. Glencore (Q169339) has two LEI claims with different validity ranges.
• Class-tree depth. wdt:P31/wdt:P279* recurses arbitrarily deep. Some chains terminate at "thing" (Q35120) far up; the query is fine but interpretation matters - instances of wdt:P31 wd:Q820477 (mine) include subclasses like "open-pit mine," "underground mine," "placer mine" et cetera. The parser records the most-specific class via the instance of claim and maps it to NAICS at synthesis time.
• Coordinate precision claims. Wikidata's globecoordinate values include a precision field (degrees). When precision is worse than 0.001° (~111 m), the parser emits coordinate_confidence = LOW instead of MEDIUM - flags the record as approximate so downstream synthesis spatial matching weights it less.
• Aliases as alternate names. aliases.<lang> lists alternate names per language. The parser flattens all aliases across all languages into a single tuple at FacilityAttribute(attribute_name="alternate_names"). Mountain Pass alone has 10 languages of labels; the alternate-name set becomes a 10+ string tuple useful for cross-source name matching.
• Volunteer-edit stale records. Some Wikidata facility records were last edited in 2014. There's no "freshness" property to filter on; the parser emits everything at MEDIUM coordinate confidence and lets the synthesis layer's authoritative-source preference (MSHA's weekly refresh beats Wikidata's volunteer edit when both describe the same site at different vintages) handle the calibration.
• SPARQL endpoint flakiness. Public endpoint rate-limits hard; long queries get killed. The fetcher's universe-enumeration step partitions by (class, country_code) to keep each query under ~5K rows + 60 s budget. stamina-driven retry with exponential backoff handles transient 429s.

---

5. Cross-source value

The expected wins from adding Wikidata to FF's facility roster:

1. Multi-jurisdictional facility coverage. 56K mines + 30K power stations + 16K factories with coords land in countries FF's US-anchored fetchers don't reach today. Australia, Brazil, Chile, Peru, Indonesia, South Africa, China - all heavy mining jurisdictions where MRDS's 2016 freeze is the only existing FF view. Wikidata fills the gap with current-edit records.
2. Alternate-name coverage at no extra cost. Per-QID labels in ~10 languages flow directly into FacilityAttribute(attribute_name="alternate_names"), expanding the matcher's name-fuzzy-match surface. The MSHA all-caps convention + MRDS's English-only names + Wikidata's multi-language tuples compose naturally at synthesis time.
3. Operator-side LEI bridge into the matcher. Every Wikidata operator entity with LEI populated (~110 mines × ~9 % rate, plus the long tail of power-station + factory operators) lands MEDIUM/HIGH-tier same_entity_as bridges via the A.6 matcher. This compounds - Wikidata × CorpWatch × (future) GLEIF L2 give a three-way LEI/CIK-anchored corporate hierarchy without any new matcher work.
4. GeoNames + OSM cross-refs as future spatial-resolver fuel. Wikidata's 4M GeoNames-tagged entities and 566K OSM-tagged entities aren't immediately consumed (FF doesn't have GeoNames or OSM fetchers today), but emitting them as FacilityIdentifier rows future-proofs the cross-source resolution surface against either of those sources landing as a follow-on Phase-2 / Phase-3 fetcher.