SEC EDGAR via CorpWatch - landscape for Facility Foundry v1 onboarding

Phase 2 desk landscape, written 2026-05-06. Reference document for the SEC EDGAR Exhibit 21 fetcher built against the v1 schema + ADR 010 lineage substrate. Cross-references: docs/adr/010-lineage-records-share-canonical-pipeline.md, docs/LINEAGE.md, docs/DATASOURCES.md, docs/phases/02-multi-jurisdictional-and-lineage.md.

Live-data observations (verified 2026-05-07 against the Sept-2025 dump on disk at tmp/sources/corpwatch.org/corpwatch_api_tables_csv/). Numbers here supersede earlier 2024-era estimates from the CorpWatch README and the OpenSanctions metadata - the actual dump is 2.3-2.9× larger across the board.

1. The canonical bulk URL has moved. The original http://api.corpwatch.org/documentation/db_dump/corpwatch_api_tables_csv.tar.gz returns 404. The Internet Archive mirror is the live source: https://archive.org/download/corpwatch_api_data_dumps/corpwatch_api_tables_csv.tar.gz

• 1,598,109,530 B (~1.5 GB compressed; ~7.6 GB uncompressed across the CSV set). OpenSanctions' us_corpwatch metadata confirms this as the official canonical source. Download from archive.org is slow (sustained ~hundreds of KB/s); the FF fetcher supports a --bulk-source <path> flag that points at a pre-extracted directory so dev cycles don't re-download.

1. Vintage anchored on meta.csv:update_date. The on-disk dump reports update_date: 2025-09-13 in meta.csv, with previous_update: 1755327600 (= 2025-08-16) and begin_date: 2003-1-1. Vintage = 2025-09-13 for the FF fetcher. (The archive.org "Last-Modified" header is later, but the canonical content date is what meta.csv records.)

2. Coverage is far larger than the README suggests. Verified row counts on disk:

   Table	Rows	Bytes	What it is
   companies.csv	1,421,198	90 MB	one row per cw_id; 749K filer entities (have CIK) + 672K cw_id-only stubs
   company_relations.csv	3,517,388	184 MB	distilled (parent_cw_id, child_cw_id, year) edge table
   relationships.csv	4,821,343	755 MB	per-filing-quarter raw observations; carries percent, parse_method, hierarchy, cik
   company_locations.csv	2,622,964	349 MB	address rows with country_code, subdiv_code, most_recent flag
   filers.csv	7,625,247	2.6 GB	per-(filer_id, filing_id) row; carries incorp_country_code + incorp_subdiv_code
   company_names.csv	2,435,331	250 MB	name variants - not ingested in A.5
   filings.csv	22,731,743	3.2 GB	every SEC filing, most with has_sec21=0 - not ingested
   meta.csv	5	100 B	dump metadata (update_date, begin_date, …)

   Vulcan Materials Co (cw_id=1896, cik=103973), MP Materials, Rio Tinto, Newmont, Freeport-McMoRan, SanDisk Corp (cw_id=11775) are all canonically present with full subsidiary trees.

3. relation_type is 100% NULL in company_relations.csv - every one of the 3.5M rows. The CorpWatch schema reserves the column to distinguish edge kinds (e.g. ownership-percent vs parent-control), but the September-2025 dump does not populate it. A.5 emits kind="controls" for every distilled edge. Stake percentages live on relationships.csv.percent (the per-filing-quarter raw observations), aggregating parent_cw_id × cw_id × year → max-percent is the path for the deferred A.5.6 owns_share enrichment.

4. Direction convention (verified by empirical join). A sample row in company_relations.csv has relation_id=17127, source_cw_id=11775, target_cw_id=72185 pointing to relationships.relationship_id=8 where parent_cw_id=11775, cw_id=72185. So: source_cw_id = parent (filer side); target_cw_id = subsidiary (the entity listed in the parent's Exhibit 21). FF's lineage Relationship direction convention is the opposite (from_entity = subsidiary, to_entity = parent), so the fetcher swaps source/target on emit: from_entity = "<scheme>:<target_cw_id>", to_entity = "<scheme>:<source_cw_id>". This is the easiest error to get backwards - fixture-test it explicitly.

5. Jurisdiction has a clear precedence. filers.csv carries incorp_country_code + incorp_subdiv_code for filer entities (the gold-standard "state of incorporation" signal: 4.35M US, 287K KY, 67K CA, 31K GB, 30K VG, 22K LU, 17K BM - classic offshore-corporate distribution). company_locations.csv rows where most_recent=1 provide the fallback for cw_id-only stubs that don't appear in filers.csv. Default ZZ when neither resolves.

This is the reference for the v1 SEC EDGAR (via CorpWatch) fetcher: lineage-side only (no facility-side records emitted). Per ADR 010, each companies.csv row → one Entity; each company_relations.csv row → one ownership_relationship with kind="controls" (the relation_type=NULL reality means we don't conditionally emit owns_share in A.5; that becomes the deferred A.5.6 enrichment via relationships.csv.percent). The fetcher writes exclusively to the resolved tier's entity/source_id=sec_edgar_corpwatch/ and ownership_relationship/source_id=sec_edgar_corpwatch/ partitions.

---

1. Source overview

Publisher. CorpWatch - a US accountability non-profit. CorpWatch parses Exhibit 21 attachments from public companies' 10-K filings on SEC EDGAR and republishes the structured parent/subsidiary edges as a bulk dump. The underlying SEC EDGAR filings are public-domain federal data (17 U.S.C. § 105); CorpWatch's parser code is MIT-licensed.

Why via CorpWatch and not directly from SEC. SEC publishes the underlying 10-K filings but does not publish a pre-parsed Exhibit 21 dataset. Each filing's Exhibit 21 is a free-form HTML or text attachment - extracting the structured parent/subsidiary list requires regex parsing of heterogeneous formats. CorpWatch has been doing this work since 2003; doing it ourselves would be reinventing the wheel for marginal fidelity gain. See the four-source group in LINEAGE.md for context - CorpWatch direct is the default; the OpenSanctions republish is a sibling path under CC-BY-NC; Wharton's WRDS Company Subsidiary Data is the aspirational higher-fidelity alternative pending institutional access.

License posture.

• SEC content (the underlying Exhibit 21 text): public domain (17 U.S.C. § 105).
• CorpWatch parser code: MIT (github.com/michalgm/corpwatchapi).
• CorpWatch dataset itself: no explicit license assertion but derived entirely from public-domain SEC filings. Attribution to CorpWatch is the right move regardless. FF includes Provenance.fetched(source_id="sec_edgar_corpwatch", ...) on every emitted Entity + Relationship - the source-id prefix names both the upstream-of-upstream (SEC EDGAR) and the parser provider.

Coverage. US-listed registrants that file 10-Ks containing an Exhibit 21:

• Included: Form 10-K filers (typically US-incorporated public companies + American Depositary Receipt issuers + some foreign private issuers that elect to file 10-K). All major US public companies in the FF facility footprint - Vulcan Materials, MP Materials, Newmont, Freeport-McMoRan, Alcoa, Nucor, Cleveland-Cliffs, Steel Dynamics, etc.
• Not included: Foreign private issuers filing Form 20-F (BHP, Glencore, Rio Tinto plc, Anglo American, Antofagasta, Vale, Yamana Gold, etc.). These need GLEIF L2 (LEI-anchored ownership) as the complementary path - see Workstream A.6.
• Not included: Private US companies with no SEC filing obligation. The FinCEN CTA Beneficial Ownership Registry (effective 2024) would close this gap for ~32M small businesses but isn't currently part of FF's source roster.

---

2. The artefact

One tarball:

• corpwatch_api_tables_csv.tar.gz - 1.5 GB compressed, expands to ~7.6 GB of CSV. Sourced from https://archive.org/download/corpwatch_api_data_dumps/.

Inside the tarball, 18 CSVs. The four we ingest in A.5 are bolded; the rest are either consulted opportunistically or ignored.

---

3. Records emitted per row (admitted)

For each admitted row the parser emits:

From companies.csv → one Entity per row (1.42M total):

• entity_id = "sec_cik:<cik>" when cik != 0 (~749K entities; the filer side); entity_id = "corpwatch_cw_id:<cw_id>" otherwise (~672K cw_id-only stubs from Exhibit 21 disclosures). CIK-anchored entities get confidence=MEDIUM; cw_id-only stubs get confidence=LOW (subsidiaries inferred only from a parent's EX-21, no independent SEC filing).
• canonical_name = company_name (CorpWatch's standardised name, upper-case for filers, mixed-case for relationship-derived stubs).
• jurisdiction - resolved via the precedence in the _jurisdiction helper:
  1. Filer entities: latest filers.incorp_country_code / filers.incorp_subdiv_code (preferring the most-recent filing per cw_id).
  2. Non-filer fallback: company_locations row with most_recent=1 for the cw_id, using country_code / subdiv_code.
  3. Default: ZZ when neither resolves.
• lei - none; CorpWatch doesn't emit LEIs (GLEIF L2 will join LEIs in via a separate fetcher in A.6).
• entity_type = "corporation" default; "natural_person" is not used (CorpWatch only carries companies, not directors).
• provenance - Provenance.fetched(source_id="sec_edgar_corpwatch", source_url="https://archive.org/download/corpwatch_api_data_dumps/corpwatch_api_tables_csv.tar.gz", fetch_date=...).

From company_relations.csv → one lineage Relationship per row (3.52M total):

• Direction flip. CorpWatch's source_cw_id is the parent; target_cw_id is the subsidiary (verified empirically via the relationships.csv join - see live-data observation #5 above). FF's convention is the opposite, so: from_entity = "<scheme>:<target_cw_id-resolved>" (the subsidiary) to_entity = "<scheme>:<source_cw_id-resolved>" (the parent) Each cw_id resolves to its CIK-anchored or cw_id-stub entity_id via the companies.csv lookup built during Entity emission.
• kind = "controls" for every row in this dump. The company_relations.csv relation_type column is 100% NULL, so there is no signal to discriminate owns_share. Stake-percent enrichment via relationships.csv.percent lands in the deferred A.5.6 follow-up.
• stake_percent = None (deferred to A.5.6).
• valid_from = date(year, 1, 1) from the year column on the relation row. Single-year resolution - CorpWatch doesn't anchor edges to filing dates with finer granularity.
• valid_to - left null. CorpWatch doesn't track when subsidiaries stop being subsidiaries; absence in a later filing is the signal, but A.5 doesn't materialise that.
• confidence = MEDIUM - the edge comes from a SEC-filed Exhibit 21 disclosure (anchored), with a small-but-real regex-parser fidelity caveat (see §"Quirks" below). The combine-with-LOW-stub-Entity → the resulting traversal hops through these edges still report weakest-link composite tiers honestly.
• relationship_id = mint_relationship_id() - fresh ULID; CorpWatch relation_id is preserved as a join key only via provenance, not as the FF-side identifier.
• provenance - Provenance.fetched(source_id="sec_edgar_corpwatch", source_url=..., fetch_date=...). Per-edge link back to a specific SEC filing accession is reachable via (parent_cw_id, cw_id, year) → relationships.csv.filing_id → filings.filename; deferred to A.5.6 when we'd want to surface that.

Not emitted in A.5:

• relationships.csv per-filing-quarter raw observations - collapsed into company_relations.csv already; A.5.6 will revisit when we pull stake percentages.
• company_locations.csv rows beyond the jurisdiction lookup - the per-cw_id address detail isn't part of the lineage record shape.
• company_names.csv - variant names; consumed by Phase 3 cross-source entity identity work.
• filings.csv / company_filings.csv / filings_lookup.csv - provenance back-link reference only; not emitted as records.
• SIC / UN reference tables - overlap with FF's facility-side NAICS / ISIC / ISO-3166 already.

---

4. Quirks

• Regex parser fidelity. CorpWatch parses Exhibit 21 with regular expressions over heterogeneous HTML/text. Most rows are correct; small-but-real error rate on:
  • Deeply nested subsidiary trees (the parser flattens; real structure is partially captured).
  • Non-English subsidiary names (Cyrillic, CJK characters in some foreign subsidiaries).
  • Edge cases where a subsidiary name contains a comma followed by "LLC" - the parser sometimes splits there.
• Subsidiaries usually have no CIK. Most subsidiaries don't file with SEC themselves, so CorpWatch invents a cw_id for them. Cross-source matching against MSHA / MRDS owner names happens via name + jurisdiction match (Phase 3 entity-identity work) - or via name + parent-CIK as a stronger anchor.
• Some filings produce CIRCULAR loops. The README warns:

  NOTE: Some groups of companies may form circular loops due to peculiarities of their filings (Same companies listed as parents and children), or because multiple CIKs refer to the same filing. The walk_ownership traversal already handles this - cycle terminus.

• Update cadence is uncertain. The Internet Archive snapshot last refreshed 2025-09-18. Future refreshes depend on whether archive.org continues to mirror CorpWatch's data. Don't pin Phase-3 SLAs to it.

---

5. Cross-source value

The expected wins from adding CorpWatch to the lineage substrate:

1. Resolves stub-grade entities to canonical anchors. MSHA's msha_legal_entity_id:0071891 ("Vulcan Materials Company") gets matched (Phase 3) to CorpWatch's sec_cik:0001396009 for the same company - promoting the entity from confidence=LOW (stub) to confidence=MEDIUM (CIK-anchored).
2. Adds multi-hop chains. MSHA's chain stops at "Vulcan Materials Company [apex]"; CorpWatch tells us Vulcan Materials Company has subsidiaries (Vulcan Construction Materials LLC + ~30 others). Walk_ownership's BFS picks these up automatically once both edges are in the same lineage tier.
3. Multi-jurisdictional. CorpWatch carries non-US subsidiaries of US-listed parents (Vulcan Materials' Mexican operations, Freeport's Indonesian + Peruvian assets, Newmont's Australian + Ghanaian operations). Combined with MRDS' non-US facility coverage, this lets dossiers for foreign mines render the parent chain back to the US-listed apex.