BrightQuery Legal Entities - landscape for Facility Foundry v1 onboarding

Matcher role (2026-05-11): additive contributor, not a primary anchor. Per the SourceProfile contract in synthesis/_source_profile.py, BQ is registered with:

• primary_for = frozenset() - BQ never claims authority for any identifier scheme. CorpWatch is primary_for={"cik"} (SEC EDGAR registry of record); GLEIF is primary_for={"lei"}.
• contributes_ids = {"cik", "lei", "bq_id"} - BQ records still participate in cik-key / lei-key / bq_id-key anchor passes as members, enriching those blocks. The canonicality rank's primary-for tier ensures CorpWatch beats BQ within cik blocks and GLEIF beats BQ within lei blocks.
• canonicality = 2 - below CorpWatch (4) and GLEIF (5) but above stub sources (MSHA / MRDS / EPA FRS / Wikidata at 1) - so when neither CorpWatch nor GLEIF is in the block, BQ wins canonicality over name-only stub entities.
• trusted_for_name_match = False - BQ entities skip the name-block pass entirely. BQ's bq_legal_entity_name strips disambiguating qualifiers (II / III / 2 / Series A) during ingest (e.g. SEC CIK 0001539297 is "LOUNGANI CAPITAL GROUP II, L.P." per SEC EDGAR but BQ records it as "LOUNGANI CAPITAL GROUP, L.P"), which previously caused false-positive name bridges between sequential fund vintages.

Net effect: BQ's CIK + LEI columns continue to deliver cross-walk value (LEI ↔ CIK identifier-anchored bridges fire normally), while the matcher avoids relying on BQ's name field. BQ remains queryable in the resolved tier for source-specific workflows (the planned LLM- classification feed reads BQ's bq_organization_website directly).

Phase 3 / Workstream A.8 landscape, written 2026-05-10 (live-data verified). Reference for the BrightQuery fetcher built against the v1 schema + ADR 010/011 lineage substrate. Cross-references: docs/adr/003-bq-as-source.md, docs/adr/010-lineage-records-share-canonical-pipeline.md, docs/adr/011-cross-source-entity-reconciliation.md, docs/LINEAGE.md, docs/DATASOURCES.md, docs/phases/02-multi-jurisdictional-and-lineage.md §"Workstream A.8".

Live-data observations (verified 2026-05-10 against the internal BrightQuery Vespa endpoint at http://136.118.253.76:8080). Discovery via the pyvespa client; the probe scripts that produced the numbers below live at tmp/sources/brightquery.com/discover_v{1..6}*.py (kept out of src/ - research artefacts, not shipped code). Pre-research notes in docs/DATASOURCES.md had the order of magnitude right ("~300M companies, ~500M places of business") but the Vespa deployment is a US-strong subset, not the full BQ corpus - 115.6M legal-entity / org × tax-year docs and 269.9M places of business in the deployed cluster.

1. Vespa cluster is bq_cluster, version 8.582.13. Single cluster, exactly five document types - no hidden tables. Total cluster doc count: 470,600,510.

2. Five document types with their cardinality:

   Doc type	Total docs	Sample-field count
   terminal_screener	115,645,747	120
   bq_executives	81,586,353	69
   bq_location_new	269,906,647	47
   bq_stock_event_timeline_tbl	1,979,218	9
   bq_stock_class_tbl	1,482,545	14

   terminal_screener is the legal-entity backbone - contains the cross-source identifier triple-bridge (CIK + LEI

   • bq_legal_entity_id), corporate hierarchy, financials, NAICS
   • IRS industry codes, address with lat/lon, and the website / LinkedIn fields that anchor the LLM-classification feed. bq_location_new is the places-of-business table that joins entities to their physical sites - the critical bridge from BQ entities to FF facilities.

3. Each terminal_screener row is one (organization × tax-year × time-slice) tuple, not one organization. Microsoft (CIK 0000789019) returns 91 hits; Apple (CIK 0000320193) returns 95. Altria returns 2; Vulcan Materials 26. Distinct organizations is materially less than 115.6M - back-of-envelope estimate ~10-20M distinct organizations after dedup on bq_organization_id. The fetcher dedups before emit; the Vespa visit / streaming flow uses (bq_organization_id, bq_global_legal_entity_id) as the natural key.

4. Cross-source identifier coverage on terminal_screener (denominator: 115,645,747 docs):

   Field	Populated count	Coverage rate	Notes
   bq_organization_cik	1,131,996	1.0 %	SEC filers (US public)
   bq_organization_lei	808,426	0.7 %	LEI registrants (GLEIF)
   bq_organization_ein	26,960,294	23.3 %	US tax-IDs (private + public)
   bq_organization_ticker	81,312	0.07 %	Publicly-traded
   bq_organization_website	20,855,286	18.0 %	LLM-feed candidate; biggest cross-source surface
   bq_organization_linkedin_url	17,585,713	15.2 %	LLM-feed candidate
   bq_isin	118,488	0.10 %	International securities
   bq_openfigi	61,019	0.05 %	Bloomberg figi
   bq_permid	0	0 %	Refinitiv field exists but empty

   Coverage is per-document-row, not per-distinct-entity - the actual entity-level coverage rates are higher (because Apple's 95 rows all carry the same CIK + LEI). For per-entity estimates assume average row-fanout ~5-10x, so distinct CIK-bearing entities ≈ 100K-200K; distinct website-bearing entities ≈ 2-4M.

5. CIK ↔ legal-entity-id round-trip works. Verified live:

   SELECT bq_organization_name, bq_legal_entity_id, bq_organization_lei,
          bq_organization_ticker, bq_organization_website
   FROM sources terminal_screener
   WHERE bq_organization_cik CONTAINS "0000789019"
   → MICROSOFT CORP (BQ_US_DE_6658187, LEI INR2EJN1ERAN0W5ZP974,
     MSFT, https://www.microsoft.com)
   

   Reverse direction (entity_id → CIK) is symmetric - same query with the WHERE clause flipped to filter on bq_legal_entity_id.

6. CIK + LEI + ticker triple-bridge confirmed on the same row. The Altria sample carries bq_organization_cik="0000764180", bq_organization_lei="XSGZFLO9YTNO9VCQV219", bq_organization_ticker="MO", plus bq_isin (multi-class array of CUSIP-derived ISINs) and bq_openfigi (Bloomberg figi codes). This makes BQ the single best CIK ↔ LEI bridge source - better than GLEIF (no CIK) or CorpWatch (no LEI) on their own.

Endpoint surface

The deployed Vespa container exposes the standard JDisc handlers:

• / - BindingsOverviewHandler; lists registered handlers + bindings.
• /state/v1/health → {"status": {"code": "up"}}.
• /state/v1/metrics - Vespa metrics.
• /search and /search/* - YQL queries via the SearchHandler.
• /document/v1/* - per-document GET/PUT/DELETE + visit sweep.
• /ApplicationStatus - application package metadata. Confirms Vespa 8.582.13 + the bq_cluster dispatcher binding.

The pyvespa client (vespa.application.Vespa(url=...)) wraps all of this with query() (YQL), visit() (streaming sweep), and get_application_status(). The FF fetcher uses pyvespa rather than raw httpx - Vespa's wire protocol is non-trivial and pyvespa handles connection pooling, paging, and error surfacing.

Document-type schemas

terminal_screener - legal-entity backbone (115.6M docs)

Identity fields (always populated):

• bq_legal_entity_id - FF's anchor scheme. Format BQ_<country>_<state>_<numeric> (e.g. BQ_US_VA_02675999, BQ_US_DE_6658187). Maps directly to FF's pre-declared SCHEME_BQ_ENTITY_ID = "bq_entity_id" constant.
• bq_legal_entity_name (e.g. "ALTRIA GROUP, INC").
• bq_legal_entity_jurisdiction_code - e.g. US_VA. ISO-2-style with state-subdiv suffix.
• bq_legal_entity_company_number - state filing number.
• bq_organization_id - separate organisation-level ID (synthesizes multiple legal entities into one parent organisation, like Altria Group + its 100+ subsidiaries).
• bq_global_legal_entity_id - global cross-cluster anchor.

Cross-source identifiers (sparse - see coverage table above):

• bq_organization_cik, bq_organization_lei, bq_organization_ein, bq_organization_ticker, bq_isin (array), bq_openfigi (array), bq_permid.

Hierarchy:

• bq_legal_entity_parent_status - 'Ultimate Parent' / 'Subsidiary'.
• bq_legal_entity_children_count, bq_legal_entity_children_ids (array of BQ_* entity IDs).
• bq_legal_entity_immediate_children_count, bq_legal_entity_immediate_children_ids.
• bq_legal_entity_immediate_establishment_count, bq_legal_entity_immediate_establishment_ids (children that are physical establishments rather than legal entities).

Address (registered + organisation):

• bq_legal_entity_address1_* - line, city, state, zip5, zip9, county, county_fips, lat, lon, CBSA code/name, RDI (commercial / residential), valid_indicator.
• bq_organization_address1_* - separate organisation-level address. Often differs (headquarters vs registered agent).

Industry classification:

• bq_organization_naics_code, bq_organization_naics_name, bq_organization_naics_sector_code, _sector_name.
• bq_organization_irs_industry_code, _irs_industry_name.
• bq_business_code, bq_industry_name (top-level).
• bq_organization_subsector_code, _subsector_name.

Public-market / status indicators:

• bq_organization_public_indicator - 'Public' / 'Private'.
• bq_public_indicator - boolean.
• bq_sp500_indicator - 'Yes' / 'No'.
• bq_organization_capital_markets_universe_indicator.
• bq_organization_active_indicator, bq_legal_entity_active_indicator.
• bq_organization_year_founded, bq_organization_date_founded.
• bq_organization_lfo - legal form (Corporation / LLC / etc.).
• bq_organization_company_type - Holding Company / Employer / etc.
• bq_organization_ofac_indicator - sanctions list match.

Financial fields (populated for SEC filers + LEI registrants):

• bq_revenue_mr, bq_net_income_mr, bq_gross_profit_mr, bq_operating_income_mr, bq_operating_expenses_mr, bq_total_assets_mr, bq_payroll_mr, bq_return_on_assets_mr, bq_return_on_sales_mr, bq_gross_profit_margin_mr, bq_net_profit_margin_mr, bq_revenue_growth_qoq_mr, bq_revenue_growth_yoy_mr.
• Per-employee derivatives: bq_revenue_mr_per_emp, bq_net_income_mr_per_emp, etc.
• bq_report_date_mr - most-recent reporting date.

LLM-feed candidates:

• bq_organization_website - corporate website URL (20.9M populated, 18.0% of docs, by far the richest LLM-feed surface).
• bq_organization_linkedin_url - LinkedIn company page (17.6M populated, 15.2%).
• bq_organization_alternative_names - array of name aliases (e.g. Altria carries ['jmc', 'altria group richmond', 'altria group inc', 'altria group', 'usstc']).
• bq_place_categories_all - semicolon-delimited place categories ("Tobacco supplier;Restaurants").
• bq_vertical_categories_all - vertical categories ("Cannabis, Manufacturing").
• bq_organization_name_array - token-split name (useful for matcher blocking).

bq_location_new - places of business (269.9M docs)

The entity → physical-location join. Each row is one location:

• bq_location_id - unique location anchor.
• bq_legal_entity_id, bq_organization_id - foreign keys back to terminal_screener.
• bq_location_address_line_1, _city, _state, _state_name, _zip5, _country_code, _country_name, _county_fips, _county_name, _cbsa_code, _cbsa_name, _rdi, _dpv_match_code, _mailable.
• bq_location_geo_position - lat/lon dict.
• bq_location_class, bq_location_type, bq_location_headquarter_indicator.
• bq_location_website, bq_location_website_domain - per-site website (e.g. a refinery's plant page distinct from the corporate website).
• bq_location_employment_mr, bq_location_revenue_mr - per-location financial signals.
• bq_location_confidence_score, bq_location_active_indicator.

This is the table the FF fetcher will use to bridge BQ entities to FF facilities via spatial-+-name match (ADR-008 style). Per-location lat/lon + website lets us ADR-008-merge BQ locations onto the existing 594K FF facility universe.

bq_executives - executives (81.6M docs)

People records keyed by bq_executive_id. Joined to entities via bq_organization_id. Carries:

• bq_executive_first_name, _last_name, _middle_name, _name, _highest_title, _titles (array), _officer_indicator, _signatory_indicator.
• bq_executive_address1_* (full residential / commercial address).
• bq_executive_linkedin_url.
• Joined organisation context: bq_organization_name, bq_organization_address1_*.

Out of scope for FF Phase 3 - the FF schema doesn't carry person records. Could become useful for ADR-011 entity-resolution if a future slice wants to anchor entities by their officer surface.

bq_stock_class_tbl (1.48M) + bq_stock_event_timeline_tbl (1.98M)

Stock-structure and corporate-event tables, keyed by bq_company_number. Out of scope for the initial FF fetcher - they're useful for tracking equity events (IPO, splits, M&A-driven entity changes) but don't carry the cross-source identifier bridge.

Coverage gap analysis vs existing FF fetchers

Once landed, BQ slots in alongside GLEIF and CorpWatch as the third lineage-fetcher anchor:

BQ's value-add for FF:

1. CIK ↔ LEI direct bridge. GLEIF and CorpWatch each carry one of these but not both. BQ carries both on the same row for ~100K-200K entities. This unlocks a transitive ID-completion pass: for any entity with one of CIK / LEI, look up the other on BQ and emit the missing same_entity_as HIGH bridge - closes a real gap in the v1.1 matcher's reach.

2. Non-SEC company coverage. Of 115.6M terminal_screener docs, only 1.13M have CIK populated (1.0%). The remaining 99 % are either private companies, non-US filers, or non-publicly-traded entities - the long-tail operator population that MRDS and EEA need to bridge to. Concretely, ~27M EIN-bearing entities means the entire US tax-registered universe (private + public + nonprofit) is in scope.

3. Website + LinkedIn URL anchor for LLM classification. 18.0% of docs carry bq_organization_website, 15.2% carry bq_organization_linkedin_url. These are the substrate for the parallel LLM-classification track - feed the website to an LLM with a ff_process_role taxonomy prompt and let it classify the facility's role. The OECD / worldsteel / USGS anchored taxonomy from the parallel research track gives the LLM a recognised label set to project onto.

4. Per-location website join via bq_location_new. The 269.9M places-of-business table carries per-location websites distinct from the corporate one. A copper smelter operator's corporate page is generic; the smelter's plant page often has process-role detail. ADR-008 spatial-name match between bq_location_new rows and FF facilities, then LLM-classify from the location's website, gives a clean per-facility signal.

5. EEA's operator-axis gap closes. EEA emits no operator field in the user-friendly CSV bundle (B.2.0). With BQ on-line, EU operator strings on EEA facilities can resolve to bq_legal_entity_id: anchors via a name-+-jurisdiction match - and from there to CIK / LEI as available.

Fetcher design

Cache strategy: NOT BULK_FILE. Per ADR 003, BQ is queried by identity / by entity_id traversal, not bulk-downloaded. The fetcher's contract is:

• Identity-resolve - given a name + jurisdiction (or a CIK / LEI / EIN / ticker), return the BQ entity record.
• Hierarchy-traverse - given a bq_legal_entity_id, return its bq_legal_entity_immediate_children_ids for one-hop traversal.
• Per-entity dossier - given a bq_legal_entity_id, return its full field set (for downstream LLM classification).
• Per-location scan - given a country / region / NAICS code, return matching bq_location_new rows for ADR-008 facility match.

Cache shape: per-entity JSON, SQLite-backed thread-safe key-value store keyed by bq_legal_entity_id. Mirrors the Wikidata fetcher's per-QID cache. TTL configurable per use case (operator resolution → long TTL; LLM-feed dossier → shorter TTL since BQ updates continuously).

Record kinds emitted:

• Entity - one per BQ entity admitted into the resolved tier (canonical_name = bq_legal_entity_name, jurisdiction = bq_legal_entity_jurisdiction_code mapped to ISO-2, entity_type derived from bq_organization_lfo, lei = bq_organization_lei when present, cik = bq_organization_cik when present). Scheme: bq_entity_id:<bq_legal_entity_id>.
• Relationship(kind="controls") - parent / subsidiary edges from bq_legal_entity_immediate_children_ids. Confidence = HIGH (BQ-asserted from filings + commercial intelligence).
• OperatorReference resolutions - the existing operator-name resolution pass can now bridge MSHA / MRDS / EEA operator strings to bq_entity_id: anchors.
• Optionally: FacilityIdentifier(scheme="bq_location_id", value=<id>) on facilities matched from bq_location_new via ADR-008. Pending design discussion - would FF treat BQ locations as facility-axis records, or only as bridge evidence?

Vintage anchor: the extracted_on field on each terminal_screener row (e.g. '2026-01-27 08:44:39') is the freshest signal. Pin per-fetch as vintage = max(extracted_on) across the queried subset.

Auth model: the v6 probe was unauthenticated and worked end-to-end. Production access may require API keys / mTLS / per- tenant scoping - confirm with the BQ data-engineering owner before scaling up beyond research probes.

Rate-limit envelope: unknown; the v1-v6 discovery probes together ran maybe 50 queries with no throttling observed. Given Vespa's per-cluster QPS sizing, the fetcher should default to modest concurrency (≤ 10 in-flight) and back off on 429 / 503.

ADR 011 matcher implications

BQ entities carrying both cik and lei slot directly into the v1.1 LEI-key matcher's HIGH bridge path. With BQ in the resolved tier:

• Wikidata's 819 LEI-bearing operators already bridge to gleif_lei: anchors (1,008 HIGH bridges as of v1.3).
• CorpWatch's 1.42M sec_cik: anchors gain direct LEI bridges via BQ for the 100K-200K entities that BQ carries with both IDs. The matcher emits new same_entity_as HIGH edges sec_cik:<cik> ↔ gleif_lei:<lei> for every such pair - closing a major gap in the matcher's reach.
• Non-SEC, non-LEI entities resolve via name + jurisdiction match against BQ (existing v1 name-block path; BQ's bq_organization_alternative_names array helps with name- variation coverage).

Matcher v1.2 onboards bq_id as a hard-anchor block-key (2026-05-10). The Entity model gains a bq_id: str | None cross-reference field parallel to lei; the matcher's hard-anchor-key pass (emit_anchor_key_edges, the v1.2 generalisation of v1.1's emit_lei_edges) blocks by bq_id in addition to lei, so any two entities sharing a BQ ID emit a HIGH same_entity_as bridge regardless of name + jurisdiction divergence. The two passes share seen_pairs so the same edge isn't emitted twice when an entity has both lei and bq_id populated. _canonicality_rank gains a bq_id is not None tier between LEI presence and confidence, so BQ-resolved entities edge out same-tier peers without BQ resolution.

This means once the A.8.2 BQ fetcher lands and starts populating bq_id on Wikidata / CorpWatch / MSHA / MRDS / EEA entities (via the matcher's name-block resolution + the BQ fetcher's identity- resolve pass), every cross-source pair sharing a bq_id value bridges HIGH automatically - closing the remaining gaps where two sources have the same BQ entity but their CIK / LEI / name don't all align.

Phase 3 / A.8 sub-step plan

• A.8.0 - Landscape research + doc (this file).

• A.8.1 - Fetcher skeleton + pyvespa wiring + auth model decision. BrightQueryFetcher registered under source_id="brightquery"; cache strategy = identity-resolve (NOT bulk file); pyvespa client config; --bulk-source flag for dev path against a snapshot.

• A.8.2 - terminal_screener parser → Entity + Relationship emit. Dedup on bq_organization_id; emit one Entity per distinct organisation; emit controls Relationship per parent-child pair. Confidence keyed off bq_legal_entity_parent_status ('Ultimate Parent' → HIGH apex).

• A.8.3 - bq_location_new parser → ADR-008 candidate stream. Stream locations into the synthesis-tier candidate set; let the facility-side matcher decide on HIGH / MEDIUM / LOW merges against existing FF facilities.

• A.8.4 - Live ingest + spot-checks (landed 2026-05-10; partial.) The fetch streamed 182,000 terminal_screener rows (1.1 GB NDJSON) before Vespa returned a 504 Gateway Timeout on offset-paging deeper than that. Salvaged the partial; parser deduped to 37,108 distinct legal entities (5x row fan-out as forecast in A.8.0). Synthesis ingested those alongside the existing 4.58M-entity universe.

  Headline numbers post-A.8.4 / matcher v1.3:

  Metric	Pre-BQ	Post-A.8.4	Post-v1.3	Delta vs A.8.4
  entity/LINEAGE	4,581,131	4,618,239	4,618,239	0
  lei_blocks (matcher)	3,056,380	3,056,475	3,056,475	0
  bq_id_blocks (matcher)	0	37,108	37,108	0
  cik_blocks (matcher, v1.3)	-	-	757,543	new
  LEI-key HIGH bridges	787	2,502	2,502	0
  CIK-key HIGH bridges	-	-	29,053	new
  bq_id-key HIGH bridges	-	0	0	nothing else carries bq_id yet
  ENTITY_ALIAS total edges	391,825	419,151	426,622	+7,471
  HIGH same_entity_as (all paths)	54,748	77,296	85,961	+8,665

  Matcher v1.3 (commit 9ebcf30) onboards cik as a hard-anchor block-key alongside lei and bq_id. All 29,053 CIK-key HIGH edges go from bq_entity_id: → sec_cik: (BQ ↔ CorpWatch bridge), exactly as predicted: BQ has 36,937 distinct CIK-bearing entities, CorpWatch has 749,488, the intersection is 28,882 (matches the 29,053 emitted edge count up to per-CIK fan-out from multi-legal-entity organisations on BQ's side).

  22,548 LEI-key HIGH edges connect bq_entity_id: ↔ gleif_lei: via the LEI-key path (unchanged from A.8.4).

  The bq_id-key path still emits 0 HIGH bridges because no non-BQ entity carries bq_id yet - the matcher only fires when ≥ 2 entities share a bq_id value. Backfilling bq_id onto cross-source entities (Wikidata / MSHA / MRDS / EEA stubs) would light it up; a sibling synthesis pass that joins BQ rows on shared CIK/LEI is the unlock.

  CIK ↔ LEI direct bridge is now reachable transitively through BQ. Altria's three anchors - sec_cik:0000764180 (CorpWatch), gleif_lei:XSGZFLO9YTNO9VCQV219 (GLEIF), bq_entity_id:BQ_US_VA_02675999 (BQ) - form a HIGH-confidence triangle: BQ↔CorpWatch via CIK-key, BQ↔GLEIF via LEI-key. The CIK↔GLEIF leg is reachable as a 2-hop pivot through BQ at LineageLookup time (no separate edge needed).

  Microsoft + Apple didn't make the 182K cut. Vespa offset- paging timed out before reaching their alphabetically-later rows. The full ingest needs the Vespa visit API (continuation-token-based streaming) instead of offset paging

• a small refactor to _vespa.py:stream_yql. Carried as the primary follow-up before A.8.5.

Filed against this slice - A.8.4.x carry list:

1. _vespa.py - switch from offset paging to the visit API so deep streams don't 504 on Vespa-side timeout. Landed 2026-05-10 - visit_documents() helper added to _vespa.py and wired through the fetcher. Uses /document/v1/<ns>/<doctype>/docid/ with continuation tokens. Live throughput on the deployed cluster with the SEC-filer selection (~1 % selectivity): ~5-40 docs/sec at wdc=10, variable per page (server-side visitor timeout caps each request). Operator runs the full ingest as a long-running background job (~7-60 hours wall-clock for the SEC-filer subset, depending on visitor stalls). See _vespa.py:108-145 for the throughput note.

2. Cross-source ID enrichment pass - a synthesis-tier module that uses BQ rows as the hub to backfill missing CIK / LEI / bq_id fields on cross-source entities. Promotes the bq_id-key matcher path from 0 → ~tens-of-thousands of HIGH bridges and unlocks the CIK ↔ LEI direct bridge for entities BQ has resolved against both.

   Landed 2026-05-10 - verified end-to-end: matcher v1.3 (commit 9ebcf30) onboards cik as a hard-anchor block-key alongside lei and bq_id. BQ + CorpWatch parsers now populate the new Entity.cik field (zero-padded to 10 digits to match CorpWatch's sec_cik: suffix), so a CorpWatch entity and a BQ entity sharing a CIK get a HIGH same_entity_as bridge directly. 29,053 BQ↔CorpWatch HIGH bridges emitted on the 37,108-entity partial cut (synth_037f492ee419). With BQ now bridging both CorpWatch (via CIK) and GLEIF (via LEI), the CIK↔LEI leg is reachable transitively through BQ - no separate LEI-onto-CorpWatch enrichment pass needed. The bq_id leg remains open: backfilling bq_id onto Wikidata / MSHA / MRDS entities would light up the bq_id-key matcher path.

3. bq_location_new ingest - same offset-paging blocker as terminal_screener; fixed by the visit-API switch above. Operator can now run the bq_location_new visit alongside the terminal_screener visit.

4. Throughput optimisation (future). The visitor's per-page latency is highly variable (1-175 s on the deployed cluster); candidates: predicate-slicing the corpus (e.g. by jurisdiction code) to fan out parallel visits, or YQL keyset paging on a sortable attribute if BQ exposes one. Both worth probing once the live ingest validates the rest of the pipeline.

• A.8.5 - LLM-classification feed. Use bq_organization_website + bq_location_website as the LLM prompt input; project to the OECD / worldsteel / USGS-anchored ff_process_role taxonomy from the parallel research track.

What's still TBD before A.8.1

• Confirm auth model. Public probes worked unauthenticated but production access likely needs an API key / VPN / mTLS - confirm with BQ data-engineering owner.
• Vintage / refresh semantics. Is the deployed cluster a point-in-time snapshot, or does it auto-refresh? The extracted_on field is per-row; the cluster-level version isn't visible from /ApplicationStatus.
• Rate-limit envelope. Per-tenant QPS / per-day budget.
• bq_organization_id vs bq_legal_entity_id semantics. Some samples show one organisation having multiple legal entities (Altria's children chain). Confirm which is the right anchor for FF's entity_id (probably bq_legal_entity_id since it's jurisdiction-anchored, but the matcher needs bq_organization_id for grouping).
• License / redistribution terms. Internal corpus per ADR 003; FF's emitted Entity records carry Provenance with source_id="brightquery" but the underlying SEC / IRS / state filings are public-domain - citation should point at the primary filing where possible, with BQ noted as the access path.

Discovery probe history

The v1-v6 probe scripts at tmp/sources/brightquery.com/ are research artefacts (not shipped code):

• discover.py - first probe; surfaced the 5 doc types via hits=3 wildcard.
• discover_v2.py - broader 500-hit sweep + candidate-table guesses; confirmed schema is exactly 5 types.
• discover_v3.py - full source-ref enumeration via error message; confirmed total cluster doc count 470M.
• discover_v4_terminal.py - deep terminal_screener field profile (private LLC samples first).
• discover_v5_public.py - public-company filter; surfaced CIK + LEI + website fields.
• discover_v6_coverage.py - CIK round-trip verification + per-field coverage rates (the table above).

Raw response JSON is in tmp/sources/brightquery.com/responses_v{1..6}/ for cross-checking against future schema changes.