Shard schema

Topic shard · 0.3.0

A curated topic landing produced by the auto-topic pipeline. Carries the display name, neutral description, date boundary, headline counts per record type, per-actor record counts (the mostActiveActors array), most-recent records, yearly timeline, and provenance for the topic itself (embedding model, labeling model, review timestamps).

This page is the field reference for developers and AI clients. If you came here looking for a topic, browse the topic index instead.

Draft field reference. Canonical .json file not published.

Verified example pending /schemas/topic@0.3.0.json (pending) Authoritative source Use with AI

Required fields

Optional fields

Dialect

JSON Schema draft 2020-12

Version

0.3.0

Required fields

Every shard validating against topic@0.3.0 ships these fields. Provenance lines show where each value comes from — the load-bearing column for the structured-evidence promise.

slug string<slug>

URL-safe topic identifier.

Provenance: Derived from displayName by the auto-topic pipeline; stable once published.

e.g. nhs-dentistry
displayName string

Human-readable topic name.

Provenance: Generated by topic-labeling LLM (decision #18); reviewed and merged by operator via GitHub PR.

e.g. NHS dentistry
neutralDescription string

Neutral, ≤500-char summary of what the topic covers.

Provenance: Generated by topic-labeling LLM under NEUTRALITY_POLICY.md constraints; reviewed and merged by operator.

e.g. "Records concerning the National Health Service dental service in England, Scotland, Wales, and Northern Ireland, including access, workforce, and contract reform."
dateBoundary string

Inclusive date floor for records assigned to this topic.

Provenance: Configured per topic by operator at publish; usually the start of the current Parliament.

e.g. 2024-07-04
headlineCounts array<object>

Counts per record-type for the at-a-glance band.

Provenance: Aggregated over records assigned to this topic, grouped by record type.
Nested fields (2)
- recordType enum (7 values)
  
  Record-type bucket the count applies to.
  - written_question
  - written_statement
  - oral_contribution
  - oral_evidence
  - division_vote
  - bill
  - committee_report
  Provenance: Determined at intake from the source endpoint that supplied the record.
- count integer
  
  Number of records of that type assigned to this topic.
  
  Provenance: Count over the topic-assignment table at shard-build time.
provenance object

Topic-level provenance — pipeline run that created this topic.

Provenance: Emitted by auto-topic pipeline at GitHub PR creation time.
Nested fields (5)
- embeddingModel string
  
  Embedding model + version used to cluster records.
  
  Provenance: Pipeline config; pinned per run.
  
  e.g. BAAI/bge-small-en-v1.5
- clusterId integer
  
  Stable HDBSCAN cluster identifier.
  
  Provenance: Centroid-matched against the previous run.
- labelingModel string
  
  LLM that produced displayName + neutralDescription.
  
  Provenance: Pipeline config; per-run pinned.
  
  e.g. gpt-4-turbo-2024-04-09
- detectedAt string<ISO 8601>
  
  Timestamp the topic candidate was emitted.
  
  Provenance: Pipeline run timestamp.
- approvedAt string<ISO 8601>
  
  Timestamp the operator merged the candidate PR.
  
  Provenance: GitHub merge event.

Optional fields

Present where the source carries the value or the derivation succeeds. Code consuming this schema should treat them as nullable.

mostActiveActors array<object> · optional

Up to 25 actors with the most records on this topic.

Provenance: Ranked by record count per entity over the records assigned to this topic.
mostRecentRecords array<object> · optional

Up to 50 most-recent records assigned to this topic.

Provenance: Records assigned to this topic, ordered by published date, limit 50.
timelineYearly array<object> · optional

Count of records per calendar year.

Provenance: Aggregated by year over the published-at field of records assigned to this topic.

Related schemas

Entities referenced by this shard kind. A topic shard's entities[] / nested references resolve to shards of these kinds.

Versioning + dialect

Schema id: topic@0.3.0
Dialect: https://json-schema.org/draft/2020-12/schema
Canonical .json: /schemas/topic@0.3.0.json (pending)
Verified example: Not linked until a verified live example is available.

Each planned schema follows semver. Patch bumps are additive and backwards-compatible. Minor bumps may add required fields or change validation rules. Major bumps will use a new URL path once canonical files are published.

Published structured data will declare its version in a $schema field. AI clients should respect that version when files become available.