Skip to content

Submission bundles

A bundle is one flat directory for one Defuser capability fingerprint and one measured target. submission.yaml uses submission schema version 4 and requires both protected-content digests in manifest provenance.

Interactive bundle layout
output/submissions/
  YYYYMMDD_<display-slug>_<fingerprint8>_<target>_<revision>/
    submission.yaml
    suite.lock
    experiments.parquet

Static bundles replace suite.lock and experiments.parquet with metrics.json.

suite.lock is a bundle suite snapshot from configs/suites/suites.lock: it contains one frozen suite revision and exactly the mission bodies that revision references. Submissions CI later compares this snapshot with the suite registry in the verified GPTNT release.

Manifest

The measured block discriminates the two bundle shapes. A suite identity selects an interactive bundle. A static task and dataset identity selects a static bundle.

Submitter pydantic-model

Bases: BaseModel

Who is submitting.

Written blank on build. The submitter fills these in and CI checks they are non-empty.

Attributes

name class-attribute instance-attribute 

name: str = ''

contact class-attribute instance-attribute 

contact: str = ''

@github handle or email.

affiliation class-attribute instance-attribute 

affiliation: str | None = None

SubmissionPlayer pydantic-model

Bases: BaseModel

One player in a submission, with role, full capabilities, fingerprint, and attribution.

fingerprint == capabilities.fingerprint, so it is a computed field and serialised into the manifest for readability. identity is the model's leaderboard from its player config.

Attributes

role instance-attribute 

role: PlayerRole

capabilities instance-attribute 

capabilities: PlayerCapabilities

identity instance-attribute 

identity: PlayerIdentity

Leaderboard attribution loaded from the player configuration for these capabilities.

fingerprint property 

fingerprint: str

The capability fingerprint, serialised alongside the entry.

Methods:

for_role classmethod 

for_role(role: PlayerRole, capabilities: PlayerCapabilities) -> Self

Build a role-tagged entry, resolving the model's leaderboard identity from its config.

A submission must be attributable, so a model with no identity block is a hard error.

InteractiveSubmission pydantic-model

Bases: Submission[SuiteIdentity]

submission.yaml for an interactive suite.

StaticsSubmission pydantic-model

Bases: Submission[StaticsIdentity]

submission.yaml for a statics evaluation task.

The Defuser is first in players. Each entry combines role, recorded capabilities, a recomputed capability fingerprint, and leaderboard attribution loaded from the player configuration. submission_id and the directory name are derived from run date, player, target, and fingerprint.

Interactive payload

experiments.parquet contains one SubmissionExperiment row per selected execution:

SubmissionExperiment pydantic-model

Bases: ExperimentSummary

One experiment in a submission.

This is one row in the output bundle and made of the whole ExperimentSummary plus outcome truth and per-player usage.

Attributes

final_bomb_state instance-attribute 

final_bomb_state: BombState

Terminal bomb state used to validate the outcome and mission pairing.

defuser_usage instance-attribute 

defuser_usage: RunUsage

Sum of Pydantic AI usage recorded by defuser steps in this execution.

expert_usage instance-attribute 

expert_usage: RunUsage | None

Sum of Pydantic AI usage recorded by expert steps in this execution.

pairing_key property 

pairing_key: SubmissionPairingKey

(defuser fingerprint, expert fingerprint, mission key) tuple to identify this run.

pairing_description property 

pairing_description: str

Human-readable description of the pairing, for reporting.

Methods:

from_summary classmethod 

from_summary(*, summary: ExperimentSummary, final_bomb_state: BombState, usage_by_role: dict[PlayerRole, RunUsage]) -> Self

Extend an ExperimentSummary with its final bomb state and each player's usage.

The row extends ExperimentSummary with the terminal bomb state and usage summed separately for the Defuser and Expert. It does not contain complete step trajectories.

Validation checks the reduced lock identity and digest, exact mission snapshot, one valid run per expected player pairing and mission, player fingerprints, provenance, bundle naming, and payload shape without reading live suite or mission configuration.

Static payload

metrics.json is copied verbatim from the task output. The manifest supplies the task, Hugging Face repository and split, requested and resolved revisions, player, capabilities, run date, and benchmark provenance.