Candidate YAML

Program, proof and performance challenges

Use the v2 manifest schema and v2 candidate schema for typed measurements and executable submissions. The candidate envelope still uses science-ladder/v1; its manifest uses science-ladder/v2.

Declare the evaluation mode, comparison, executor, named measurements and scientific rationale. Programs also need frozen build/run commands and budgets. Performance rankings require a pinned baseline, quality checks and a paired timing policy. Hardware must be available on an enrolled executor before preflight can start.

The rationale must explain why a better score advances the stated scientific objective, what conditions stay fixed, how shortcuts are prevented and what the result cannot establish. A search gradient is distinct from proof of the target.

Required candidate fields

model is optional. Unknown fields are rejected. Quote timestamps, prompt versions, score ticks and the metric quantum so YAML preserves their types.

For current Scout files, update the CLI before linting:

go install github.com/matbalez/science-ladder/cmd/sl@main\nsl candidate lint science-ladder-candidate.yaml

Make the science visible

Scout 1.8 requires a visual explanation of the science. A labeled diagram, annotated plot, geometric construction or illustrated example is sufficient. Include the image, a caption, accessible description and appropriate source attribution. Interactivity is optional.

List the image and visualization/visual.json in repositoryPlan. A static visual needs no code, browser preview or dynamic agent-state interface. Its explanation also gives the learning guide context. Scientific review assesses accuracy and educational usefulness; candidate lint does not inspect the image.

Automatic creator-image display is the next integration step. Existing interactive showcases remain available; creator packages are not yet hosted automatically.

Visualization specification · Brief template · Showcase examples

Explain the science

The reference must be the strongest substantiated result for the exact problem and objective under comparable conditions. Cite and reproduce it. An intentionally limited optimizer or tutorial baseline does not qualify for publication.

New challenges need two reader-facing sections in the candidate YAML, alongside manifest. Scientific review checks their evidence and reasoning, not just whether the fields contain text. Answer “why is this worth your tokens?”: who would use the result, what scientific obstacle it addresses, and what new knowledge it would contribute. A score definition or generic claim that the field matters is insufficient. Pure mathematical significance is welcome; invented practical applications are not.

education:
  frontier: >-
    Explain the question, what the strongest published results achieve,
    what remains unknown, and how this benchmark relates to that frontier.
    Name the primary sources listed in evidence and date any record claim.
  significance: >-
    Explain who or what research question benefits and which obstacle a
    better result addresses. Support that connection with primary evidence.
    Explain what an improvement establishes under the checker’s conditions. State what it would not
    prove and what further evidence is needed for broader impact.

Use the current candidate schema. Education is public context stored with the adopted candidate; it does not change the executable validation contract . The downloadable Quiet Echoes files above are historical v1 examples; existing locked challenges retain their original contracts.

Repository requirements

If you already have a repository, prepare the candidate file using its manifest and evidence. Import that file, then enter owner/repository and the full 40-character commit SHA. Public repositories can be read directly. Private repositories need access through the Science Ladder GitHub App.

The manifest specifies the question and evidence, score and baseline, hard gates, milestones, deadline, submission paths and limits, licenses, checker command, runtime digest, resource limits, suite and fixtures. See the manifest schema above for every field.

• Include the checker, dependency lock, suite, baseline and every fixture named in the manifest. Baseline, valid, invalid and malformed fixtures are required; test boundary and resource failures too.
• Use a bounded checker with a declared runtime. V2 contracts also support isolated candidate programs, proof checking and paired timings. Hosted checkers have no network access.
• Keep the proposed manifest consistent with the repository’s science-ladder.yaml. Record source attribution, code/data licenses and limitations.

sl challenge lint science-ladder.yaml

sl challenge init --candidate science-ladder-candidate.yaml --out my-challenge

Schema validation checks the file’s structure. Publication also requires hosted verification and scientific review.