Guide
The challenge compiler
How to describe a problem, turn it into a fair test, and check a certificate, including what each step can't do. This page is generated from COMPILER.md in the repository, so it matches the code. It is the technical reference: the later sections (counted rounds, the restricted subset, derived claims) assume you know the basics, and the plain-words version is on Other claims.
If you read only this
- You write three small pieces: make a test question from a secret number, check an answer exactly, and a plain baseline solver.
- compile_domain.py check NAME takes seconds and tells you what to fix. compile NAME --quick gives a draft in minutes. compile NAME is the real one and can take hours.
- A certificate lets anyone recompute a result, but only shows the costs were honestly measured if the verified sandbox or a re-run backs it. Checking one never runs its code unless you pass --execute.
- Past speedups: a cost can be an exact instruction count that anyone reproduces, and a certificate can back a benchmark score or a verified negative result. These are newer and have never been run: see Other claims.
You declare a problem. The compiler turns the declaration into a verified, calibrated challenge: an instance family with a size ladder, a CPU cap and an instance count chosen from measurements, a power analysis saying which speedups it can and cannot certify, a self-tested pipeline, a server plan, and, afterwards, a certificate anyone can check.
Everything downstream of the declaration is the protocol that already exists (commit-reveal seeds, the difficulty-stratified lower bound, the hash-chained log, the sandbox). The compiler writes no new trust rules; it automates the part that used to take a week of hand calibration per domain (PROTOCOL.md, "Power at the official configuration"), and it refuses to emit a challenge it cannot validate.
Quick start
python3 compile_domain.py init mything # a WORKING example in domains/mything/ (it already passes check); never overwrites
# ... edit the lines marked EDIT: derive(), verify(), DOC, SIZE_RANGE, and the reference solver ...
python3 compile_domain.py check mything # seconds: what is wrong, and how to fix it (shape, determinism, anti-cheat, reference)
python3 compile_domain.py compile mything --quick # 1-2 minute DRAFT ladder, for iterating (practice only)
python3 compile_domain.py compile mything # the real compile + self-test: tens of minutes to hours; prints READY / NOT READY
python3 compile_domain.py show mything # that verdict again, with the exact next commands
python3 compile_domain.py recheck mything # seconds: recompute every checkable claim in the compiled record and compare (see "Checking a compile")
python3 compile_domain.py status # every declaration: registered, self-tested, or stale (and why)
python3 submit_check.py my_solver.py --family mything
Every command takes the NAME or a path (domains/mything). compile_domain.py --help prints this workflow. init --blank gives an empty skeleton instead of the working example.
A compiled domain is a family like gm, mlwe and factor: judge.py, official.py, ops.py, the portal and the site all accept --family mything. domains/dlog/ is a worked example the compiler was not written for (discrete logarithm by exhaustive search).
The declaration (domains/<name>/decl.py)
| name | meaning |
|---|---|
NAME | 2-16 lowercase letters, equal to the directory name (ladder files are recognised by that pattern) |
VERSION | any string; bump it whenever derive() changes (it is recorded in every results file and refuses to compare across versions) |
SIZE_RANGE, SIZE_STEP | the sizes the compiler may use; it finds the ladder inside this range |
REFERENCE_SOLVER | a file inside the directory with a top-level solve; every submission is measured against it |
derive(master, size, idx) | a pure function of its arguments returning {"args": [...JSON...], "planted": <a valid answer>} |
verify(inst, ans) | exact check; must reject everything that is not a real answer |
witness(inst) | optional: the answer to feed verify when planted is not itself an answer |
DOC, SIGNATURE | optional text for the Compete page and the challenge document |
REFERENCE_NOTE | one or two sentences: what the reference algorithm is and what is known to beat it. Strongly advised: check warns without it, and the challenge document, the domain card and the compiled record say "not stated" |
The solver is called as solve(*args) and returns an int, a list of ints, or None (the runner's contract, unchanged).
decl.py is operator code. It runs in the judge's process exactly like instances.py does. Submissions never see it: they run in the sandbox with only runner.py and their own file. Do not compile a declaration you did not read.
The passes
- parse. The declaration has the required names;
NAMEis not a built-in; the reference lints clean (an import the sandbox image lacks is a FAIL, because the baseline is measured there) and imports none of the judge's modules. - validate. Each is a PASS / WARN / FAIL with the reason, and any FAIL stops the compile and writes nothing:
- shape:
argsis a JSON-serializable list and under 1 MiB (it crosses a process boundary); - determinism: the same
(master, size, idx)gives the same instance; different indices and different seeds give different ones (an instance that ignores the seed would make the commitment commit to nothing); - process purity: the same instances come out of fresh interpreters under two different
PYTHONHASHSEEDs (catches hash-order, clock and global-randomdependence, which would make another machine derive different instances); - witness:
verifyaccepts the planted witness at three sizes, and the witness (or a large part of it) is not in the arguments; - junk rejected:
verifyrejects 19 junk answers (None,0,1,-1, strings, empty lists, ...). A verifier that accepts1has an answer space a solver can guess; - no trivial answer: no argument, neighbour of an argument, or the argument list verifies, so the answer is not read off the question;
- witness reuse: one instance's witness is not accepted by other instances (otherwise a lookup table of past answers solves the challenge, the "egg" problem in EGG.md);
- reference solves: the reference solves the smallest instances with an answer
verifyaccepts, deterministically.
- shape:
- profile. The reference is run, locally, on a public profiling seed (never a competition seed) at growing sizes until its cost passes a third of the probe budget: a calibration profile of median cost, per-instance spread (sd of log2 cost) and timeout rate.
- optimize. From the profile: the CPU cap (30, 60, 120 or 240 s, the smallest that gives a usable ladder), and 3-6 sizes whose predicted median costs run from 2 s up to a point where about 10% of instances time out, spanning at least three octaves (the estimator's difficulty bins are octaves of baseline cost above 0.5 s). The chosen sizes are then measured (8 instances each) and the ladder is re-chosen if a prediction was off by more than an octave.
- power. The real estimator (
extrapolate.measured_by_difficulty, throughpower.synth_trial) on synthetic ladders drawn from the profile, with a synthetic submission that is exactlyebits faster. The compiler picks the smallest instances-per-sizen(12, 24, 48, 96 or 192) for which all three seeds are credited at >= 1 bit with probability >= 80% at a 2-bit true speedup, then tabulates power by speedup. Output: the range of speedups the challenge can certify and the zero-effect false-credit rate. - emit.
compiled.json, plusout/CHALLENGE.md(local kit),out/server_plan.json(theops.pycommands and CPU-hours),out/ci_check.shand a CI workflow template,out/CERTIFICATE.md. - self-test. The compiled domain is run through the real pipeline (
judge.py evaluate-> harness ->extrapolate.py) on a throwaway seed in local mode, with three controls:- zero effect: the reference submitted against itself must earn < 1 bit;
- failure: a solver that never answers must earn 0;
- known effect: the same ladder with every solved cost scaled by 2^-1 must be recovered (a credit between 0.4 and 1.3 bits). If any control fails the domain's record says so, and
rounds.create_roundrefuses an official round for it. The control ladders are kept inresults/(namedcontrol_zeroandcontrol_none, so do not give a real submission those names: the site's Results page leaves them out on purpose, as they are controls, not results).
What registers a domain, and what un-registers it
families.py calls declared.load_all() at import, so every process sees the same registry. A domain registers only if compiled.json is fresh: it records the SHA-256 of decl.py and of the reference solver, and a mismatch (any edit to either) un-registers the domain until it is compiled again. compile_domain.py status lists the reason. A round's rules fingerprint (log entry 0) includes the hashes of decl.py, compiled.json and the reference, so the declaration cannot change under a running round without the evaluator refusing, and a certificate can show which declaration its numbers came from.
Submitting and verifying from the command line
svp.py (guide: SUBMIT.md) is the user-facing path: submit a solver to a round, status, and verify --round R --id I, which downloads the certificate described below and checks it against the server's own log and your own receipt. svp.py practice SOLVER --family F certifies a local run on your own seeds, no server needed. ops.py certs builds the certificates after the reveal.
Provable costs (counted.py, count_runner.py)
The one thing a certificate cannot prove by checking is that recorded timings are what a real run took. Step counting replaces the clock with a number anyone can reproduce exactly: the count of Python bytecode instructions a solver executes (sys.monitoring, Python 3.12+; imports, the garbage collector and the hash seed are controlled, so for a deterministic solver nothing about the machine enters the count; it still depends on the Python version and build, and a solver that depends on memory addresses, the clock or randomness counts differently between runs, which the determinism self-check refuses to certify). counted.py evaluate writes ladders in the ordinary format (cost = count / 1e6, a unit called a "counted second", plus the exact steps of every record), so the estimator, certificate.py build and the site read them unchanged. A counted certificate carries count (unit, interpreter, its own size ladder), verify adds a cost model check (every cost follows from its step count; every unsolved instance is charged the cap; every header agrees), and --rerun K --execute re-executes K instances of both solvers and demands identical counts and statuses: exact equality, no tolerance, no trust in anyone's machine, sandbox or clock. On a different Python version the re-run is a SKIP. The counting program is pinned too: every counted ladder and certificate records the hash of count_runner.py (it is not one of the protocol's hashed files), and the cost-model check fails if this checkout's differs, so counts from another counter are named as such and a later fix to the counter cannot silently change what an old certificate means (it invalidates it: re-measure with the new one). Limits, stated in the verdict: it is not a time; it cannot see work inside a C function (one instruction however long); the ladder is chosen by whoever ran it, so it is not an official round result (unless a counted round declared it, below); and it needs Python 3.12+ and pure-Python solvers. It is not wired into the compiler's calibrated challenges (those are in seconds). svp.py practice --count is the way to produce one of your own; see SUBMIT.md.
Counted rounds
ops.py round-new ... --count --count-dims 38 40 42 [--count-n 8 --count-cap 20] creates a counted round: the sizes, the instances per size, the cap, the unit, the Python version and the hash of the counting program are committed in log entry 0 (with count_runner.py and counted.py in the rules fingerprint) before any submission, so the ladder is no longer the prover's: it is the round's, the same for every participant. Source publication is forced (the proof is re-execution), and only pure-Python families are allowed (counted.family_problem: not gm or mlwe, whose baseline is fpylll). ops.py worker --unsandboxed evaluates it: no Docker, the solvers run as the operator's user under step counting, so use a throwaway user or VM; the worker refuses another Python version or counter than the round committed to (even with --accept-code-change). A solver that cheats by reading the operator's files behaves differently when anyone re-runs it, so the first re-execution exposes it. The status of a counted result is COUNTED (3 committed seeds, commitments public first, one solver and one baseline file throughout) or COUNTED, PROVISIONAL with the reasons; it is not judge.py's OFFICIAL, which means "timed in the Docker sandbox". The certificate's log check ties the counted ladder to entry 0 (a counted ladder in a round that did not declare it, or the reverse, fails), and the cost-model line then says "ROUND-DECLARED". Every record must carry a step count a re-run can reproduce (a wall-clock timeout cannot, so a certificate containing one does not verify), and the unit must equal this checkout's.
The restricted subset (what a counted solver may be). The solver runs in the SAME process as the counter, so unrestricted it could switch its own counting off (sys.monitoring.set_events(3, 0)), walk up to the runner's stack frame that holds the counter (sys._getframe, __traceback__, gi_frame), forge a result line on the real stdout and _exit, or read files: and a re-run would reproduce all of it identically, so the "proof" would pass. count_runner.py therefore enforces a restricted subset itself, before and while the solver runs (check_source, the same check rounds.submit, practice --count and svp.py submit make early):
- an AST whitelist: no dunder or underscore names or attributes, no frame/generator/traceback/code attributes (
f_,gi_,cr_,tb_,co_), nostr.format("{0.__class__}"), noeval/exec/compile/open/getattr/setattr/vars/globals/type/object/super/id, no async code, no relative imports; - a minimal
__builtins__(arithmetic, containers,print, the common exceptions) and a guarded__import__; - imports only from
math,itertools,heapq,bisect,functools,collections,fractions, and each import yields a PROXY exposing only named functions (a real module would leak others:fractions.sys,collections.abc.sys); sys.monitoringis removed before the solver loads, a final check fails a run whose counting was reconfigured, and, on Linux only (on macOS the limit would trip over the process's own address space), the process has a 2 GiB address-space limit;- the module body is counted as well as
solve()(otherwise precomputation at import time, such as a lookup table, would be free, and a solver could spin there for ever).
The rules live in count_runner.py, so the counter hash pins them: a different rule set is a different counter. The baseline must be inside the subset too (counted.family_problem). A pure-arithmetic solver needs nothing outside it (solvers/factor_trial_py.py and factor_trial_fast_py.py are inside). What it is NOT: a defence against bugs in CPython itself, against memory or time spent inside ONE C call (10**10**10: the instruction limit cannot interrupt it, the wall-clock backstop does, and that submission then fails to verify), or a replacement for running the worker as an unprivileged throwaway user (it refuses root). Source stays mandatory for counted rounds. Counting a round costs about 100x a timed run per instance: size the ladder small (the worker prints a worst case).
Derived claims (claims.py)
The machinery proves one atomic statement: program P, on inputs committed before P was seen, produced these outputs at this cost. A speedup is one reading of it. claims.py reads three others off a certificate that verifies, recomputing each from the raw per-instance records (a claim file is not trusted either: claims.py check CLAIM CERT... redoes it and demands identical output):
claims.py correctness CERT [--out F] a benchmark score: solved/total on the committed questions (per size, per seed, Wilson 95% lower bound), for the solver AND the baseline
claims.py worst-case CERT [--out F] COUNTED certificates only: the largest step count seen per size, and the chance a fresh instance exceeds it (<= 1/(n+1))
claims.py negative CERT... --threshold BITS [--out F] "no submission in this round reached BITS": needs a certificate for EVERY submission the round's log accepted
What each does NOT show, printed with every claim: a correctness score is a rate on THIS test set within the cap (with timed costs "within the cap" depends on the measuring machine; a counted certificate makes it reproducible by anyone); the worst case is the largest value SEEN, not a bound over every input; a negative result is about the submissions made (participants chose to submit) and about what could be certified (credit is a conservative lower bound: the minimum over seeds, an inconclusive seed counts as 0), so it says "no speedup of that size was shown", not "none exists". A submission whose evaluation failed has no certificate: it is listed as missing and the negative claim is then incomplete and does not hold. The log must also contain the round's round_closed entry with no submission after it (a PREFIX of a hash-chained log still verifies, so certificates built from a truncated copy would otherwise hide later submissions), and the claim records log_head: compare it with a head hash published somewhere the operator cannot edit. The test set is the family's generator under committed seeds, so this is a benchmark of those problem families (factoring, dlog, ...), not of arbitrary data; a new benchmark is a declared domain (compile_domain.py). Tests: test_claims.py (unrun).
Certificates (certificate.py)
python3 certificate.py build --family F --submission solvers/x.py --seed-dir S --reveal-seeds \
--round-data D --round-id R --out x.cert.json
python3 certificate.py verify x.cert.json [--rerun 5]
A certificate carries the claim; the rules fingerprint; the commitments and (with --reveal-seeds, which is the reveal) the seeds; both solvers' source; every raw per-instance record of every ladder; for a declared domain its three files; and the round's log. verify, on any machine with a checkout of this protocol, checks and reports each as PASS / FAIL / SKIP:
| check | what it proves |
|---|---|
| rules | this checkout's estimator, harness and instance code hash exactly as the certificate names |
| domain | the declaration, compiled ladder and reference are the hashed files, load, and are fresh |
| log | the hash chain verifies; entry 0 holds the commitments and the same rules fingerprint (including the domain files); the submission's hash was logged |
| seeds | each revealed seed hashes to a commitment |
| sources | the solver sources are what every ladder header's solver_sha256 names |
| ladders | one baseline/submission pair per committed seed (a dropped seed would raise the minimum), each on exactly the rules' size ladder with equal instance counts per size (a dropped hard size would otherwise hide failures) |
| coverage | re-deriving every instance from the seeds gives exactly the ids in each ladder: none missing, none repeated, none foreign |
| credit | the credit recomputed from the raw records, with this checkout's estimator, equals the claim exactly |
| rerun | (optional) K instances re-measured on the verifier's own machine: both solvers still solve them and the speedup reproduces within 2 bits |
**A certificate is untrusted input, so verify executes nothing from it by default.** A declared domain's decl.py is hash-checked and read as data (its coverage check is then SKIPPED), and --rerun is refused. --execute imports the certificate's decl.py and, with --rerun, runs its solver sources on your machine, unsandboxed: use it only in a throwaway VM or container. Built-in families derive instances with your own checkout's code and need no flag for coverage.
verify --rerun K picks the same K instances every time unless you give --sample-seed; an independent verifier should choose its own seed so that the operator cannot know which instances will be re-measured. --tolerance BITS sets how far the re-measured speedup may differ (default 2 bits: loose on purpose, because the real machine-to-machine spread has not been measured; see "Checking a compile" for how to measure it).
Without a log a certificate can still verify but is at most PROVISIONAL (the publication time of the commitments cannot be shown); a ladder measured in local mode is PROVISIONAL regardless (only the Docker sandbox meters untrusted code). verify says which.
Checking a compile (recheck)
python3 compile_domain.py recheck dlog # about 5 seconds
python3 compile_domain.py recheck dlog --measure 3 # also re-measure each ladder size with 3 instances on this machine
recheck recomputes what is arithmetic and compares it with the record: the declaration and reference hash as recorded; the validation checks give the recorded verdicts; the ladder, n, CPU cap and CPU-hours follow from the recorded plan; and the whole power table (seeded, so it must come out identical) is recomputed through the real estimator. Any difference is a FAIL. The arithmetic needs no trust beyond the protocol code you checked out. **But recheck is not safe on a stranger's domain:** like compile, it executes the declaration's decl.py (validation derives instances with it) and the reference solver, unsandboxed, and --measure runs the reference many more times. Read a declaration before rechecking it, or do it in a throwaway VM.
What it does not recompute: the profile's timings and the self-test, which are measurements. --measure K re-measures the ladder sizes here on the same public profiling instances (instance i is a function of the public profiling seed) and prints the gap in bits per size. A compile now records every instance's cost, so the gap compares the SAME K instances and is the machine, not which instances were drawn. A record made before that (dlog's, today) keeps only a median over all its instances, so the comparison includes sampling noise: with a per-instance spread of about 1.7 bits and K=3, noise alone gives a typical gap of 0.6 bit and one above 1.8 bits one time in ten. The output then says so and widens its warning threshold, and spread flags the recommendation as inflated. That does not prove the recorded timings; it tells you how far this machine is from the one that made them. A gap that is the same at every size is a hardware speed difference and cancels in a speedup ratio; a gap that changes with size is not. Two or more machines' --measure output is the data needed to set --tolerance honestly. None has been collected yet. To collect it: on each machine run compile_domain.py recheck NAME --measure 3 --save NAME.<machine>.json (never overwrites; it stores a hash of the host name, not the name), then merge them anywhere with compile_domain.py spread *.json. spread recommends a tolerance (the largest within-machine range of gaps across sizes, rounded up to a quarter bit, never below 1 bit because the re-measurement compares single instances whose own noise is about 0.6 bit and spread does not measure it) only when at least two distinct machines measured the same declaration; with fewer it says so and the default stays loose. The recommendation is an estimate from a few sizes, not a bound.
Attestations (attest.py)
python3 attest.py keygen --out my.key # a key you keep; publish the public key it prints
python3 attest.py sign x.cert.json --key my.key --out x.me.att.json --rerun 5 --execute # verify on YOUR machine, sign the outcome
python3 attest.py check x.cert.json a1.json a2.json --trusted keys.txt
A verifier runs certificate.py verify (and optionally --rerun K, on instances drawn from the verifier's own random seed) and signs the result with an Ed25519 key: the outcome of every check, the re-measured gap, a hash of the certificate, and a self-reported machine description. check verifies the signatures and counts distinct keys.
What an attestation is worth is what its signer's identity is worth. A key proves the same party signed; it does not prove independence from the operator, because an operator can create five keys. So check reports a key as independent only if it is listed in the --trusted file you built yourself from keys you obtained out of band, and every other key is reported as "unvouched", never counted. A signed statement that the certificate does NOT verify (a dissent) makes check fail, from any key. A valid signature proves only who signed, so the statement's contents are still untrusted input: check accepts exactly the shape sign writes (a real boolean verified, short plain-text check names, a finite gap) and ignores anything else with a warning, so a hostile attestation can neither crash a caller nor count. The same goes for the measurement files spread merges, and for certificates on the site: whatever they make the verifier raise costs only that file. One key signing twice is one party. Needs the cryptography package, which the project's own venv does NOT have (install it only where you sign or check attestations). Without it, sign and keygen refuse with a message, check and the site report the attestation files as "NOT checked" and count none of them (a dissent among them would not be seen), and the signature tests in make test are skipped, not passed. Attestations do not touch the certificate, so adding one never changes its hash.
To show them on the site, put NAME.cert.json, NAME.<who>.att.json and (optionally) trusted.txt in one directory and build with python3 build_site.py --certs DIR (keep DIR outside the site's output directory: the build deletes and recreates that directory, and refuses to run if an input lives inside it). Each certificate is re-verified at build time (nothing in it executed), shown with its checks, and its attestations are counted as pinned or unvouched exactly as attest.py check does. make attest CERT=... KEY=... OUT=... is the one-command version for an outside verifier.
The pre-registered real-solver test (dlog_experiment.py)
results/PREREGISTERED_dlog_realsolver.txt was written before any data existed: two real dlog solvers (a 4-entry baby-step scan, true speedup about 1.3-1.6 bits, and full baby-step/giant-step, far above the ceiling) on three seeds. python3 dlog_experiment.py applies its criteria mechanically from the saved ladders and claim files (and re-hashes every registered code file, failing if any changed), and the Results page shows the same table. Criteria that cannot be evaluated yet print PENDING, never PASS.
What this does not do (read before relying on it)
- A certificate proves consistency, not honest measurement. It shows the commitments were public first, the instances are the seeds' own, the rules are the named ones, and the credit follows from the records. It cannot show the CPU costs in those records are what a sandbox measured: an operator could write plausible ones. Only the Docker meter (unverified),
verify --rerun(re-measuring instances yourself, needs--execute) and independent attestations from keys you chose to trust address that, and the first of those has never been run on a real host.build --reveal-seedsis refused until the round is revealed. - A private solver is hidden from the public, not from the operator.
certificate.py make-private(orbuild --private; in a round, any submission whose owner did not choose "publish source") removes the solver's source and every failure message from its ladders (a traceback quotes code). The log, the fingerprints, the questions, the rules and the credit arithmetic still verify, and the solver's hash is the one the round's log committed to before the seeds were revealed; but nobody except the operator can inspect or re-run it, so its timings rest on the operator's sandbox alone (the verdict says so). If the owner publishes the file,verify --source FILEchecks that it hashes to the committed value and, with--rerun --execute, re-measures it: the certificate is then as strong as a full one. Hiding a solver from the OPERATOR too would need trusted hardware (not available) or a zero-knowledge proof of execution, which is infeasible here: these solvers run for seconds to minutes (billions of cycles), a zkVM would count abstract cycles not CPU seconds, and it cannot run numpy or fpylll. - The reference defines the baseline. A certificate says "faster than this reference by this many bits on its hardest instances". It does not say the reference is a good algorithm. A weak reference makes big speedups cheap; the compiler cannot judge that and neither can the estimator. What it does is make the author say it: a missing
REFERENCE_NOTEmakescheckwarn (it does not fail), and the note is copied into the compiled record, the challenge document and the domain card, and reads "not stated" when missing. When someone beats the reference by a large margin, replace the reference, bumpVERSIONand recompile; old results then refuse to mix with new ones. - The power analysis rests on an assumed pair noise. Real per-instance noise of a submission against the baseline cannot be known before a submission exists. The default is 0.6 bit (
--pair-noise); compare it with the measured values in PROTOCOL.md and re-run with yours. The compile also fixes everything from 8-instance medians per size, so the ladder is a good choice, not an optimal one. - Certifiable range has a ceiling (about 5 bits at a 120 s cap, as in PROTOCOL.md): a solver that is 2^5 times faster than the reference costs less than the estimator's 0.5 s floor on the instances the baseline finds hard. The compiled
CHALLENGE.mdreports the actual range; beyond it a model baseline is needed (not built). - Validation finds the defects it knows. The anti-cheat checks catch a verifier that is guessable, derivable from the arguments, or reusable across instances. They cannot prove a domain has no shortcut: a domain whose instances are structured so that a solver can skip the work is the author's to rule out (EGG.md is the worked example of a family that could not be tested).
- The sandbox is still unverified. Everything here inherits
sandbox/README.md: the Docker meter has never passedtest_sandboxon a real host, so no compiled domain can have an OFFICIAL result yet, exactly likegm,mlweandfactor. - The self-test is local mode. It runs at the chosen official
n(--selftest-nlowers it to save machine time, but a known-effect control run below the n the power analysis chose can fail for lack of power alone), without Docker. It tests that the compiled pipeline behaves; it is not an official run. - **What the compiler's notes caught on
dlog.** The reference's cost does not double smoothly per size: size 30 measured a 39-43 s median against 3.4-3.8 s at sizes 28-29 (twice, with 8 and with 16 instances), a cliff that is probably CPython's big-integer arithmetic leaving its single-digit fast path once the modulus passes 2^30. The compile re-planned to a 3-size ladder (28-30, 2.6 octaves), and its output says so: "narrow ladder", "NOT independently confirmed", "still off by more than one octave at size 30". It did not hide the cliff or claim a confirmation. The honest reading is that this reference is a poor baseline for a ladder (a perfectly valid, deliberately plain demo), and that a declaration author should read those notes, not just the exit code. - The dlog challenge's size-29 profile is slightly optimistic. Its compiled record dropped one slow instance and counted another twice, so the recorded size-29 median (3.75 s), spread and the ladder chosen from them lean toward cheaper instances. How far is not measured (that needs all 16 instances at the 120 s cap). Recompiling the domain settles it; that is held until the pre-registered experiment ends, because the compiled record is part of what that registration hashed.
- The profile is noisy. The ladder and the power simulation rest on 8-instance medians per size (
--n-prof). On the first real run (dlog) neighbouring sizes came out as 3.7 s, 2.05 s and 43 s where the true cost doubles per size, so the compile re-plans, and the resulting ladder can be short (3 sizes there) and irregular. Raise--n-proffor a steadier profile. - Python 3.14 here. Tested on the interpreter at hand; 3.10-3.13 and Linux are untested like the rest of the project (PROTOCOL.md).
Files
| file | role |
|---|---|
declared.py | loads and checks declarations, registers fresh compiled domains into families.py's six tables |
compile_domain.py | the passes and the CLI (init, compile, check, selftest, emit, status) |
certificate.py | build and verify for speedup certificates |
domains/<name>/ | decl.py, the reference, compiled.json (+ compiled.prev.json), out/ |
recheck.py | compile_domain.py recheck: recompute a compiled record's checkable claims; --measure compares this machine with the record |
attest.py | keygen, sign, check for independent signed attestations of a certificate |
counted.py, count_runner.py | deterministic step counting: exactly reproducible costs (Python 3.12+), the counted ladder writer and the determinism self-check |
certs_site.py | the site's Certificates section (build_site.py --certs DIR) |
dlog_experiment.py | mechanical judge of the pre-registered dlog real-solver experiment |
test_compiler.py, test_certificate.py, test_recheck.py, test_attest.py | the tests (make test runs them) |