lserafin/fmm-17-32

670 exactly verified integer schemes for rectangular fast matrix multiplication, dimensions 17-32: 13 improved upper bounds, machine-checkable certificates

★ 0Forks 0PythonGitHub ↗Compare

README

Concrete Integer Certificates for Rectangular Fast Matrix Multiplication Bounds (maximum dimension 17–32)

DOI

Among the sources in our 2026-07-07 freeze audit, we found no earlier release of concrete integer factor-matrix schemes with exact software verification for the 670 rectangular matrix multiplication formats published here:

  • 670 bilinear schemes with integer coefficients, each strictly below the upper bound recorded in the Lille FMM catalogue (machine-readable baseline: fmm_digest, pinned by commit in results/summary.json), each with construction provenance and an exact validity certificate over ℤ (headline numbers: results/summary.json, full table: results/records.csv);
  • 13 upper bounds that improve every public source in our freeze audit: 12 beating present entries of the derived bound table in Chatain Lacelle's matmulcatalog plus one format absent from it. For example, ⟨21,26,30⟩ in 8844 (previously 8862 derived / 8958 catalogued), ⟨14,20,26⟩ in 4239 (4246 / 4298), ⟨8,12,26⟩ in 1527 (absent / 1528);
  • independent confirmations of 422 derived bounds whose pinned comparison table does not include the corresponding factor matrices. Among these, 31 supplement rational-tagged derived records with explicit same-rank integer witnesses, valid over every field. The remaining 235 derived entries are lower than our materialized bounds, with a total gap of 3,854 and a median gap of 10.

Repository layout

verify.py            standalone exact verifier (Python stdlib only)
verify_rust.py       released integer schemes -> independent Rust checks + CRT proof
Makefile             reproduction targets (see Reproduction section for scope)
SHA256SUMS           checksums for the archived release payload
schemes/             670 schemes, one gzipped JSON per format ({a}x{b}x{c}_r{rank}.json.gz)
provenance/          one JSON per scheme: complete construction trace + source hashes
results/
  records.csv        per-format table: bounds, derived-band domain tag, status, construction
  summary.json       headline numbers + pinned baseline commits
  export.log         per-scheme export and inline-verification trace
  verify_all.log     release-gate log: verify.py over all 670 schemes (all valid)
  rust_construction_verification.log  Rust multi-prime verification at construction
  closure_{lille,merged}.json         complete deterministic closure derivations
  closure_{lille,merged}_report.json  closure classifications used by the report
candidates/
  README.md          maintainer doc for the 13 candidate SOTA updates
  *.csv / *.json     CSV + patch JSON + drop-in fmm_sota.json fragment
  logs/              per-candidate verifier logs
report/
  report.tex         technical report (numbers imported from generated.tex)
  generated.tex      auto-generated macros + figure data (make reproduce-report)
  report.pdf         compiled report
tools/
  export_publication.py   working-tree -> schemes/, provenance/, results/
  gen_report_data.py      results/ -> report/generated.tex
  gen_digest_candidates.py  results/ -> candidates/ (maintainer package)
  parse_lille.py          fallback parser for the Lille homepage HTML
  pipeline/               full discovery pipeline (audit + its frozen seed
                          table, scheme algebra, grouping + materialization,
                          scanner patch)
verifier-core/       vendored Rust modular verifier source, lockfile, and tests

Context: who did what

The serendipitous product was introduced by W. D. Smith and generalized by A. Sedoglavic. This repository follows A. I. Perminov's computational formulation and open implementation and uses the base-scheme collection distributed through his repositories (arXiv:2606.02480); Perminov's published sweep covers formats with all dimensions ≤ 16. B. Chatain Lacelle's matmulcatalog (arXiv:2606.13408) distributes derived 17–32 comparison files with ranks, coefficient-domain and provenance tags, and structural derivation metadata, but its pinned band files do not include the corresponding factor matrices. These bounds were not present in the pinned Lille digest used for the freeze audit.

This repository contributes an independent recomputation using a greedy realizable-savings grouping rule relative to the available concrete part schemes, materialization of every released scheme, an exact Python verifier plus independent construction-time Rust verification logs, and the 13 improvements. Square formats ≥ 17 are deliberately out of scope; see Khoruzhii–Gelß–Pokutta (LITA) and Schwartz–Zwecher for that territory.

Verify everything yourself

python3 verify.py schemes/4x13x18_r629.json.gz     # a small scheme, seconds
python3 verify.py schemes/21x26x30_r8844.json.gz   # headline improvement, ~45 s
python3 verify.py --all                            # all 670 schemes (hours)

verify.py is stdlib-only Python and checks the Brent equations exactly over ℤ: arbitrary-precision integers, no floating point, no sampling. During construction every scheme was additionally verified by an independent Rust verifier modulo primes whose product exceeds twice the maximal possible residual coefficient (a CRT-certified exactness proof; see the technical report). The exact Rust source used at construction is included under verifier-core/. To rebuild it and repeat the Rust/CRT check directly on a released scheme:

make rust-verifier
make test-rust
python3 verify_rust.py schemes/4x26x28_r1859.json.gz

verify_rust.py performs only release-format loading, coefficient reduction, the explicit residual bound, and process orchestration; the Brent-equation accumulation is implemented independently in Rust. Cargo dependencies are pinned by verifier-core/Cargo.lock.

Scheme format

schemes/{a}x{b}x{c}_r{rank}.json.gz is gzipped JSON (all indices zero-based, row-major):

{"shape": [a, b, c], "rank": r,
 "L": [[[index, coeff], ...], ...],   // r sparse rows over A, i = i1*b + i2
 "R": [...],                          // over B, j = j1*c + j2
 "P": [...]}                          // over C, z = e*c + f (row-major)

File names use the canonical sorted key (a ≤ b ≤ c); the stored shape may be any permutation of it (the construction orientation); rank is invariant under axis permutation, and verify.py checks each scheme exactly as stored. Integer coefficients throughout: the schemes are valid over arbitrary associative unital coefficient rings (integer coefficients act through the canonical central image of ℤ), including the non-commutative coefficient rings that arise in recursive use. For i=(i1,i2), j=(j1,j2), and z=(e,f), validity is Σ_t L[t][i]·R[t][j]·P[t][z] = [i2=j1]·[e=i1]·[f=j2]. Each scheme has a matching schema-v2 provenance file in provenance/. It records the base scheme and its SHA-256 hash, base/factor ranks, every selected merge group and relative sign, zero-based singleton product indices, the cost and saving of each merge, recursively expanded part-scheme derivations with SHA-256 hashes for file leaves, structured previous bounds (rank, source snapshot, and the derived record's coefficient-domain tag), and the verification method. The requested format on a part derivation fixes its orientation; the constructor's deterministic cyclic/transpose permute_to operation maps the recorded source_format or canonical child construction to that orientation. The following identities are checked for every record:

groups union singletons = {0, ..., base_rank - 1}  (disjoint)
rank = factor_rank * number_of_singletons + sum(merged_cost)
rank = base_rank * factor_rank - sum(saving)

Reproduction

make verify-all        # re-verify every published scheme exactly over Z (self-contained)
make test-constructor  # exercise grouping/sign regressions and exact materialization
make test-provenance   # check construction partitions, costs, signs, hashes, derivations
make test-rust         # run Rust verifier tests + CRT-driver regression tests
make verify-rust       # build Rust verifier and repeat Rust/CRT verification for all schemes
make reproduce-report  # rerun closure audits; regenerate every result macro,
                       # table row, and figure dataset in the report (self-contained)
make candidates        # regenerate the 13-candidate maintainer package (self-contained)
make closure           # reproduce the Lille-only and merged-table closure audits
                       # from the frozen seed table (self-contained)
make reproduce-freeze  # regenerate schemes/, provenance/, results/; requires the
                       # discovery working tree (external inputs documented in
                       # tools/export_publication.py), default ../fmm-challange;
                       # override with DISCOVERY_WORKTREE=/path/to/checkout
make report            # regenerate report data and compile report/report.pdf
make release-manifest  # hash the release payload after rebuilding the PDF

tools/pipeline/ contains the full discovery pipeline: catalogue audit (composition_closure.py), scheme algebra (fmm_schemes.py), grouping + materialization (realize_serendipitous.py), and the patch extending Perminov's scanner past dimension 16. The technical report in report/ documents method, conventions, and the freeze audit table (source commit hashes).

The Rust targets require Rust 1.85 or newer (edition 2024 support); the first Cargo build may download the dependencies pinned in verifier-core/Cargo.lock. Build output under verifier-core/target/ is intentionally ignored.

For catalogue maintainers

candidates/ packages the 13 candidate SOTA updates in maintainer-friendly form: CSV + patch-style JSON + a drop-in fmm_sota.json fragment + per-candidate verifier logs, all pinned to the audited digest commit and documented in its own README.

Citation

If you use these schemes, please cite the archived release series doi:10.5281/zenodo.21249498 and credit the sources below. For exact reproducibility, use the version-specific DOI that Zenodo displays for the release used.

Generative-AI assistance

OpenAI Codex and Anthropic Claude Code were used under the author's direction for exploring candidate approaches, developing and debugging software, running reproducible repository workflows, and drafting and revising the report. The author reviewed all AI-assisted material and takes sole responsibility for the code, citations, scientific claims, and conclusions. The released computations, verification runs, and headline results are produced by the repository pipeline rather than accepted from model output: every scheme ships with an exact, machine-checkable certificate (verify.py), and every headline result and closure count is regenerated from released data and the frozen closure seed (make reproduce-report).

Credits

License

MIT (see LICENSE).

Contributors

lserafin

Issues