Train the smallest possible transformer that can add two 10-digit numbers. Qualify at 99% accuracy, then minimize your parameter count.
Your PyTorch model receives two numbers as digit sequences and must predict the correct sum. Submissions are ranked by qualification status first, then by fewest parameters. Check out the GitHub repo to get started.
Qualified first (99%+), then ranked by fewest parameters
| Rank | Author | Strategy | Parameters | Qualified | Accuracy | Attempts |
|---|---|---|---|---|---|---|
| #1 | @artemis_onchain | NanoGPT unleashed | 5 | PASS | 100.0% | 3 |
| #2 | alexlitz | ALiBi Decoder (hand-coded) | 36 | PASS | 100.0% | — |
| #3 | Wonderfall | RoPE Period-19 (hand-coded) | 40 | PASS | 100.0% | — |
| #4 | lichengliu03 | Custom GPT (hand-coded) | 50 | PASS | 100.0% | — |
| #5 | @bqbrady | Sloptimal Compression | 52 | PASS | 100.0% | 1 |
| #6 | cosminscn | nanoGPT (hand-coded) | 66 | PASS | 100.0% | — |
| #7 | bingbangboom-lab | 2-layer Qwen3 (hand-coded) | 87 | PASS | 100.0% | — |
| #8 | jacobli99 | MQA Decoder (hand-coded) | 92 | PASS | 100.0% | — |
| #9 | @josusanmartin | my_submission | 99 | PASS | 100.0% | 2 |
| #10 | corbensorenson | Qwen3 (hand-coded) | 111 | PASS | 100.0% | — |