Anthropic released FlashProve, an open-source prototype for provable inference, on November 1, the company said.[1] For a 400B-parameter mixture-of-experts model (Llama 4 Maverick, 4-bit quantization), a single H100 GPU generates proofs at over 100 tokens per second, with a proof size of about 19 MB per 1,024-token request, according to Anthropic.[1] Because the proofs are compact and fast to verify, users can trust responses even if they lack the resources to run the model themselves, the company said.[1]
Anthropic releases FlashProve, an open-source prototype for provable inference
For a 400B-parameter model, a single H100 GPU generates proofs at over 100 tokens per second, Anthropic says.
1 primary · 0 independent · 0 commentary · 1 vendorSources
- Progress on provable inference for large language models Anthropic PrimaryVendor