Codacus dropped a killer guide for hosting providers and self-hosters looking to slash compute costs. Instead of slow token-by-token JSON generation, he replicated Jev's architecture directly in llama.cpp using a new /v1/decision server endpoint.

By forcing the model to choose from pre-defined schemas rather than write text, response times plummeted from 3415ms to just 306ms (17.3ms per decision in bulk!) using a tiny 2GB model in 2.4GB of VRAM. A massive win for lightweight, high-throughput API endpoints.