Side project / under development
LATOR.
An excuse to learn more about AI infrastructure.
LATOR stands for Lajos's AI Tracer & Router. It's a Rust-based gateway project where I'm experimenting with the bits around LLM inference: sending requests to different providers, streaming responses, recording traces, replaying requests, and comparing model performance.
What it is
A small gateway, with lots of interesting problems.
It speaks the OpenAI-compatible API, with Rust, Tokio and Axum handling the HTTP side and SQLite for local persistence. I'm exploring routing and fallback, authentication, metrics, replay, and benchmarking. Some of these are implemented; others are still on the roadmap.
app→LATOR→LLM backends
I'm mainly interested in what happens when model APIs are slow, unreliable, or difficult to observe—and how to test that properly.
Tools
RustTokioAxumSQLiteOpenAI-compatible APIsSSE streamingPrometheus
This is an ongoing personal project, not a production product.