← ~/projects
$ cat projects/ai-framework-rust.md
AI Framework using Rust
Memory-safe AI framework in Rust, with Python bindings via PyO3.
Sole authorJune 2024AI/ML
// why: Python AI stacks force a perf vs safety trade — Rust doesn't.
$ cat projects/ai-framework-rust.md▌
TL;DR
- Memory-safe AI stack: routing, agents, vectors, caching in Rust
- Two providers day one: OpenAI + Anthropic adapters
- Adopt gradually: PyO3 bindings, no rewrite needed
// what does it do?
Open-source web framework for AI apps in Rust: HTTP routing, LLM abstraction, async vector storage, multi-step agents, semantic caching. Python bindings via PyO3, no GC tax.
// why this project?
Existing Python AI frameworks give up either performance or memory safety. In Rust you don't have to pick one.
// what broke / learned
Currently extending it — async vector storage was the rabbit hole.
// results
- Built from scratch to avoid the memory-safety trade-offs of Python AI stacks. Ships OpenAI and Anthropic provider adapters, async vector storage, and multi-step agent pipelines. PyO3 bindings let existing ML teams adopt it incrementally instead of rewriting everything at once.
OpenAI and Anthropic adaptersAsync vector storagePyO3 Python bindings
Tech stack
RustPyO3TokioHTTPLLM APIsVector DB