API Gateway

Overview

Wanted to learn Rust by building something actually useful. Microservices at scale need a gateway, and existing solutions (Kong, AWS API Gateway) are either expensive or complicated to set up. This is a simple, fast gateway that does rate limiting, auth, and routing. It's handling all traffic for a side project that does ~500K requests/day on a $5 Hetzner VPS.

Technical Details

• •Built with Tokio (async runtime) and Axum (web framework) - Rust's async ecosystem is maturing nicely
• •Rate limiting using a token bucket algorithm, backed by Redis for distributed rate limiting across instances
• •JWT validation with public key caching (supports RS256, ES256, HS256)
• •Dynamic routing configuration loaded from a YAML file - no restarts needed, watches for file changes
• •Request/response transformation with Lua scripts for complex cases (inspired by Kong)
• •Connection pooling to upstream services with circuit breaker pattern
• •Prometheus metrics endpoint exposing latency histograms, status codes, rate limit hits, etc.
• •Graceful shutdown that waits for inflight requests to complete
• •Docker multi-stage builds resulting in 15MB images (Rust binary + Alpine)

Challenges

• •Async Rust has a steep learning curve. Lifetimes + async + tokio = lots of time fighting the compiler. Worth it though, zero runtime errors once it compiles.
• •Redis connection pool management was tricky - connections would die randomly under load. Switched to a library with better reconnection logic.
• •Lua integration for request transformation is cool but caused security concerns - had to sandbox it properly to prevent people from doing file I/O or network calls.

Results

• •Handles 6K+ req/sec on a single core (tested with wrk)
• •Memory usage is stable at ~25MB under load
• •Successfully replaced an Nginx + Node.js setup that was using 4x more resources
• •Zero downtime for 4 months of production use (excluding planned maintenance)