The foundation every OTTO module stands on
Shared is OTTO's infrastructure library — 40+ modules providing geo calculations, memory allocators, JSON/protobuf/CSV parsers, rate limiting, circuit breakers, worker pools, and font rendering. Zero external dependencies. Every OTTO component from Surge to Carta to Ralph links against Shared. If it's not domain-specific, it lives here.
Write it once, use it everywhere
Every OTTO module needs Haversine distances. Every API server needs rate limiting. Every parser needs arena allocation. Without a shared library, each module reinvents these primitives — differently, inconsistently, with different bugs.
Shared enforces a single implementation for cross-cutting concerns. When Carta, Velo, Locus,
and Surge all use sh_ratelimit_check(), there's one token bucket implementation
to audit, test, and harden. When FuelWise and Surge both need JSON parsing, there's
one sh_json_parse() with one fuzz corpus.
mmap. Designed for edge deployment from day one.
The rule. If functionality isn't domain-specific to a single module, it belongs in Shared. Geo math, data structures, serialization, HTTP hardening, resilience patterns, memory management — all live here. Domain modules (Surge, Ralph, Velo) contain only their domain logic.
40+ modules across 10 categories
┌──────────────────────────────────────────────────────────────────┐
│ OTTO Domain Modules │
│ Surge │ Ralph │ Velo │ Carta │ Locus │ FuelWise │
└────┬────┴────┬────┴───┬────┴────┬────┴────┬────┴────┬──────────┘
│ │ │ │ │ │
▼ ▼ ▼ ▼ ▼ ▼
┌──────────────────────────────────────────────────────────────────┐
│ shared/ │
│ │
│ ┌─────────┐ ┌──────────┐ ┌──────────┐ ┌───────────────┐ │
│ │ Geo │ │ Memory │ │ Parsing │ │ HTTP/Server │ │
│ │ haversine│ │ arena │ │ json │ │ ratelimit │ │
│ │ eov │ │ pool │ │ csv │ │ workqueue │ │
│ │ spatial │ │ hashmap │ │ xml │ │ cors │ │
│ │ dist │ │ heap │ │ protobuf│ │ httpserver │ │
│ │ polyline│ │ │ │ pbf │ │ completion │ │
│ │ │ │ │ │ geojson │ │ worker_pool │ │
│ └─────────┘ └──────────┘ └──────────┘ └───────────────┘ │
│ │
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌───────────────┐ │
│ │Resilience│ │ Config │ │ Render │ │ Observability │ │
│ │ circuit │ │ args │ │ font │ │ log │ │
│ │ backoff │ │ capacity│ │ render │ │ trace │ │
│ │ retry │ │ query │ │ │ │ metrics │ │
│ └──────────┘ └──────────┘ └──────────┘ └───────────────┘ │
│ │
│ ┌──────────┐ ┌──────────┐ │
│ │ Math │ │ Hashing │ 319 tests · ~24K LoC │
│ │ piecewise│ │ hash │ 40 headers · 42 source files │
│ │ stepfunc │ │ sha256 │ 0 external dependencies │
│ │ units │ │ │ │
│ └──────────┘ └──────────┘ │
└──────────────────────────────────────────────────────────────────┘
Geography & Coordinates
| Module | Header | Purpose |
|---|---|---|
| sh_geo | sh_geo.h |
Haversine distance, SHCoord type, coordinate parsing/validation, bearing calculations |
| sh_eov | sh_eov.h |
HD72/EOV ↔ WGS84 conversion (Hungarian national grid, 7-parameter Helmert) |
| sh_spatial_grid | sh_spatial_grid.h |
Fixed-cell spatial index for radius queries. O(1) cell lookup, configurable resolution |
| sh_dist | sh_dist.h |
Point-to-segment distance, great circle intersections |
| sh_polyline | sh_polyline.h |
Google Polyline encoding/decoding for route geometry compression |
Memory & Data Structures
| Module | Header | Purpose |
|---|---|---|
| sh_arena | sh_arena.h |
Bump allocator. O(1) alloc, O(1) reset. Per-request scratch space for API servers |
| sh_pool | sh_pool.h |
Typed contiguous pool. Offset-based access, growable. Used for OSM coordinate storage |
| sh_hashmap | sh_hashmap.h |
Open addressing with Robin Hood probing. FNV-1a hashing. Used by Velo, Locus, Nexus |
| sh_heap | sh_heap.h |
Binary min/max heap. Used by Velo for Dijkstra priority queue |
Parsing & Serialization
| Module | Header | Purpose |
|---|---|---|
| sh_json | sh_json.h |
Full JSON parser + ShJsonWriter streaming encoder. Arena-backed. Mandatory for all OTTO APIs |
| sh_csv | sh_csv.h |
RFC 4180 CSV parser with configurable delimiters. Used by Nexus for data ingestion |
| sh_xml | sh_xml.h |
Streaming XML parser. Used by Nexus for XLSX extraction (Office Open XML) |
| sh_protobuf | sh_protobuf.h |
Wire-format protobuf decoder/encoder. No .proto files, no codegen. Used by Carta for MVT |
| sh_pbf | sh_pbf.h |
OpenStreetMap PBF parser. Delta-decoded coordinates, string tables. Powers Velo + Carta + Locus |
| sh_geojson | sh_geojson.h |
GeoJSON emitter for API responses. Feature collections, geometries |
| sh_pdf2struc | sh_pdf2struc.h |
PDF text extraction with column/table reconstruction. Used by Nexus for document ingestion |
HTTP & Server Hardening
| Module | Header | Purpose |
|---|---|---|
| sh_ratelimit | sh_ratelimit.h |
Token bucket rate limiter. Per-IP tracking, configurable RPS/burst, thread-safe |
| sh_workqueue | sh_workqueue.h |
Bounded producer-consumer queue. Backpressure via 503, expiration for stale items |
| sh_completion | sh_completion.h |
Work item completion signaling. Replaces manual mutex/cond boilerplate |
| sh_worker_pool | sh_worker_pool.h |
Thread pool with callback dispatch. Auto-detects CPU count, clean shutdown |
| sh_httpserver | sh_httpserver.h |
Mongoose integration helpers. Socket write timeouts for slow-client protection |
| sh_cors | sh_cors.h |
CORS header generation. Origin allowlisting, preflight handling, credentials |
Resilience
| Module | Header | Purpose |
|---|---|---|
| sh_circuit | sh_circuit.h |
Circuit breaker (closed/open/half-open). Thread-safe. Configurable thresholds via env vars |
| sh_backoff | sh_backoff.h |
Exponential backoff with jitter. Configurable base delay, max delay, jitter factor |
| sh_retry | sh_retry.h |
HTTP retry orchestrator. Combines circuit breaker + backoff. Respects Retry-After headers |
Configuration & CLI
| Module | Header | Purpose |
|---|---|---|
| sh_args | sh_args.h |
CLI/env arg parsing. sh_parse_int() with bounds checking. Type-safe defaults |
| sh_capacity | sh_capacity.h |
M/M/c queuing theory. Calculates rate limits, queue depths, burst capacity from SLA targets |
| sh_query | sh_query.h |
URL query string parser. sh_query_get_int_bounded() for safe integer extraction |
| sh_adaptive | sh_adaptive.h |
Auto-tune rate limits from measured response times. P50/P90/P99 tracking, EMA smoothing |
Observability
| Module | Header | Purpose |
|---|---|---|
| sh_log | sh_log.h |
Structured logging. JSON or text output. Configurable level via SH_LOG_LEVEL env |
| sh_trace | sh_trace.h |
Trace ID propagation. Extract from X-Trace-Id headers, auto-generate if missing |
| sh_metrics | sh_metrics.h |
Metrics collection. Counters, histograms, timers. Prometheus /metrics endpoint output |
Math & Units
| Module | Header | Purpose |
|---|---|---|
| sh_piecewise | sh_piecewise.h |
Piecewise linear function evaluation and composition. Used for cost functions |
| sh_stepfunc | sh_stepfunc.h |
Step function evaluation. Used for time-dependent travel speeds |
| sh_units | sh_units.h |
SI ↔ imperial conversion. sh_miles_to_m(), sh_mpg_to_l100km(). Internal SI mandate |
| sh_hash | sh_hash.h |
FNV-1a hash function. Used by sh_hashmap and Nexus diff |
| sh_hash_sha256 | sh_hash_sha256.h |
SHA-256 implementation for content integrity verification |
Rendering
| Module | Header | Purpose |
|---|---|---|
| sh_font | sh_font.h |
Embedded bitmap font atlas (28K LoC data). Glyph lookup for Carta PNG tile labels |
| sh_render | sh_render.h |
Software rasterization primitives. Anti-aliased lines, polygon fill, text rendering |
Clean C APIs for every concern
Arena allocator
#include "sh_arena.h" // Create arena for per-request scratch space SHArena *arena = sh_arena_create(64 * 1024); // 64KB // Bump-allocate (O(1), no individual frees) double *coords = sh_arena_alloc(arena, n * sizeof(double)); int *indices = sh_arena_calloc(arena, m, sizeof(int)); // Use allocations... process(coords, indices); // Free everything at once (O(1)) sh_arena_free(arena);
Rate limiter
#include "sh_ratelimit.h" // 10 RPS, burst of 100, track 4096 IPs ShRateLimiter *rl = sh_ratelimit_create(10.0, 100.0, 4096); // In HTTP handler ShRateLimitAddr addr; sh_ratelimit_addr_ipv4(&addr, client_ip); if (!sh_ratelimit_check(rl, &addr)) { mg_http_reply(c, 429, "Too Many Requests"); return; } // Adaptive: auto-tune from measured response times sh_ratelimit_update_rate(rl, new_rps, new_burst);
JSON parsing + writing
Parse (decode)
#include "sh_json.h" SHArena *a = sh_arena_create(4096); ShJsonValue *root; sh_json_parse(body, len, a, &root); double lat = sh_json_as_double( sh_json_get(root, "lat"), 0.0); ShJsonValue *arr = sh_json_get(root, "items"); for (size_t i = 0; i < sh_json_array_len(arr); i++) { ShJsonValue *el = sh_json_array_get(arr, i); } sh_arena_free(a);
Write (encode)
ShJsonBuf jb; sh_json_buf_init(&jb); ShJsonWriter jw; sh_json_writer_init( &jw, sh_json_buf_write, &jb); sh_json_write_object_start(&jw); sh_json_write_key(&jw, "status"); sh_json_write_string(&jw, "OK"); sh_json_write_key(&jw, "count"); sh_json_write_int(&jw, 42); sh_json_write_object_end(&jw); char *json = sh_json_buf_take(&jb); // {"status":"OK","count":42}
Circuit breaker + retry
#include "sh_circuit.h" #include "sh_retry.h" ShCircuitBreaker *cb = sh_circuit_create(NULL); // defaults ShRetryContext ctx; sh_retry_init(&ctx, cb, NULL); while (sh_retry_should_attempt(&ctx)) { int status = make_http_request(); double delay = sh_retry_after_response(&ctx, status, retry_after); if (!sh_retry_should_continue(&ctx)) break; usleep((useconds_t)(delay * 1000)); } sh_circuit_free(cb);
Worker pool
#include "sh_worker_pool.h" #include "sh_workqueue.h" ShWorkQueue *q = sh_workqueue_create(1000, 5.0); // 1000 items, 5s timeout ShWorkerPoolConfig cfg = { .queue = q, .callback = render_callback, .poll_timeout_ms = 100 }; ShWorkerPool *pool = sh_worker_pool_create(0, &cfg); // 0 = auto CPU count // Shutdown: stop → join → free sh_worker_pool_stop(pool); sh_worker_pool_join(pool); sh_worker_pool_free(pool); sh_workqueue_free(q);
Design decisions
Memory management philosophy
Shared provides three memory strategies, each with clear tradeoffs:
| Strategy | Alloc | Free | Best For |
|---|---|---|---|
| SHArena (bump) | O(1) | O(1) reset | Per-request scratch, parsing temporaries |
| SHPool (typed) | O(1) | O(1) reset | Many same-type arrays (coordinates, nodes) |
| Guarded malloc | O(1) amortized | O(1) | Results that outlive the call, unbounded size |
The hybrid pattern. Arena for temporaries, malloc for results. API servers allocate per-request arenas; algorithm scratch lives in arenas; final results are malloc'd and owned by the caller. Arena resets on request completion clean up everything else automatically.
SI units mandate
All OTTO internal calculations use SI units: meters, liters, L/100km, kilograms.
sh_units.h provides boundary conversion only:
sh_miles_to_m(), sh_mpg_to_l100km(),
sh_gallons_to_liters(). Imperial units never appear in internal state.
One formula, no conversion errors.
Hashmap design
Open addressing with Robin Hood probing. FNV-1a hashing. Load factor triggers resize at 75%. Used by Velo for node ID mapping during PBF parsing, by Locus for string deduplication, and by Nexus for change detection. Tested for collision resistance on real OSM data.
Who uses what
| Module | Arena | Pool | JSON | PBF | Rate | Workers | Geo |
|---|---|---|---|---|---|---|---|
| Surge | ✓ | — | ✓ | — | ✓ | ✓ | ✓ |
| Ralph | ✓ | — | ✓ | — | ✓ | ✓ | — |
| Velo | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| Carta | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| Locus | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| FuelWise | ✓ | — | ✓ | — | ✓ | ✓ | ✓ |
| Nexus | ✓ | — | ✓ | — | — | — | ✓ |
Codebase
LoC note. The raw count is ~52K, but ~28K of that is
sh_font_data.c — an embedded bitmap font atlas for Carta's
PNG tile label rendering. Actual logic is ~24K LoC across 42 source files.
Using Shared in your module
1. Build from source
# Clone and build git clone https://github.com/ottofleet/otto.git cd otto && make shared # Run all 319 tests make test-shared # With AddressSanitizer make test-shared CFLAGS="-fsanitize=address,undefined -g"
2. Link in your module
# In your module Makefile INCLUDES = -I../shared/include LIBS = ../shared/libshared.a # Use any module #include "sh_arena.h" #include "sh_json.h" #include "sh_ratelimit.h" #include "sh_geo.h"
3. Build for WASM
# Shared compiles to WASM with zero changes emcc -O2 shared/src/sh_arena.c shared/src/sh_json.c \ -I shared/include -o shared.wasm # All parsers, geo functions, and data structures # work in the browser. No #ifdefs needed.
Common questions
Why not use existing libraries (jsmn, cJSON, protobuf-c)?
Three reasons: (1) WASM compatibility — every dependency must
compile to WebAssembly without modification. Most C libraries assume POSIX.
(2) Audit surface — every line of code in OTTO is auditable.
No transitive dependencies, no supply chain risk. (3) Arena integration
— our JSON parser allocates from SHArena, which means zero individual
frees and predictable memory behavior. Third-party parsers use malloc.
What's the font atlas?
sh_font_data.c (~28K LoC) is an embedded bitmap font atlas for Carta's
PNG tile rendering. Carta renders map labels on server-generated PNG tiles using software
rasterization — no FreeType, no HarfBuzz, no system fonts. The atlas is a C array
compiled into the binary, so tile rendering works identically on Linux, macOS, and WASM.
How does the rate limiter work?
Token bucket algorithm with per-IP tracking. A hash table maps client IPs to bucket
state. Tokens refill at the configured RPS rate, burst allows short spikes. Thread-safe
for multi-worker servers. The adaptive tracker (sh_adaptive.h) measures
P50/P90/P99 response times and auto-tunes the rate limit to match actual server capacity.
Can I use Shared independently of OTTO?
Yes. Shared has no dependencies on any OTTO domain module. It's a standalone static library.
Link libshared.a and include the headers you need. Each header is self-contained
— sh_arena.h doesn't pull in sh_json.h.
How is thread safety handled?
Shared libraries never use static/global mutable state. All state is passed through parameters or context structs. The rate limiter and work queue APIs are explicitly thread-safe (internal mutexes). Arena and pool allocators are single-threaded by design — each thread gets its own arena.
What license?
AGPLv3 with a Trucking Exception. Same terms as all OTTO modules. Trucking fleets using Shared for their own operations are covered by the exception. SaaS products embedding Shared need source disclosure or a commercial license.