commit b8519ffa73911b2294da61966f7c242b056d591f4d45f93a070429155c4a765d from: Tracey Emery date: Tue Nov 10 17:22:07 2026 UTC import ROADMAP.md and update README.md commit - 05867825afa7e5332e28ab224a39dbc9424bc5fc79fca965142619832a172f1d commit + b8519ffa73911b2294da61966f7c242b056d591f4d45f93a070429155c4a765d blob - 1a60cde5900ca7ea5c7c385f0fece4fd978845cf4814a5584bba94ad830735e7 blob + 4dc090e70cafe1bb5eecf60971d300f5c725b4353660058a8fe7353e0d96ca92 --- README.md +++ README.md @@ -75,7 +75,8 @@ Not yet implemented ------------------- * adding or removing bridges without a restart -* fan-out offload to the sockets processes (see ROADMAP.md) +* fan-out offload to the sockets workers, only if benchmarks show the + broker bound on per-recipient encoding (see ROADMAP.md) Author ------ blob - /dev/null blob + dfd75581346f4d19be84c062c85ac85b0f8b2e789625238e1e67790bae9d0942 (mode 644) --- /dev/null +++ ROADMAP.md @@ -0,0 +1,93 @@ +# mqttd roadmap + +Goal: an OpenBSD-native, privilege separated MQTT broker that is small in +memory, predictable in latency and simple to operate, with full +Sparkplug B and Sparkplug Aware behaviour. + +## Done + +MQTT 3.1, 3.1.1 and 5.0 codec; QoS 0, 1 and 2; retained messages; wills +with delay; sessions with expiry and offline queues; shared +subscriptions; `$SYS` statistics. + +Client-certificate identity, ACLs by user and client id, per-user +limits, rate limiting, pf banning, external authenticator, enhanced +authentication (AUTH packets) through it. + +Snapshot persistence, SIGHUP reload, structured `event=` logs. + +Bridging to remote brokers, both directions, prefix rewriting, reconnect +with backoff, local queueing while the remote is down. + +Sparkplug Aware mode: `$sparkplug/certificates/` copies of NBIRTH and +DBIRTH, cleared on NDEATH and DDEATH, timestamp injected into NDEATH +wills at delivery. + +MQTT over WebSocket listeners, PROXY protocol v1 and v2 behind relayd or +haproxy, unix socket listener for local collectors. + +`mqttctl show stats`, `disconnect all`, `show bridges`. + +Listener add and remove on reload; bridges still need a restart. + +Process layout: one listen process accepts, consumes PROXY headers and +dials bridges, then hands each descriptor to the sockets worker with the +fewest connections. Workers pledge `stdio recvfd`. `workers` replaces +`prefork`. + +Per-connection backpressure: a subscriber over its output watermark gets +QoS 0 dropped and QoS 1 and 2 parked instead of stalling the pipe or +being disconnected; retained messages owed to a new subscription are +still delivered. + +Dedicated login class in `login.conf.d` so the descriptor limit, and +with it client capacity, is a chosen number. + +Snowflake sink, first cut: a `snowflake` block is a bridge whose remote +is a dedicated process (chrooted, `pledge("stdio inet")`). Rows go +through the SQL REST API as multi-row INSERTs with key-pair JWT +authentication; the parent holds the private key, signs tokens and +resolves the hostname. Publishers are acknowledged only after the +warehouse confirms, so delivery is at least once with the bridge queue +and persistence covering outages. `host`, `tls servername`, `tls ca` and +`tls noverify` cover the wildcard, regional and PrivateLink naming cases. + +## To do + +Full test pass on the OpenBSD host: protocol behaviour, reload, bridges, +persistence, and the rebirth storm and congestion checks added with the +process split. + +Paho conformance suite against a running broker. + +Fuzzing of mqtt.c with a corpus of valid and truncated packets. + +Bridge add and remove on reload without a restart. + +`congestion` setting with `drop` and `disconnect` values, for deployments +that would rather a Sparkplug host take one reconnect than see sequence +gaps. + +Benchmarks: connect storm at `workers 1` and `workers 4`, steady state +publish throughput, and per-sink Snowflake throughput, all recorded with +the hardware and configuration used. + +Fan-out offload, only if the benchmarks show the broker core bound on +per-recipient encoding: the broker sends each sockets worker one message +plus a list of (connection, packet id, QoS, retain, subscription ids) +tuples and the worker encodes the PUBLISH per connection. + +Broker sharding, only if routing itself saturates a core after the +offload: several broker processes owning sessions by client-id hash with +a router step between them. No pthreads: OpenBSD's libevent is not +thread-safe, the trie and session trees would need locking, and the +privsep model relies on single-loop processes. + +## Exploring + +Snowpipe Streaming as a lower-latency, cheaper-per-row alternative to +SQL INSERTs for the Snowflake sink. + +Decoding Sparkplug B payloads into columns, metrics as VARIANT, versus +storing the protobuf as received; needs a decoder for the Sparkplug +payload schema.