circolare

In distributed systems, we've long known that straight-line message queues create bottlenecks-but the resilience of circular message propagation. Which I call Circolare, offers a shockingly elegant solution. Four years ago. While designing a real-time telemetry platform for offshore wind farms, I stumbled into a pattern that challenged everything I thought I knew about fault tolerance and event routing. We called it Circolare, a nod to the Italian word for circular. And it's since reshaped how we think about messaging in edge-native architectures.

At first glance, a ring-based messaging topology seems like a relic of old token-ring networks, best left to museums alongside FDDI. But dig deeper and you'll find that circular propagation solves a fundamental tension in distributed systems: the trade-off between strong consistency guarantees and the ability to survive partial network failures without a centralized coordinator. This article unpacks Circolare from protocol theory to production implementation, revealing why it's quietly reemerging in the age of IoT, edge computing. And serverless event flows.

The Origin of Circolare in System Design

Circolare didn't start as a formal protocol. It grew from the frustration of watching standard publish-subscribe brokers-even those with clustering capabilities like NATS or RabbitMQ-degrade under a partitioned network. In a typical hub-and-spoke or full-mesh cluster, a single partition can strand messages or force a leader election that pauses the entire system for seconds. We needed a way for nodes to keep delivering data predictably despite arbitrary link failures. And a circular topology, long studied in the SWIM protocol family, offered an elegant alternative.

The core idea is simple: every node knows exactly one upstream neighbor and one downstream neighbor, forming a logical ring. Messages travel in one direction-clockwise-and every node acts as both a consumer and a relay. This deterministic routing eliminates the need for a central routing table and drastically reduces the metadata overhead that plagues large-scale mesh networks. Even when multiple adjacent nodes fail, the ring can heal by reconnecting severed ends using a lightweight membership protocol, ensuring the Circolare chain never breaks for long.

How Circolare Ring Topology Handles Partial Network Failures

In November 2021, an undersea cable cut between two fiber segments of our data center interconnect caused a split-brain scenario for our Kafka cluster. Recovery required manual intervention and resulted in 14 minutes of message backlog. This incident drove us to prototype a Circolare overlay that could gracefully degrade throughput while preserving message ordering across surviving nodes. The ring's unidirectional flow means that as long as the set of live nodes remains contiguous, messages continue to circulate without interruption.

When a failure fragments the ring into multiple disconnected arcs, each arc independently becomes a self-contained Circolare ring. Nodes in each fragment detect the missing neighbor via heartbeat timeouts and locally re-seal the loop by pointing to the next available node. This behavior, inspired by the self-healing properties of token-ring networks documented in RFC 1042, ensures that no single fault cascades into a full system outage. In our wind-farm deployment, we observed a 99. 93% message delivery rate during transient islanding events, compared to 85% with a traditional broker-based setup.

Circolare vs Traditional Publish-Subscribe: A Performance Benchmark

To quantify what Circolare buys you, we ran a controlled experiment on a 64-node cluster of Raspberry Pi 4 devices, each running a lightweight Go implementation of the protocol. We compared throughput and latency against a similarly sized NATS cluster configured with full mesh connectivity. The workload simulated 10 Hz telemetry streams from wind turbine sensors, each payload 2 KB in size-an exact mirror of our production scenario.

Under normal conditions, Circolare exhibited 12% lower aggregate throughput due to the extra hop‑by‑hop relaying. However, when we introduced 30% random packet loss-typical for rural wireless backhaul-the Circolare ring maintained 97% of its baseline throughput. While NATS lost over 40% because retransmission storms overwhelmed the broker. The key takeaway: circular propagation trades some peak efficiency for radical predictability in degraded networks, a trade-off that makes it invaluable for environments where connectivity is the exception, not the norm.

Implementing Circolare with Lightweight Gossip Protocols

Actualizing Circolare doesn't require reinventing the wheel. We piggybacked on the battle-tested gossip layer of the NATS cluster configuration but swapped its full-mesh routing table for a deterministic ring. Each node maintains a monotonically increasing sequence number and appends it to every circulating message. When a node sees a message it already forwarded, it drops the duplicate-a mechanism similar to the duplicate detection in Ethernet's spanning-tree protocol, but applied at the application layer.

Node membership is managed by an efficient SWIM-like gossip that piggybacks neighbor changes on the message stream itself. This collapses discovery and data propagation into one channel, slashing the control-plane chatter that often limits scaling on constrained devices. The entire runtime, written in Rust to minimize garbage collection pauses, consumes less than 8 MB of RAM per node-making Circolare viable even on ARM Cortex‑M class microcontrollers that power many industrial sensors.

Circolare for Edge Computing and IoT Sensor Meshes

Edge deployments live and die by the reliability of their inter-node communication. On an offshore platform, you can't just add another fiber link; you have to wring every ounce of resilience from the hardware you already have. Circolare shines here because it transforms each edge gateway into a full participant that can make local decisions without waiting for a cloud-based arbiter. In our current architecture, 42 gateways sprinkled across the North Sea form a self-organizing ring that streams vibration data to the nearest onshore collector, with failover loops that activate within 200 milliseconds of a gateway going dark.

This isn't just theory. A 2023 trial with a European energy consortium showed that a 15-node Circolare mesh reduced data gaps during satellite uplink outages by 67%, compared to a star-topology MQTT broker. The ring's natural load-balancing also eliminated the "hot neighbor" problem that plagues tree-based aggregation hierarchies, resulting in a more uniform battery drain across battery-powered sensor nodes.

Security Implications of Circular Message Propagation

Any architecture that forwards messages hop by hop invites scrutiny around data integrity and unauthorized injection. With Circolare, every message is signed with a per-node Ed25519 key. And the ring collectively verifies the signature chain. If a malicious or compromised node attempts to alter a message, the cryptographic chaining breaks. And a salted hash of the last valid state is gossiped to all neighbors, triggering an automatic re-keying procedure.

We also implemented a rotating ring token, inspired by legacy token-ring security but modernized with time-based one-time passwords (TOTP) derived from a pre-shared seed. Only the node holding the current token can initiate a new message stream, preventing replay attacks. In penetration tests, the Circolare ring withstood attempts to inject fabricated telemetry data with zero false positives, while the same attacker successfully poisoned a plain RabbitMQ cluster within minutes.

Monitoring Circolare Networks: Observability Pitfalls and Tools

Hop-by-hop architectures are famously hard to monitor because a single missing ACK can look like an endpoint failure. We learned this the hard way when an early production incident led to 30-minute debug sessions in Grafana dashboards. To tame the complexity, we instrumented each Circolare node with OpenTelemetry traces that embed the ring position as a span attribute, enabling Jaeger to reconstruct the entire message journey from origin to consumer.

Key metrics we track include ring rotation time (the latency for a token to complete one full circuit), neighbor reconnect count, message duplicate rate. An unexpected spike in duplicates often signals an intermittent network flap that the self-healing mechanism is masking. We've open-sourced a Prometheus exporter that turns these traces into real