Primers
Robot communication and control stacks
How the parts of a robot talk to each other, how robots talk to computers and to each other, and the software stacks built on top. There are many choices at every layer, and most of them are good at something. This primer is about telling them apart: what each is for, and when to pick one over another.
Pick your level at the top of the page.
High school
Three layers
A robot’s messages pass through layers, a bit like a letter:
- The link is the road: the wires inside the robot, or the radio between a robot and a computer.
- The messaging is the postal service: how a message is addressed, who gets it, and what happens if it’s lost.
- The stack is everything built on top: ready-made programs for driving, mapping, or moving an arm, that already know how to talk to each other.
Inside the robot: wires
The small computer in a robot, a microcontroller, talks to its sensors and motor drivers over a few wires. Each kind has its own rules: I²C shares two wires among many sensors, SPI is faster with a few more wires, serial (UART) is one wire each way between two devices, and CAN is a sturdy shared cable that cars use to join dozens of computers together.
Between robots and computers: radio
Wi-Fi reaches across a house and carries a lot. Bluetooth Low Energy is short range and very low power, the way a phone talks to a watch. Some chips have their own simple radio links too, like Espressif’s ESP-NOW, which joins small boards directly without a Wi-Fi router.
Two ways to send a message
- Publish and subscribe is like a radio station. A sensor publishes readings on a named channel, a topic, and any program that wants them subscribes. The sender doesn’t know or care who’s listening.
- Request and reply is like a phone call. One program asks another a question and waits for the answer: “what’s your battery level?” or “drive to the door, and tell me when you’re there”.
Some systems put a broker in the middle, a post office every message goes through. Others connect programs directly. A broker is simpler to set up and lets programs come and go, but if it stops, everything stops.
Picking one
| If you’re building… | A common choice |
|---|---|
| A few small gadgets on Wi-Fi reporting to one computer or a web page | MQTT, with a broker |
| A robot with a camera, a map and many programs on one computer | ROS 2 |
| A drone | A flight controller that speaks MAVLink |
| Two boards side by side in one robot | A serial link, or CAN |
Check yourself
- A temperature sensor sends a reading every second to whoever wants it. Publish/subscribe or request/reply?
- What’s the risk of sending every message through one broker?
- Why don’t cars use Wi-Fi to join the computers inside them?
<details><summary>Answers</summary>
- Publish/subscribe: it sends to a topic, and doesn’t need to know who’s listening.
- If the broker stops, nothing gets through.
- A wire is far more reliable and predictable. Radio can be blocked, jammed or delayed, and a car’s brakes can’t wait. (CAN was designed for exactly this.)
</details>
College
The links
| Link | Speed, roughly | Reach | Typical use |
|---|---|---|---|
| I²C | 100 kbit/s to 1 Mbit/s | Centimetres, one board | Sensors, small displays |
| SPI | Tens of Mbit/s | Centimetres | Displays, SD cards, fast sensors |
| UART (serial) | Up to a few Mbit/s | Up to a metre or so | Board to board, GPS, debugging |
| CAN | 1 Mbit/s; CAN FD faster in its data phase | Tens of metres | Motor controllers, cars, robot arms |
| Ethernet | 100 Mbit/s and up | 100 m | Computer to computer, cameras, lidar |
| Wi-Fi | Tens to hundreds of Mbit/s, shared | A building | Robot to computer, fleets |
| Bluetooth LE | Hundreds of kbit/s | Metres | Phones, low-power sensors |
| ESP-NOW | Small packets, no router | Like Wi-Fi | Espressif boards talking directly |
CAN is worth knowing in detail. Every device on the bus can send, frames carry 8 bytes (64 in CAN FD), and when two send at once the frame with the lower ID wins without either being corrupted. So urgent messages get low IDs and always get through first. That predictability is why it’s inside cars and many robot arms.
The messaging
How each one works:
| Pattern | Broker | Runs over | |
|---|---|---|---|
| MQTT | Publish/subscribe | Yes | TCP, WebSockets |
| DDS, under ROS 2 | Publish/subscribe; ROS 2 adds request/reply | No: peers find each other | UDP, with multicast discovery |
| Zenoh | Publish/subscribe, plus queries | Optional routers | TCP, UDP, QUIC, serial, WebSockets |
| ZeroMQ | Publish/subscribe, request/reply, pipelines | No | TCP, between processes |
| gRPC | Request/reply and streams, typed | No | HTTP/2 |
| WebSockets | A two-way pipe; you design the messages | No | TCP |
| MAVLink | Small fixed messages for vehicles | No | Serial, UDP |
Where each fits, and what it costs:
| A late joiner gets the last value | On a microcontroller | Licence | |
|---|---|---|---|
| MQTT | Yes, if it was sent retained | Yes, widely | Open standard (OASIS). Mosquitto broker: EPL-2.0 or EDL-1.0 |
| DDS | Yes, with transient local durability | Through micro-ROS and an agent | Open standard (OMG). Fast DDS: Apache-2.0. Cyclone DDS: EPL-2.0. RTI Connext: commercial |
| Zenoh | Not by default | Yes, zenoh-pico | EPL-2.0 or Apache-2.0 |
| ZeroMQ | No | Rarely | MPL-2.0 |
| gRPC | No: not publish/subscribe | Rarely | Apache-2.0 |
| WebSockets | No | Yes | Internet standard (RFC 6455) |
| MAVLink | No | Yes: made for it | MIT for its generated code |
All but RTI Connext are free to use; the running cost is the computer they run on. A broker for a small fleet fits on any small computer.
When MQTT, when something else
MQTT fits when many small devices on Wi-Fi report to, and take commands from, a central place: a computer, a dashboard, a web page (browsers can speak it over WebSockets). Devices can sleep and drop off and rejoin. Retained messages give a newcomer the current state straight away, and a last will tells everyone when a device vanishes. The broker is one simple, well-understood place to secure and log.
Something else fits better when:
- Large, fast data between programs on one robot: camera images, lidar scans, maps. ROS 2 (on DDS, or Zenoh) is built for this, and can pass data through shared memory without copying it over the network.
- You want a robotics ecosystem, not just messages: drivers, navigation, arm planning, simulation, recording and playback. That’s ROS 2’s real value.
- Mixed networks, from microcontrollers on serial to servers across the internet, with no single broker. Zenoh was designed for this.
- A typed request/response API between services, like a control panel asking a robot for its configuration. gRPC gives you defined message types and generated code in many languages.
- Hard real-time control: a motor current loop, an arm joint. None of the above. That loop runs on the microcontroller, or over a bus built for it, like CAN or EtherCAT.
- A drone or rover with an existing autopilot: MAVLink, because that’s what PX4 and ArduPilot speak.
Control stacks
- ROS 2 (Apache-2.0) organises a robot as many small programs, nodes, joined by topics, services and long-running actions. On top sit ready-made packages: ros2_control (Apache-2.0) for driving motors and joints through hardware drivers, Nav2 (mostly Apache-2.0) for navigation, MoveIt 2 (BSD-3-Clause) for arm motion planning.
- micro-ROS (Apache-2.0) brings ROS 2 to microcontrollers, including the ESP32. The microcontroller speaks a lightweight protocol, XRCE-DDS, to an agent on a bigger computer, which joins it to the rest of ROS 2.
- PX4 (BSD-3-Clause) and ArduPilot (GPLv3) are complete autopilots for drones, planes, boats and rovers, controlled from a ground station over MAVLink.
- Your own firmware plus MQTT is a stack too, and for a fleet of small, simple robots it’s often the right size: each robot runs its own fast loops and reports over MQTT.
Check yourself
- A robot streams a 30 frames-a-second camera to a mapping program on the same computer. MQTT or ROS 2, and why?
- Twenty classroom sensors report to a web dashboard. A new browser tab should show every sensor’s latest reading at once. What do you use, and which feature?
- Why shouldn’t a motor’s current loop run over Wi-Fi?
- Two motor controllers on a CAN bus send at the same instant, with IDs
0x120and0x080. Which gets through first?
<details><summary>Answers</summary>
- ROS 2: large, fast data between programs on one computer, which it can share without copying. MQTT would send every frame through a broker.
- MQTT, with each reading published retained: a new subscriber gets the latest straight away. The browser connects over WebSockets.
- Wi-Fi’s delay varies, from milliseconds to far more when the air is busy, and a current loop runs thousands of times a second. It belongs on the microcontroller.
0x080: the lower ID wins arbitration.
</details>
University
Where each loop lives
A robot is a stack of control loops, each with its own deadline, and each belongs where its deadline can be met:
| Loop | Rate, typically | Where it runs | Over |
|---|---|---|---|
| Motor current, joint position | 1 to 20 kHz | The microcontroller or motor driver | On-chip, or CAN or EtherCAT |
| Wheel speed, balance, obstacle stop | 100 Hz to 1 kHz | The microcontroller | Local sensors |
| Path following, local planning | 10 to 100 Hz | The robot’s main computer | ROS 2 topics, or in one program |
| Mission, fleet coordination, the operator | 0.1 to 10 Hz | Elsewhere | Wi-Fi: MQTT, Zenoh, ROS 2 |
Wi-Fi’s latency is usually a few milliseconds but has a long tail: tens or hundreds of milliseconds when the channel is busy or the robot roams between access points. So a loop closed over Wi-Fi must tolerate that, and the robot must be safe if the link goes silent. A command should carry a deadline or expire (a deadman): a robot that hears nothing new for half a second stops.
Delivery guarantees
- At most once: send and forget. Lost messages stay lost. Right for a stream where the next reading replaces the last, like a 50 Hz pose.
- At least once: resend until acknowledged. A message may arrive twice, so the receiver must cope: make commands idempotent (“set speed to 0.3”) rather than incremental (“speed up by 0.1”), or number them and drop repeats.
- Exactly once: costs a longer handshake. MQTT’s QoS 2 does it between a client and the broker, not end to end.
MQTT’s QoS 0, 1 and 2 map onto these. Each guarantee covers one hop: QoS 1 from a publisher means the broker has it, not that any subscriber has. And at QoS 0, a client library saying a message was “published” means only that it left the client.
DDS has a richer set of QoS policies, set per topic and matched between publisher and subscriber: reliability (best effort or reliable), durability (volatile, or transient local, which keeps recent samples for late joiners, like MQTT’s retained), history depth, deadline (a sample must arrive this often), and liveliness (notice when a publisher dies, like MQTT’s last will). A publisher and subscriber with incompatible policies simply don’t connect, which is a common first ROS 2 puzzle.
Brokers, peers and discovery
A broker is a single point of failure and an extra hop, but it’s also a single place for authentication, access control, logging and bridging, and it means no device needs to know where any other is.
Brokerless systems must find each other. DDS does it by default with multicast on the local network. On wired
Ethernet that works well. On Wi-Fi, multicast frames aren’t acknowledged and are often sent at the slowest rate, so
discovery traffic is slow and unreliable, and grows with every node. The usual fixes are a discovery server
(Fast DDS has one), or routing through Zenoh, which ROS 2 supports as a middleware (rmw_zenoh, a fully supported
option since the Kilted release in 2025). The broker comes back, in a different form.
Messages on the wire
How a message is encoded matters as much as how it travels. JSON is readable and self-describing, but large and slow to parse. Protocol Buffers (gRPC’s default) and CDR (DDS’s) are compact binary formats with a schema, generated code, and rules for adding fields without breaking older readers. MQTT doesn’t care: its payload is bytes, so the encoding is your choice, and so is keeping both ends agreed on it.
Security
Each needs it turned on. MQTT brokers offer TLS, passwords or client certificates, and per-topic access control. DDS has the DDS Security specification, which ROS 2 calls SROS 2. Zenoh and gRPC support TLS. A robot on a shared network with security off will take commands from anyone who can reach it.
Check yourself
- A robot receives “move forward 10 cm” over QoS 1 and drives 20 cm. What happened, and how would you design the command instead?
- Fifty ROS 2 nodes on robots sharing a Wi-Fi network take minutes to find each other, and the network slows. Why, and what would you change?
- A teleoperated robot’s operator loses Wi-Fi mid-drive. What should the robot do, and what makes that happen?
- A ROS 2 subscriber asks for reliable and the publisher offers best effort. What happens?
<details><summary>Answers</summary>
- At-least-once delivery sent it twice: the acknowledgement was lost, so it was resent. Send an absolute target (“go to position 1.20 m”) or number each command and drop repeats.
- Multicast discovery on Wi-Fi: unacknowledged, slow, and growing with every node. Use a discovery server, or rmw_zenoh with a router.
- Stop. Commands that expire: the robot stops when it hears nothing new within a short deadline. MQTT’s last will, or DDS liveliness, tells everyone else it’s gone.
- They don’t connect: the policies are incompatible, and the subscriber gets nothing (ROS 2 reports the mismatch as an event).
</details>
Try it
Four ways to send a small message there and back, side by side: plain UDP, ZeroMQ, MQTT through a broker (Mosquitto), and Zenoh. Then a listener that starts late, to see what each gives it. You need only a container runtime. Start a shell with Python in it:
container
container run --rm -it docker.io/library/python:3.14-slim bash
docker
docker run --rm -it python:3.14-slim bash
podman
podman run --rm -it docker.io/library/python:3.14-slim bash
Inside it, install a broker and three libraries, start the broker, and save the program:
apt-get update && apt-get install -y mosquitto
pip install paho-mqtt==2.2.0 pyzmq==27.3.0 eclipse-zenoh==1.10.2
mosquitto -d
cat > comms.py <<'EOF'
"""The same small message there and back, 1,000 times, four ways, on one machine. Then a listener that joins late.
Everything runs on this one machine, in Python, so the times are for comparing the four, not real network numbers.
"""
import socket
import statistics
import threading
import time
import paho.mqtt.client as mqtt
import zenoh
import zmq
N = 1000
MSG = b"x" * 32
def report(name, times):
us = sorted(t * 1e6 for t in times)
print(f"{name:28} median {statistics.median(us):6.0f} µs slowest 1% {us[int(len(us) * 0.99)]:6.0f} µs")
def plain_udp():
echo = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
echo.bind(("127.0.0.1", 9000))
def serve():
for _ in range(N):
data, addr = echo.recvfrom(1500)
echo.sendto(data, addr)
threading.Thread(target=serve, daemon=True).start()
me = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
times = []
for _ in range(N):
t = time.perf_counter()
me.sendto(MSG, ("127.0.0.1", 9000))
me.recvfrom(1500)
times.append(time.perf_counter() - t)
report("UDP, no library", times)
def zeromq():
ctx = zmq.Context()
rep = ctx.socket(zmq.REP)
rep.bind("tcp://127.0.0.1:5555")
def serve():
for _ in range(N):
rep.send(rep.recv())
threading.Thread(target=serve, daemon=True).start()
req = ctx.socket(zmq.REQ)
req.connect("tcp://127.0.0.1:5555")
times = []
for _ in range(N):
t = time.perf_counter()
req.send(MSG)
req.recv()
times.append(time.perf_counter() - t)
report("ZeroMQ, direct", times)
def connected(name):
c = mqtt.Client(mqtt.CallbackAPIVersion.VERSION2, client_id=name)
c.connect("127.0.0.1", 1883)
c.loop_start()
while not c.is_connected(): # wait for the broker's yes before going on
time.sleep(0.01)
return c
def mqtt_broker():
echo = connected("echo")
echo.on_message = lambda c, u, m: c.publish("demo/pong", m.payload)
echo.subscribe("demo/ping")
me, got = connected("me"), threading.Event()
me.on_message = lambda c, u, m: got.set()
me.subscribe("demo/pong")
time.sleep(0.5) # let both subscriptions reach the broker
times = []
for _ in range(N):
got.clear()
t = time.perf_counter()
me.publish("demo/ping", MSG)
got.wait()
times.append(time.perf_counter() - t)
report("MQTT, through a broker", times)
echo.disconnect()
me.disconnect()
def zenoh_peers():
quiet = '{scouting: {multicast: {enabled: false}}, '
a = zenoh.open(zenoh.Config.from_json5(quiet + 'listen: {endpoints: ["tcp/127.0.0.1:7447"]}}'))
b = zenoh.open(zenoh.Config.from_json5(quiet + 'connect: {endpoints: ["tcp/127.0.0.1:7447"]}}'))
pong = a.declare_publisher("demo/pong")
a.declare_subscriber("demo/ping", lambda s: pong.put(s.payload))
got = threading.Event()
b.declare_subscriber("demo/pong", lambda s: got.set())
ping = b.declare_publisher("demo/ping")
time.sleep(0.5)
times = []
for _ in range(N):
got.clear()
t = time.perf_counter()
ping.put(MSG)
got.wait()
times.append(time.perf_counter() - t)
report("Zenoh, direct", times)
a.close()
b.close()
def late_joiners():
print("\nA reading is sent, then a listener starts. What does the listener get?")
# MQTT: one reading kept by the broker (retained), one not
pub = connected("robot")
pub.publish("robot/battery", "7.4 V", qos=1, retain=True).wait_for_publish() # QoS 1: wait for the broker's ack
pub.publish("robot/speed", "0.3 m/s", qos=1).wait_for_publish()
late, got = connected("late"), []
late.on_message = lambda c, u, m: got.append(f"{m.topic} = {m.payload.decode()}")
late.subscribe("robot/#")
time.sleep(0.5)
print(f" MQTT: {', '.join(got) or 'nothing'} (only the retained one)")
# ZeroMQ publish/subscribe: no broker, nothing kept
ctx = zmq.Context()
pub0 = ctx.socket(zmq.PUB)
pub0.bind("tcp://127.0.0.1:5556")
pub0.send(b"robot/battery 7.4 V")
sub0 = ctx.socket(zmq.SUB)
sub0.connect("tcp://127.0.0.1:5556")
sub0.setsockopt(zmq.SUBSCRIBE, b"robot/")
pub0.send(b"robot/speed 0.3 m/s") # sent just after it connects: still lost, it's not joined yet
time.sleep(0.5)
pub0.send(b"robot/speed 0.4 m/s")
time.sleep(0.2)
msgs = []
while sub0.poll(100):
msgs.append(sub0.recv().decode())
print(f" ZeroMQ: {', '.join(msgs) or 'nothing'} (missed both earlier ones)")
# Zenoh: nothing kept by default either
s = zenoh.open(zenoh.Config.from_json5('{scouting: {multicast: {enabled: false}}}'))
s.put("robot/battery", "7.4 V")
zgot = []
s.declare_subscriber("robot/**", lambda x: zgot.append(f"{x.key_expr} = {x.payload.to_string()}"))
time.sleep(0.5)
print(f" Zenoh: {', '.join(zgot) or 'nothing'}")
s.close()
plain_udp()
zeromq()
mqtt_broker()
zenoh_peers()
late_joiners()
EOF
python comms.py
UDP, no library median 20 µs slowest 1% 31 µs
ZeroMQ, direct median 40 µs slowest 1% 65 µs
MQTT, through a broker median 122 µs slowest 1% 305 µs
Zenoh, direct median 125 µs slowest 1% 313 µs
A reading is sent, then a listener starts. What does the listener get?
MQTT: robot/battery = 7.4 V (only the retained one)
ZeroMQ: robot/speed 0.4 m/s (missed both earlier ones)
Zenoh: nothing
Your times will differ, but the order is usually the same. Everything ran on one machine, so these are the libraries’ own costs: tens to a hundred or so microseconds. Over Wi-Fi each round trip adds milliseconds, far more than any of them. For most robots, the choice between them is about what they do, not how fast they are.
The late joiner shows the biggest difference. MQTT’s broker kept the retained battery reading and handed it over at once. ZeroMQ, with no broker, kept nothing: the subscriber missed even the reading sent just after it connected, because joining takes a moment. Zenoh, by default, kept nothing either.
Two things to change:
- Take QoS 1 off the two publishes in
late_joiners(): save a copy withcp comms.py original.py, thensed -i 's/, qos=1//' comms.py, and run it again. The speed reading, never retained, now reaches the late listener as well. At QoS 0 the publisher’s “sent” means only that the message left the client, and the broker hadn’t handled it yet when the listener subscribed. QoS 1 waits for the broker’s acknowledgement. - Stop the broker, with
kill $(pidof mosquitto), and run it again. UDP and ZeroMQ run as before, then the program stops at MQTT withConnectionRefusedError: without the broker, nothing gets through. Start it again withmosquitto -d.
Type exit to leave.
Learn more
- The MQTT 5.0 specification and the Mosquitto broker.
- ROS 2’s documentation, and its list of distributions and their support dates.
- Zenoh, and rmw_zenoh for ROS 2.
- ZeroMQ’s guide, long and very good on messaging patterns in general.
- gRPC’s core concepts.
- MAVLink, and micro-ROS.
- CAN FD, from CAN in Automation, and EtherCAT.