MQTT broker: how to choose, secure and operate one

The broker's role in an MQTT system

A publishing client connects to the broker and sends a message with a topic name. A subscribing client asks for messages matching a topic filter. The broker routes matching publications according to the protocol, its configuration and access permissions. A gateway might publish a temperature measurement while a monitoring application and an integration service receive separate copies.

For a general introduction, read what is MQTT?. The MQTT protocol guide explains packet and delivery behaviour in greater depth. Keeping these responsibilities distinct helps during troubleshooting: a device can connect successfully while being forbidden to publish, and a message can reach the broker while no application has subscribed to its topic.

Design a usable topic structure

Topics are case sensitive strings used for routing. A structure such as site/building/device/measurement can be useful, but MQTT does not enforce that particular meaning. Choose a convention that reflects access boundaries and application needs. Keep stable identifiers separate from display names that people may change, and avoid placing credentials or unnecessary personal information in topic paths.

Subscribers can use supported wildcard filters to receive groups of topics. Permissions should limit the filters available to each client rather than granting unrestricted access by default. Test exact topic names, capitalisation and wildcard behaviour during commissioning. An apparently silent subscription is often a routing or permission problem, not evidence that the physical sensor stopped measuring.

Authentication and authorisation are separate

Authentication establishes which client is connecting. Authorisation determines what that identity may publish or subscribe to. A password protected broker can still be insecure if every authenticated client may read and write every topic. Prefer individual device or gateway identities where practical, and grant only the topic permissions needed for each operational role.

Use encrypted transport with certificate and hostname validation for connections crossing untrusted networks. MQTT itself does not make an unencrypted connection confidential. Manage secrets outside source code, rotate credentials through controlled procedures and revoke access when equipment is retired. Decide how certificate expiry and identity changes will be monitored before they cause a fleet wide outage.

Understand the delivery guarantees

MQTT offers quality of service levels with different transport guarantees. QoS 0 provides delivery without an acknowledgement exchange. QoS 1 supports at least once delivery, so duplicates can occur. QoS 2 uses an additional exchange to provide exactly once delivery within the relevant protocol interaction. Higher QoS also increases processing and communication overhead.

These guarantees do not establish that a business application saved a measurement or acted on it exactly once. Broker forwarding and database persistence are different operations. Use stable event identities and idempotent storage when retries could produce duplicates. Measure end to end receipt separately from connection status, especially when telemetry supports billing, operational history or important maintenance decisions.

Retained messages are not an archive

A retained publication allows the broker to make a stored message available to later matching subscribers. It is often useful for a latest known state, provided the receiving application also checks timestamps. An old retained temperature value should not appear to be a current reading merely because a new subscriber has just received it.

Retention is not a substitute for a historical store, and one retained value does not reconstruct everything that happened while an application was offline. Define which topics use retention and how obsolete values are cleared. For IoT sensors, distinguish latest state, event notifications and time series history; each may require different treatment even when all three use the same broker.

Sessions, persistence and reconnects

Session configuration influences what state can survive disconnection. Client identifiers, subscription settings and version specific session options need deliberate management. Reusing the same client identifier for two independent clients can cause connection disruption. Conversely, changing identities on every restart can prevent the recovery behaviour expected from a persistent session.

Broker persistence and offline queues also have limits. Specify storage capacity, message expiry, queue policies and the behaviour when limits are reached. Test a consumer that disconnects and returns after a realistic outage. A broker described as persistent still needs verified restart recovery, and its persistence does not guarantee that an unacknowledged reading exists safely in the final application.

Availability and operating visibility

Monitor more than whether the service process is running. Useful observations include active clients, failed authentication, rejected publications, queue growth, dropped messages and storage pressure. Watch for reconnect storms after network outages. They can create substantial load even when the normal reporting rate is modest, especially if thousands of devices retry at the same time.

If clustering or failover is required, verify the chosen implementation's behaviour rather than assuming all brokers work identically. Test subscriptions, retained messages and session recovery across a node failure. Document maintenance procedures and ownership. A simple single service can be suitable for a pilot, but its operating limits should be explicit before critical applications depend on it.

Connect the broker to useful applications

Payloads need a shared contract because the broker does not interpret their business meaning. Include device identity, measurement time, values and units where appropriate. Validate messages before storing them. See IoT devices for acquisition considerations and industrial IoT for the additional context required around machines and operational networks.

When selecting a broker, evaluate protocol versions, authentication, topic permissions, observability, operational support and tested recovery alongside throughput. The connector catalogue helps scope integration with consuming applications. Start with a representative workload, deliberately test failures and confirm stored readings. A dependable MQTT broker deployment combines appropriate messaging behaviour with secure access and a verified application data path.