Tom's Life Bits

Let me start with asking you a question. How many of you uses facebook messenger in your smart phone? Most of you will. But how many of you know that you are getting your notifications through MQTT protocol? Probably not many. I came to know it just a few days ago. Why FB is using MQTT in their messenger? Now a days, if you go to any IoT related sites, you willl be seeing this term MQTT quite often. It seems like MQTT is the HTTP of IoT age. But then I asked a question myself – why should I choose MQTT over some well known, widely used, universally supported protocol like HTTP?

“MQ Telemetry Transport (MQTT) is a publish/subscribe messaging protocol particularly well-suited for working with limited computational power and lean network connectivity. IBM and systems provider Eurotech first developed MQTT, and then contributed the protocol to OASIS( the Organization for the Advancement of Structured Information Standards ). The “MQ Integrator SCADA Device Protocol” is an old name for what is now known as MQTT. MQTT is designed to be open, simple and easy to implement, allowing thousands of lightweight clients to be supported by a single server. These characteristics make it ideal for use in constrained environments or low-bandwidth networks with limited processing capabilities, small memory capacities and high latency. The MQTT design minimizes network bandwidth requirements while attempting to ensure reliability of delivery. The protocol is already used in a wide variety of embedded systems. Hospitals use the protocol to communicate with pacemakers and other medical devices. Oil and gas companies use MQTT to monitor thousands of miles of oil pipelines.”[more here] “MQTT is extremely efficient publish/subscribe reliable messaging protocol. A protocol that enabled devices to open a connection, keep it open using very little power and receive events or commands with as little as 2 bytes of overhead. It offers reliable behavior in an unreliable or intermittently connected wireless environments. “Last will & testament” lets all other subscribers know immediately if a client disconnects abruptly. “Retained message” enables any new node connecting to a broker to get messages on topic that are retained by the broker.”[more here] Complete specification of MQTT is available at http://public.dhe.ibm.com/software/dw/webservices/ws-mqtt/MQTT_V3.1_Protocol_Specific.pdf

To use MQTT, first thing you need is a broker. A broker is a server that routes published messages to subscribers. You can either setup one yourself with MQTT broker softwares like Mosquitto which is closely related to Eclipse Paho project or use those available on internet for free use such as iot.eclispe.org port 1883, CloudMQTT etc. You can also choose the option of a more secure connection SSL/TLS connection to broker and this is available with test.mosquitto.org.

Now you have fixed your broker, you can tell your client to connect to that broker and publish/subscribe messages. Each message is associated with a topic. A topic is a place holder to store your data and forward it to clients whom want to listen to your client. Lets pretend you have a MQTT sensor node in your garden and it will update measured field temperature and soil measure to a remote server. What you can do is publish the data “temp=21C,hum=65%” to a topic like “user/vish/area/garden/zone1/data”. As you can see topic name resembles closely to some kind of REST request. Also remember - MQTT is data agnostic - so you can send a JSON object like {“temp”:“21C”,”hum”:“65%”} also, which may make it easy for a client to parse. And if you want to send it in XML, you can do that also. If some client wants this data, they can just subscribe to that topic from the broker and broker takes care of forwarding it to them. If more than one client wants this information, they all can subscribe to the same topic and broker will take care of those information deliveries. Your client can just publish it once and go. Once your client is connecting to a broker, it can also set 'Last Will and Testament' which will be the message published to that topic's subscribers in case the client leaves the 'chat' ungracefully. You can also set QoS which is like how reliable you want to make the connection, in case of intermittent connection and packet losses.
QoS( Quality of Service ) : QoS describes how reliable you want your data delivery to be which in a way implies how hard the broker/client will try to ensure that a message is received. MQTT offers three levels of QoS settings:

1. QoS0 : Atmost once delivery - The broker/client will deliver the message once, with no confirmation.
2. QoS1 : Atleast once delivery - The broker/client will deliver the message at least once, with confirmation required.
3. QoS2 : Exactly once delivery - The broker/client will deliver the message exactly once by using a four step handshake.
 

Last Will and Testament : While connecting to a broker, a client can tell that it has a will statement which is a message that it wishes the broker to send if it disconnects unexpectedly. The will message has a topic, QoS and retain status just the same as any other message.

Retained Topics : A retained message is message the broker will keep even after sending to all current subscribers of that topic. If a new client wants to subscribe to that topic in future, then this stored message is forwarded to that client by the broker. So if you have infrequently updating topic( such us your garden temperature which is set to update once in an hour ), a new client which want to subscribe that topic can get “last known good” message in that topic.

IoT is the place of billions of devices and most of them are power constrained. When they communicate, the network load is definitely going to shoot up. We have been using HTTP for these device to device communications for a while. But let's check what happens when I want to update some status to a remote server using HTTP. I fired my terminal and send a simple message to some server and this is the result :

Can you see that?? To send just 21 bytes of payload, I ended up sending 201 bytes of header data. Let's calculate efficiency( I'm not sure whether I can call this efficiency, but for now please bare with me ).

21 / (21+201) = 9.5%

That clearly means we are in trouble if we want to sens some sensor readings to a remote server via HTTP . Let's check what happens when you send same data through MQTT to a broker. MQTT has thing thing called “Fixed Header” of 2 bytes. And if your payload is less than 128 bytes, then this will be the only additional information in the packet you send. And for every additional 127 bytes of payload, you need to add 1 byte of variable header to the packet, uptill a payload of 256MB. So now the efficiency is

21 + (21+2) = 91.3%

That's almost 10 times increase. . Let's do one more experiment. This time, send 127 bytes and calculate efficiency :

HTTP : 127 / (127+201) = 38.6%

MQTT : 127 / (127+2) = 98.4%

It's clear, MQTT rocks!! This because HTTP is a (basically)document centric protocol while MQTT is agnostic of data content and transfers simple byte arrays, making drip-feeds of updating information trivial. Also if you want to send data to multiple clients with HTTP, you will end up sending it to every subscriber individually. With MQTT, send it once to broker and broker takes care of rest. Again thumps up MQTT! Now what about those power constrained devices who can't afford to spend too much of their battery on sending data? Luckily Stephen Nicholas has done an excellent job on power profiling MQTT on android and I'm citing an overview of his results :

        >  93x faster throughput
        >  11.89x less battery to send
        >  170.9x less battery to receive
        >  1/2 as much power to keep connection open
        >  8x less network overhead
        

With that, I hope you are convinced that MQTT is the language of IoT. Then let's get started with MQTT. What I'm going to do is to first walk through a few tools I think will be very useful to fiddle with MQTT and then I will go ahead setup my local MQTT broker, add a few launchpads/arduinos as clients to it and finally add openHAB to send and receive MQTT packets and control my home network.

Tools
Before getting started, let's go through a few tools which will help you to “sniffer” to your network's mqtt communication.

1. MQTT Lens( Chrome App ) : I'm a fan of this, for it's simplicity.I will be using this in my demo. This is quite intuitive to use and opensource too. You can add it to your chrome from chrome store.