To use the correct tools to implement a task is so important. When playing with IoT applications, you will inevitably touch:
- MQTT, for handling easily the messaging,
- InfluxDB database, to store the data in a timestamped fashion (time-series arrays), and
- Grafana, to visualise the data
This is good, but do you mean that I need to install everything from scratch? Ohh, that’s a lot.
Fortunately, everything is so easy. Just use docker containers and set up that chain a few minutes.
Do you want to start with me? Okay. Let’s start.
We need to install Docker by downloading and executing the convenience script in our ubuntu machine. Type in the terminal:
curl -sSL https://get.docker.com | sh
Then, we add our user (non-root user) to the docker group in order to run containers without using the sudo command.
sudo usermod -aG docker <username>
So replace <username> with the name of the user that you use. Finally, test that everything works as expected:
docker run hello-world
Great. Now it’s time for our MQTT broker (server) to be installed and become live. One popular option is to use mosquito by the eclipse foundation. Easy:
docker pull eclipse-mosquitto docker run -it -p 1883:1883 -p 9001:9001 eclipse-mosquitto
In the second command, we run the container using the following switches:
- -it: run it in interactive mode by allocating a tty for the container process
- -p: expose the container’s internal port to the host port
Now, MQTT broker is up and running; you could use. Do you want to test it directly? Let me guide you.
First, we need to install mosquitto clients and use it directly from the terminal commands to publish and listen to topics.
sudo apt-get update sudo apt-get install mosquitto-clients
So now, if we decide that we want to publish in a topic called “test” the temperature of the kitchen, we could type in terminal:
mosquitto_pub -m "10" -t "test"
To listen for messages in this specific topic, we need to run:
mosquitto_sub -t "test"
The switches in the commands are self-explanatory. If you are a GUI type of person, you could use MQTT.fx, download the deb package and install it, easy as that:
sudo dpkg -i ./mqttfx-1.7.1-64bit.deb
Now, we need to set the InfluxDB. We need to pull it from the hub and run it.
docker pull influxdb docker run -d -p 8086:8086 -v influxdb:/var/lib/influxdb --name influxdb influxdb
In the run command, we execute the container in the detached mode (-d) and create and connect a docker volume to store data associated with the InfluxDB container (-v). Finally, we assign the container the influxdb name (–name).
Before we continue, we need to configure a database. How to do that in our case that influxDB runs in a detached mode? Easy. First, let’s get in the container with exec command
docker exec -it influxdb influx
Then, we create a database called “sensors”, we create a user “telegraf” and we give him full permissions in the newly created database.
create database sensors use sensors create user telegraf with password 'password' grant all on sensors to telegraf exit
The database is there and we need to connect with the MQTT listening to the data in a specific topic. No, you do not need to write code for that. You just need to use telegraf. Let’s install it and configure it to listen in a specific MQTT topic and store the time series data directly to the database.
docker pull telegraf docker run --rm telegraf telegraf config > telegraf.conf
Yeap, as you guessed, we need to edit the telegraf.conf file.
First, to remove the # for comment and enable the input
[[inputs.mqtt_consumer]] servers = ["tcp://192.168.1.81:1883"] topics = ["sensors"] data_format = "influx"
and then similarly the output
[[outputs.influxdb]] urls = ["http://192.168.1.81:8086"] database = "sensors" skip_database_creation = true username = "telegraf" password = "password"
Yes, as you correctly guess, the IP address of the machine that I execute both MQTT broker and InfluxDB server is the same “192.168.1.81” machine. The last step is to start the telegraf module like:
docker run -v /home/<username>/:/etc/telegraf:ro telegraf
To test the flow of data, we can publish a message and check the database to find out if the point stored there, as expected.
The command to publish a measurement into the topic is:
mosquitto_pub -h 192.168.1.81 -m "temp,site=kitchen value=20" -t "test"
The measurement called “temp”, it is equal to 20 and it has a tag with the room location (i.e. kitchen)
Now, if we login again in the database using the docker exec command, we can list the measurements writing something like:
use sensors select * from temp exit
Last but not least, we use again a docker container to set up Grafana and we run as usual
docker pull grafana/grafana docker run -d -p 3000:3000 grafana/grafana
- Installing Docker and Docker Compose on the Raspberry Pi in 5 Simple Steps
- How to Install Docker on Raspberry Pi
- How to Install Mosquitto MQTT Broker/Server on Ubuntu 16.04
- Raspberry Pi IoT: Sensors, InfluxDB, MQTT, and Grafana