Collecting data from IoT devices on Azure
Even if IoT term is present as a buzzword nowadays, it does not hide the fact it becomes a trend. All manufactures want connect their products to internet. Sometimes it makes absolutely great sense and sometimes it is just a rotten vogue. Sometimes you want to buy smart device shouting information into the world and sometimes you want to get silent thing without any communication module in it.
In this article I will ponder about those Things making sense and having good reason broadcasting data over internet. I will classify these things into a few categories from data point of view. And finally I will name a few Azure services which could be used to collect produced data.
Data, Data, Data
I have always close to electronics and hardware itself. I built my own small circuits when I was a little boy, and later on high school and university. I worked in semiconductor industry for more than 20 years for the most famous companies and firms. My clients were electronic engineers which wanted automate the production. All my software was more or less about collecting data from the manufacturing equipment. Even those data were not flown on internet, they were transfer over its older brother private intranet.
All these applications give me a view on the IoT problem from software perspective. In foundation all the Things produce a lot of data and those data needs to be collect, proceed, store and finally analyzed. Next text will be mainly focused on collecting information.
Collection of data can vary on hardware and information size and type. How many devices are in the system tens or millions? How much data they send one value or complex structure? How often they send data once per day, week or continuous flow? How data is treat when device is offline is it forgotten or it will be send later with proper time-stamp? Is application real-time or not? Is application for private sector or public consumers?
Those are only a few questions that are better to answer before designing and creating architecture. Even it sounds great to prepare robust solution with far far vision to have billions of devices in wild, I will suggest to start small and upgrade later when business and application are proven. Let me classify data collection on those questions.
Small data flow – usually a few devices connected into own system. As a typical example could be home automation. The system is installed on home computer and all devices and sensors are connected over the wifi or intranet.
Medium data flow – hundreds or thousands of devices connected to system. This category has no solid boundaries and siting somewhere between.As a typical example I can name production automation. The system is installed on several servers running in local or remote datacenter collecting different data from various production equipment.
Big data flow – millions of devices connected to system. The obvious example is collection meteoric data. The system is installed on the cloud datacenter and collects data like temperature from devices installed in all world locations.
Collecting data from devices can be implemented in several ways on Azure. From your own service that has maximum control of data flow but on other hand it requires to implement everything. To IoT Suite which is a complex service for collecting data from devices as well as managing device portfolio.
The communication protocol over the internet is http and almost all today’s devices have http module in it. So it is straight forward you have REST service for communication with your IoT. There are a few ways how to host the REST service and I select 2 of them.
This is platform as a service for hosting web applications on Windows IIS. The benefit is that you do not case about managing neither operation system nor IIS. It provides you great features like auto-scale, high availability, or fail-over. The service is simple to setup and for the most solutions it is a great start point.
This service can help to secure your API on several levels. If you need to quickly implement a security layer for your IoT stuff, this service could help you. It supports authentication and authorization, CORS and it could be a proxy for your already existing REST service.
This could simply extend your initial solution hosting your REST service in Web App. You will start API App with same interface and that will call your REST service functions. As second step you will exchange public IP to be your new API App and make you REST service private reachable only from proxy.
Azure Service Bus
The service bus is really great solution for IoT. According to your need you can use Queue or Topic. They both will reliably store all messages from your devices and implemented listeners will proceed them. If anything happen and listener will crash, the message will stay in queue and become visible after setup timeout.
With service bus queues you can simply implement opposite communication direction and send messages from server to your devices. Of course you will need as many queues as the number of your live devices.
If your devices produce enormous number of messages Azure has special service Event Hub that can collect millions of messages per second.
This solution provides really complex services for big and professional IoT application. It provides preconfigured solutions, enhanced security, supports set of operations systems and protocols, enables millions of connected devices, analyzes data, auto-scale and many more.
If you want to start with any IoT project, there is not reason why stay out of Azure. You can implement broad set of application and host them on Azure. Just select proper service which will suitable for your solution. If you want to have everything under control your own REST service hosted in Web App will be a great choice. If you have a big ambitions choose IoT Suite and start with proof of concept, later you can scale it to support millions of devices.