Forrester Research: IoT Upsets Application Development- Hello Streaming & Machine Learning!

by Mike Gualtieri of Forrester Research  (adapted from Forrester Research white paper “IoT Upsets Application Development”)


This is a follow on to the recent IoT post: Cisco Study: Few IoT projects survive proof-of-concept stage; Electric Imp on why not?


Internet of things (IoT) is the latest transformational technology, extending ubiquitous connectivity and control to the physical world. The applications are limited only by our imagination and our ability to build both cloud-based and enterprise infrastructure to deal with the flood of data and interactions that IoT represents.

Because The IoT Spigot Is Always Open, App Development Teams Need A New Stream-Driven, Machine Learning Approach!  That’s explained in this article.

Key Takeaways:

  • IoT Apps Blur And Bridge Physical And Digital Worlds: IoT applications comprise a collection of devices that may have sensors and/or actuators that bridge the digital and physical worlds, which can turn contextual information from sensors into insight and action.
  • Hello, Streaming:  Forget most of what you have learned about app development. If you want to design IoT applications, you need to design apps that can ingest and analyze streaming data in real time.
  • Hello, Machine Learning:  Machine learning can analyze sensor and action data from devices to automatically improve program logic to make an app smarter over time.

IoT Applications Turn Real-World Context Into Physical Action:

Typical enterprise applications are “request and response.” AD&D pros design apps to get input from users, process the request, and then respond to it. These traditional apps are user-driven. That approach is insufficient for IoT applications because IoT applications must be ready to receive and react to information from device sensors and actuators, typically in real time. IoT applications help companies bridge the physical and digital worlds, turning contextual information from sensors into insights and action.1 IoT applications comprise a collection of devices that may have sensors only, actuators only, or both sensors and actuators. This is a system of connected devices that can provide applications with context, actuation, and learning.

  • Context: a view of the physical world. IoT sensors measure physical parameters of people, places, or things. They emit live data such as motion, temperature, voltage, pressure, audio, video, and position, with the list going on and on. This information from a system of IoT sensors can provide applications with real-time context — what is happening right now at a single physical sensor and what is happening right now among all of the sensors in the system. That can be an amazing amount of contextual information that traditional applications can’t handle.
  • Actuation: the ability to change the physical world. IoT actuators make something happen in the physical world such as opening a lock, instructing a furnace to turn down the heat, or displaying customized context within an app. IoT applications that include IoT actuators make changes to the physical world, often in real time.
  • Learning: an increasing corpus of knowledge. IoT apps gather a tremendous amount of information from device sensors and generate log data about actuation commands and their results in the physical world. This data can be input to machine learning algorithms and advanced analytics to improve the application’s decision with every iteration

IoT Applications Are Stream-Driven:

Stream-driven means that data can drip periodically like dew from a leaf, pour down like the mighty Niagara Falls, or flow continuously like the Danube from one or many devices into an IoT application. The IoT application uses some of this data immediately to actuate devices, and it stores the rest of it to analyze later.2 The Nest thermostat is a classic easy-to-understand example. It continuously receives temperature, humidity, and motion data from the home sensor and uses that to actuate the heating or cooling system. It also uses the data to learn about the home owner’s habits and make automatic adjustments (e.g., based on patterns of travel).

Other IoT applications are more complex than that because they rely on many more devices and much more complex logic. But most require an application architecture that responds to streams of sensor data that help these components work in unison.

Multiple IoT devices. An IoT application must have a way to communicate with the devices that comprise the system of devices for the IoT application. IoT device connectivity and management solutions provide two-way communication and device control.

Stream ingestion and analytics. Sometimes, data spikes fast and furiously, and at other times, it drips in. IoT applications must be able to ingest data from IoT devices at the rate that it is emitted. Ingestion, however, is not enough. Streaming solutions must also be able to analyze data in real time. Many of these platforms also have smaller footprint versions that can perform ingestion and analytics at the edge — that is, you can deploy them on an IoT device. To help AD&D pros learn more about streaming ingestion and analytics, Forrester evaluated 15 vendor solutions in this space, including some vendors that offer commercial versions of open source, in a recent Forrester Wave™ report.

Machine learning. To know and not to do is not yet to know. That ancient proverb sums up the importance of including an advanced analytics regime in your IoT applications. Advanced analytics that uses machine learning can create predictive models or materialize other analytics that you can use in the IoT application’s logic. You can dump all of the data that comes from the IoT devices and all of the actuations that the application takes into a data lake for analysis to create models that the IoT applications can use. Creating a data lake in Hadoop and then using Apache Spark for advanced analytics is becoming increasingly common for implementing advanced analytics.

Actuation controller. IoT applications must trigger an action in the physical world. An IoT application must be able to issue commands to devices with actuators to make those devices do something such as open a gate, turn on a camera, or change the speed of a delivery drone.

Logic. The brain of an IoT application is the current context, programmed logic, and learned logic. The current context is a data model that represents the current situation that the sensors in the IoT devices report and any relevant external context from other applications. The simplest of current context might be that the room temperature is 66 degrees Fahrenheit. The programmed logic continuously monitors the current context and takes actions as necessary. Simple programmed logic could be that if the temperature drops below 68 degrees Fahrenheit, then the heat goes on. Additional programmed logic could be that the heat goes off when the temperature is 2 degrees higher than the target temperature of 68 degrees Fahrenheit. Learned logic modifies the programmed logic by analyzing the data and behavior of the application over time. Learned logic comes from advanced analytics and machine learning algorithms that recognize repeated patterns and turn those into logic.