Five Reasons to Switch From Project to Product Mentality
Articles
Many installations of IoT platforms fall short on operational readiness for managing all the data, devices, and business processes required for support. Although building out an IoT platform is a technical endeavor, operational requirements and readiness have a strong influence on the design and can become a costly afterthought. Cloud platforms provide tools to help manage IoT platforms, however, your journey to mastering the operations and management of your platform will need to be customized and tailored to business needs.
No matter how well the platform is architected there are still hundreds of ways breakdowns can occur while creating, sending, receiving, and transforming data. When breakdowns occur, the resulting discrepancies generate questions that can inundate the core IoT staff with hours of backlogged case-level research. Telemetry data will only reflect what happened, however, metadata can bring you closer to questions of why and help expose root cause. Defining the business process and generating the appropriate metadata prior to platform design will empower the business and service support teams to manage the platform much more effectively.
To understand the needs for metadata, we must first define the business processes and find all the players involved with platform support. Having workshops to provide a high-level concept of the IoT experience and defining the processes to support the platform will flush out the needs for metadata and the tools required. When designing the platform to meet the operational business requirements you may find the number of message types required to manage and maintain will far outnumber the telemetry messages gathered.
The team must think beyond the surface and explore the business processes required to support the platform. For example, when managing devices the business’s awareness of the device state is important. Although both AWS device shadows and Azure device twins give you a framework to manage desired versus actual states, you may find the need to capture this information historically to know the device state as a point-in-time. The business may require a better real-time device status that leverages heartbeat messages (messages sent every X minutes or seconds) to signify the device is online. The discovery of these requirements should happen upfront and be accounted for in the original design.
The state of device firmware also drives device messages and a need for more metadata. Yes, the shadows or twins can be leveraged to hold the current version, however, the state of a device’s firmware is much deeper than its current state. For example, if a device’s firmware update fails, the team may need to know status such as firmware downloaded, installing, successful, error information, and backout status becomes a critical meta must-have when managing millions of devices.
Edge and fog computing where data is aggregated and consumed before it reaches the cloud or final destination has also opened up the need to manage data consistency across aggregated data sources. For example, a large manufacturer of parking equipment also provides services for the support and operations of their equipment on the client sites has on-premise gateways that provide reporting and insights on a single parking facility. However, when aggregated at the regional level in the cloud clients need to see consolidated data that drills down to the same numbers the gateway devices provide. A series of audit messages containing summarized data and checkpoints to ensure the data transmit correctly, especially in the case of financial information may be required.
Gathering metadata can also be leveraged to provide insights to clients and avoid support calls. For example a project within the personal aircraft industry, telemetry events are captured during the flight from the avionics systems and captured in log files which were transmitted as soon as the aircraft touched the ground. The information is quickly aggregated then sent to a client mobile application where their aircraft information is displayed. In some cases, the pilot may land in a remote area or store the plane in a hangar that doesn’t have connectivity before the file is fully transmitted. When the pilot goes to review their flight history, flight data is missing. To proactively manage this situation we can generate messages both before and after the flight to know we are expecting a flight log and the expected size of the file size based on the time between the two telemetry messages. If a file doesn’t come in when expected or is partially transmitted we can provide information to the pilot via the mobile application, with troubleshooting information.
Metadata quickly becomes a hot commodity that will need to be leveraged across the business to support the platform and services. Leveraging current BI platforms and focusing on end-user enablement will remove the bottleneck of your IoT staff supporting data needs. The use of centralized log tables to capture information telemetry metadata is a good practice to have. Creating these logs in a non-schema bound solution within a datastore is ideal because the message formats may vary and change frequently. The datastore becomes an extrapolation layer to a schema bound OLAP data warehouse where it can be easily consumed through business intelligence tools.
A default schema for the normalization of metadata can be used and expanded upon to fit the situation. The result should be a simple analytics model that can easily be consumed by any business intelligence or analytics platform.
It’s also good practice to capture the most verbose information on any system errors to ensure the support teams have enough information to expose root causes. These models should start simple and grow over time.
There are four common domains where the metadata will play an immediate role in meeting operational needs reporting (BI), Alerts and Notifications, Application interfaces, and Analytics.
Capturing and leveraging IoT platform metadata plays a significant role in our ability to manage operations. Metadata can also be leveraged for client services and is at the core of automation and decision logic. You can ensure your IoT platform debuts with operational efficiency by defining business and support processes upfront, providing an operational focus in design, and capturing quality data, and providing a normalized repository with self-service analytics. Following these simple steps can help you proactively manage and integrate your IoT platform into your standard operations.
