Introduction
-
Heterogeneity. The inherent heterogeneity of the devices in a large-scale IoT system makes the connectivity and coordination process very difficult [14]. Moreover, due to various protocols, the generated data usually have different formats, sizes, and timestamps, which form a challenging task for further processing, transmission, and storage [15]. Meanwhile, performing general computation on servers with heterogeneous operating systems or runtime also forms a non-ignorable aspect of the challenge.
-
Accuracy. Algorithms used by AI need to be well developed and tuned to understand and interpret data so that more accurate decisions can be made [16]. Additionally, due to the highly dynamic nature of the physical world, a once trained AI model may not perform well all the time, i.e., the models need to be refined using the new-coming IoT data to achieve better performance.
-
We present the computing model and the architecture of Sophon Edge, an edge-cloud collaborative AIoT platform.
-
We present the pre-implemented operators supported in Sophon Edge, which saves development and operation efforts for the user of the platform, i.e., the developers of various AIoT applications.
-
We present the process of model evolution and updating using the Sophon Edge platform, which helps to build agile and data-driven AIoT applications [17].
-
We implement the Sophon Edge platform and demonstrate its usability through a real-world use case.
Related work
Edge computing for IoT
Applying AI to IoT
The sophon edge platform
Unified device model
Pipelined computation task
Category | Name | Usage |
---|---|---|
Input | Time Series Database | The “Time Series Database” operator is used to observe certain data tables in a specified time series database (such as InfluxDB). When new data is inserted into the table, the operator will be able to obtain the data immediately and output it in a specific format to subsequent operators. |
Device Input | The “Device Input” operator provides real-time data access to a specified connected device on the Edge Node. The product type of the device should be specified in the operator, which determines the format of the data. | |
Processing | Filter | The “Filter” operator is used to filter the input data stream. The operator sets a conditional expression. Only when the expression is true, the corresponding data will be forwarded to subsequent operators, otherwise, it will be discarded. |
Sample | The “Sample” operator samples the input data stream according to a specific step size as well as a time interval. The operator can reduce the processing frequency of real-time data and thus optimize resource usage. | |
Time Window | The “Time Window” operator provides the function of batch forwarding the data arriving in a specified period. | |
Change Trigger | The “Change Trigger” operator monitors its input and only generates outputs when data changes occur. This operator is suitable for data streams that have frequent data duplication in a period. | |
Aggregate | The “Aggregate” operator provides a variety of methods for aggregate computation on a data stream, such as sum, count, mean, mode, etc., which can complete some common statistical tasks. | |
Custom Function | The “Custom Function” operator provides the ability to calculate data streams with user-defined functions. The operator supports multiple custom computational expressions for each piece of data flowing through the operator and merges the results of the computation into the final output data stream. | |
Apply Model | The “Apply Model” operator calls a certain AI model to make inferences upon its input, and outputs the inferencing results. Note that the model to be called should have been already deployed on the Edge Node. | |
Output | Device Output | The “Device Output” operator provides the function of controlling devices, including setting the properties and calling the services of certain devices. |
Notification | The “Notification” operator provides a logging-like function that persists input data as messages on the Edge Nodes. These messages can be set to different levels, including INFO, WARN, and ERROR. The notification messages can be used as inputs for some operators. |
Category | Name | Usage |
---|---|---|
Input | File Input | The “File Input” operator reads a video file from a specific device as the data source. |
Real-time Stream Input | The “Real-time Stream Input” operator binds a specific device using the RTSP or RTMP protocol and outputs the real-time data stream to subsequent operators. | |
USB Camera Input | The “USB Camera Input” operator reads and outputs data from a specific USB camera. | |
Processing | Encode | The “Encode” operator encodes the original video stream into a specific format, e.g. H.264, H.265, JPEG. |
Decode | The “Decode” operator decodes the encoded video or JPEG images to obtain raw video information. | |
Mux | The “Mux” operator encapsulates the encoded video into a specific format, e.g. FLV, MP4. | |
Demux | The “Demux” operator parses the input data into encoded video. | |
Resolution | The “Resolution“ operator transforms the resolution of the input video stream into a specific setting, e.g. 1280*720. | |
Frame Rate | The “Frame Rate” operator limits the maximum number of frames per second of the input video stream. | |
Region Of Interest | The “Region Of Interest” operator specifies a region in the input image or video stream, so that subsequent “Apply Model” operators will only make inferences towards the specified region in the images. This avoids unnecessary processing and saves resources. | |
Apply Model | The “Apply Model“ operator binds a specific AI model, and the input video stream is filtered somehow to get the model input. Moreover, the results returned by the model are saved in the output video stream, such as drawing recognition frames. | |
Output | File Output | The “File Output” operator saves the input video stream as a local file. |
Alert | The “Alert” operator treats its inputs as alerts and sends them to the cloud. |
Edge-cloud collaborative architecture
Agile AIoT
An application example
Background
Implementation
Results
Discussion
Advantages
AWS IoT | Alibaba Cloud AIoT Native | Tencent Cloud IoT Hub | Baidu Tiangong IoT Hub | Sophon Edge | ||
---|---|---|---|---|---|---|
Device Access | MQTT | \(\checkmark \) | \(\checkmark \) | \(\checkmark \) | \(\checkmark \) | \(\checkmark \) |
COAP | - | \(\checkmark \) | \(\checkmark \) | \(\checkmark \) | \(\checkmark \) | |
HTTP | \(\checkmark \) | \(\checkmark \) | \(\checkmark \) | - | \(\checkmark \) | |
Others | LoRaWAN | - | - | - | Modbus/OPCUA/... | |
Computing | Rule-based | \(\checkmark \) | \(\checkmark \) | \(\checkmark \) | \(\checkmark \) | \(\checkmark \) |
Complex | \(\checkmark \) | \(\checkmark \) | \(\checkmark \) | \(\checkmark \) | \(\checkmark \) | |
Data Storage | Edge | - | - | - | \(\checkmark \) | \(\checkmark \) |
Cloud | \(\checkmark \) | \(\checkmark \) | \(\checkmark \) | \(\checkmark \) | \(\checkmark \) | |
AI Model | Train | Cloud | Cloud | Cloud | Cloud | Cloud |
Inference | Cloud | Edge | Edge | Edge | Edge |