HistoTrust: tracing AI behavior with secure hardware and blockchain technology

The added value of blockchain technology to meet the specific features of a smart manufacturing use case has been shown in [6]. Compared to a centralized solution based on digital certificates and PKI, the Ethereum-based solution offers a more refined management of security and privacy at the expense of performance. In [2], HistoTrust demonstrates that performance can meet the needs of a real-time usage when using a blockchain.

The EmLog framework [7] is presented as “the first attempt at preserving off-the-shelf ARM development board hosting OP-TEE”. EmLog implements a secure logging system from end-to-end between embedded constraint devices and a remote database. HistoTrust introduces an architecture design and an on-board implementation design using off-the-shelf secure hardware components, as OP-TEE and TPM 2.0 [8], that goes beyond EmLog solution and achieves the EmLog perspectives. Preserving forward security thanks to the one-way hash chain scheme introduced by Shneier and Kelsey [9], EmLog and SGX-Log [10] are not designed for multi-stakeholders contexts and may suffer of data losses in case of power failure.

In the Logs system EngraveChain detailed in the paper [11], the data history is ciphered, then registered in an Hyperledger Fabric ledger. This implementation lacks agility because the blockchain is not designed to store large volumes of data, nor confidential data even encrypted. Moreover, the ciphering of recorded data in a ledger implies a complex key management. The blockchain technology provides by design the tamper-resistance of the recorded transactions history forming the ledger. HistoTrust provides an attestation scheme securing the history of data issued from distributed devices.

An Ethereum ledger maintains the history of cryptographic attestations of data produced by distributed devices owned by multiple stakeholders. The blockchain technology allows to share these cryptographic evidences between the stakeholders, ensuring mutual trust. In addition, the raw data is kept by its owner who ensures its persistence and confidentiality.

Based on an Ethereum blockchain, BlockPro [12] presents a decentralized architecture of IoT devices. The authenticity of the data-emitting devices is achieved through a challenge to the IoT device submitted to its PUF (Physical Unclonable Function). But it is not mentioned how the address of the account issuing the transactions is built and how it is linked to the PUF. The paper [13] shows that the dissociation between IoT devices and validation nodes is a powerful architecture that HistoTrust exploits.

Attestation schemes based on the use of a TPM offer standard solutions allowing the authentication of a platform by a remote device [14, 15]. The authors of [16] highlight the question of the certification of sensor data, even by a trusted platform. The tension is tangible between privacy on the one hand and trust on the other. Privacy requires the protection of confidential data, while trust requires guarantees between the stakeholders working in a given ecosystem.

The principle of remote attestation is described in depth in [15]. The Trusted Platform Module (TPM) is the targeted device enabling the endorsement of attestation keys that the manufacturer, the vendor or the owner may own. The attestation scheme follows recommendations and standards provided by the Trusted Computing Group (TCG) [14]. Attestation aims at proving to a remote verifier the property of a target by supplying an evidence over a network. It consists in three stages: (1) key provisioning, (2) attestation process and (3) verification process.

The field of eXplainable AI (XAI) raises major attention as an important concept that increases the trust in AI-based systems and applications. The need of both interpretability and explanation methods has been recently highlighted by the NIST [1]. A large variety of approaches have been proposed to enlighten the blackbox paradigm of deep NN models [17] even for modern architectures.

The purpose of our work is not to introduce a new methodology to explain the intrinsic behavior of a Machine Learning (ML) model, but to frame the implementation of an AI in an embedded device in such a way that confidential data, presented to a third party, can be trusted to explain the behavior of an embedded NN. Our contribution is rather in the area of cyber robustness of embedded AI in the presence of multiple distributed NNs.

In a factory, many actuators participate in the assembly of a product on a production line (see Fig. 1). Physical devices that embed inference engines, i.e., a NN previously trained to recognize determined patterns in an image, generate the digital commands sent to the actuators. The device may integrate several sensors and a camera. A picture of the product is taken before acting. This picture is presented in input of the NN to request an inference that contains heuristics, i.e., probabilities that the pattern recognized in the image corresponds to the learned patterns. This inference will guide the decision about the next action the actuator should perform.

In the event of an incident creating a financial loss, it is necessary to find the causes and eventually to charge the costs to the accountable stakeholder. However, the presence of AI makes difficult the reproduction of the decisions. So, how to determine who is accountable for the damage? In particular, who is accountable for the decisions that command the actuators? If the NN recognizes the digit “2” instead of the digit “8”, is the error attributable to the learning quality? A configuration and/or system integration fault? A lack of operator guidance? Noisy input data? A physical or logical attack on the electronic devices? A network attack?

Smart robots are often equipped with cameras that allow them to photograph the part of the product on which they will operate. The image is then analyzed, potentially with a classifier, and depending on the patterns recognized, the action is determined. For this work, we use a classical digit recognition task with the MNIST dataset [18] as it represents one of the most popular benchmarks in the ML literature with which many architectures can be tested (from shallow fully-connected networks to deeper convolutional NN). MNIST is composed of 60,000 training images of grayscale handwritten digits and 10,000 examples for test. Each sample is a grayscale 28x28 image (784 pixels) with the associated label “0” from “9”. This dataset offers a school case with a known and qualified open-source model. The integration made for the use case can be generalized to other computer vision tasks, specific to the problem to solve.

For a given input image of the NN, the output inference is composed of 10 heuristics that correspond to the recognition probability of each digit from “0” to “9”. An example is shown in Fig. 2 with the recognition of the digit “2” with the probability 0,99 (99%).

In this work, we consider a deep NN model that performs a supervised classification task with the following formalism. Input-label pairs \((x,y) \in \mathcal {X} \times \mathcal {Y}\) are sampled from a distribution \(\mathcal {D}\). The NN model \(M_{\Theta } : \mathcal {X} \rightarrow \mathcal {Y}\), with parameters Θ, classifies an input \(x \in \mathcal {X}\) to a label \(M_{\Theta }(x) \in \mathcal {Y}\). The parameters are optimized during the training phase in order to minimize a loss function \({\mathscr{L}}\big (M_{\Theta }(x),y\big )\) (e.g., the cross-entropy loss) that evaluates the quality of a prediction compared to the ground-truth label. For the sake of readability, the model M_Θ is simply noted as M.

We distinguish a model, M, as an abstract algorithm from its physical implementations M. One model M (e.g., a CNN trained on MNIST for digit recognition) can be implemented for inference purpose in a microcontroller or in FPGA. Functionally, the embedded models rely on the same abstraction M but strongly differ in terms of implementation along with their respective hardware environments. Thus, there is no equivalence between M and its embedded variants.

Embed deep NN models on a constrained platform such as a 32 bits microcontroller usually needs model compression techniques to fit the model complexity to the hardware requirements [19]. More particularly, memory footprint is usually an important challenge: for a typical Cortex-M MCU, the trained parameters are stored in the Flash memory and, at inference time, the internal computations (mainly multiply-accumulations and non-linear activations) are processed in SRAM. Two classical approaches are used to fit state-of-the-art models: quantization and pruning. Although the learning process may require 32 bits floating point computations, at inference time, a low bitwidth representation of the parameters is sufficient and does not alter the performance of the model. Thus, most of the tools that enable NN embedding on MCU (such as STM32Cube.MX AI¹) propose a 8-bit quantization of the parameters. Pruning refers to techniques that cut useless connections in the network and rely on the fact that most of the models are over-parametrized. Both approaches can also help speeding up the inference process.

Two different architectures of model working on MNIST dataset have been used, a MLP and a CNN. Both needed to be small to fit hardware material limitations. As such, MLP is composed of an input (784 points due to the fact that the images must be flattened to be used ) and an output layer (10 neurons corresponding to number of label). This model has only 7850 trainable parameters which makes it a quite small model compared to others doing same task with additional intermediate hidden layers. However, model accuracy is just below 92%. Despite that state-of-the-art MLP model can reach higher accuracy on MNIST classification, this accuracy remains acceptable in light of model reduced architecture.

On the other hand, a CNN is also considered. This kind of model is divided in two parts with distinct goals. First layers and made for feature extraction (convolution, max pooling layers etc.) whereas end layers generally are regular MLP with dense layers.

CNN are particularly efficient and adapted for image recognition and classification as shown in Fig. 3. Indeed, despite its reduced size, model reaches accuracy slightly over 96% for MNIST image classification.

In order to implement deep NN models on microcontrollers such as STM32, we previously generate the model with Google Tensorflow [20]. The model architecture (number of neurons, layers, used activation functions) is created according to the target specification, an ARM Cortex-M4. Then, an empty model is trained with labelled data corresponding to the task to perform, the digit recognition, following a supervised learning paradigm. Validation and test of the dataset complete the training. The validation adjusts the hyper-parameters value and distinguish overfitting. The test qualifies the model performance with examples that have not been seen during the training phase. This allows the simulation of real model behavior while having ground-truth class for each example of the dataset. At the end, TensorFlow provides an accuracy score. The trained model characteristics (architecture, parameters and hyper-parameters values) composes the embedded NN in a “.h5” file.

During the production phase, the cryptographic attestations are registered in Ethereum ledger through a smart contract. The attestation process, detailed in Fig. 5, consists in computing the fingerprint of the latter dataset produced, which is included in the data field of an Ethereum transaction (Fig. 12). This transaction is signed in the TrustZone with sk which is also used to build the account address of the issuer device. To achieve the signature, the private key sk_c is accessed in the TPM permanent memory through the SPI bus and is deciphered in the TrustZone. The signed transaction is sent to the blockchain and a receipt is returned if the registration in the ledger is confirmed. The implementation of this attestation process is tricky because it must respect both temporal constraints and the real-time of the industrial app that produces new data. No data should be lost, due to processing time of the attestation app, power failure of the physical device, or latency of recording in the remote blockchain. In fact, the use of secure hardware components, as TPM and TEE, adds an overhead on the computing time to generate the attestation. The paper [2] presents a detailed study of the performance of HistoTrust according to the security level of the private key sk. On the one hand, on the blockchain side, a huge latency may be observed due to the time interval between two consecutive blocks. The delay between two blocks is very different from a blockchain to another. Ethereum implemented in private blockchain with the Clique algorithm [3] as consensus protocol provides by default a time interval around 12 s between two consecutive blocks. As a comparative example, two consecutive blocks are 10 min apart in the Bitcoin blockchain. On the other hand, the rate of data production by the real-time industrial app can be very high. To circumvent this problem, HistoTrust uses the receipt that confirms the registration of an attestation in the ledger to trigger the reading of a new dataset from the industrial app.

Two types of verifiers are distinguished:

This section briefly presents the IoT platform design. A STM32MP157-EV1 evaluation board is associated with a STPM4RasPI TPM Expansion Board. The STM32MP157 is a single board computer composed of a dual-core ARM Cortex-A7 core processor operating at 650Mhz forming a System-on-Module (SoM). The processor also integrates an ARM Cortex-M4 coprocessor, which makes it suitable for real-time tasks.

The dual-core ARM Cortex-A7 is very low-power processor designed for smartphone or edge devices. It includes both a normal world operating with a Rich OS and a secure world with a TrustZone operating with OP-TEE OS. The transition from the normal world to the secure world is done by setting the NS bit in the SCR register to 1. The executed code remains confidential and is protected against logical attacks.

The coprocessor ARM Cortex-M4 offers a real-time environment accessible from the normal world of the ARM Cortex-A7 to extend its computing capabilities and increase its performance while preserving low-power consumption. The functions embedded in the ARM Cortex-M4 are built upon the dedicated Hardware Architecture Layer (HAL). STMicroelectronics provides a protocol called RPMSG [21] to ensure the communication between the ARM Cortex-A7 micro-processor and the ARM Cortex-M4 micro-controller.

The daughter board STPM4RasPI completes the STM32MP157 with a TPM 2.0 from STMicrolectronics. This board is connected through the GPIO making the TPM accessible from the OP-TEE environment via the SPI bus. An Ethernet connection and a serial link enable the monitoring of the SoM. A small screen displays some information about the hardware configuration.

The ARM Cortex-A7 includes an open-source Trusted Execution Environment (OP-TEE) implementing the ARM TrustZone technology. At start, a secure boot is achieved according to the application note [22] relying on Elliptic Curve Brainpool-256 crypto-system. At start and during the execution in production mode, the integrity of the two embedded trusted apps is checked through the measurement process. To enable this, the fingerprint of the apps binary code is previously provisioned in the TPM PCR as explained in Section 5.1.2.

The integration consists to make the industrial app and the attestation app working together in the SoM as depicted in Fig. 5, while respecting the real-time constraint of the industrial app.

The industrial app is embedded in the normal world operating on a linux kernel as rich OS of the ARM Cortex-A7, with a part including the NN insulated in the ARM Cortex-M4. It handles the pictures coming from the attached camera in the ARM Cortex-A7. The pictures are transmitted to the NN in the ARM Cortex-M4, to request an inference. As output, the NN provides 10 heuristics, one by digit from “0” to “9”. The heuristics are carried to the ARM Cortex-A7. Generally, the recognized digit corresponds to the highest probability.

The communication protocol between the ARM Cortex-A7 and the ARM Cortex-M4 microcontrollers is suggested by STMicroelectronics in [21]. It implements a virtual interface, noted ttyRPMSG, that enables the exchange of small size messages and low data flows. The transmission of small images to the ARM Cortex-M4 with this protocol leads to a loss of information because the throughput is not sufficient. That’s why, HistoTrust implements a new communication scheme between the ARM Cortex-A7 and the ARM Cortex-M4 on the SoM. The virtual interface ttyRPMSG is used to notify the presence of data in a shared memory, accessible to both microcontrollers, and the direction of the communication.

Several buffers are implemented in the shared memory in order to handle full duplex communications without data loss. The data to attest composes the new entry written in the file #1. For the use case considered, the format of each new entry is as follows: The field url is a pointer to the raw data in entry of the NN, while the field hash is the hash of the raw data. The field inference is composed of the 10 values of heuristic, one for each digit from “0” to “9”. Each heuristic is a floating value coding a probability between 0 and 1.

The industrial app writes in real-time in the file #1 all the data produced that needs to be attested. The size of this buffer is not limited, as it is stored on an SD card of several GB. Only the industrial app is authorized to write in this file, while attestation app is authorized to read it. The receipt received from the blockchain confirms the registration of the attestation of the previous dataset in the ledger. This receipt triggers the read of the next dataset in the file #1. The file #1 is stored in persistent memory. If a power failure occurs, the data is saved and the attestation process resumes where it left off when the power returns. The file #1 may be ex-filtrated by its owner.

The attestation app includes a part located in the normal world and another part located in the secure world of the ARM Cortex-A7. The TPM is only accessed from the secure world, thanks to the integration of the SYS layer of the TPM stack in the OP-TEE environment. The lightweight mbedTLS library is also integrated into the OP-TEE environment. It provides cryptographic primitives and allows to build dedicated functions such as the Ethereum digital signature. In the normal world, low level commands enable the connection of the device with the remote blockchain through JSON-RPC convention.

All the devices are distributed on a local network following a star topology around an access point. A proxy allows the communication with the outside to enable raw data ex-filtration. A consortium Ethereum blockchain is locally deployed. Each stakeholder of the use case owns a validator node with a complete copy of the ledger, and has one vote in the consensus protocol. The validator nodes are depicted with a computer in Fig. 6. Thus, the governance of the system is ensured with equity and fairness by all the stakeholders.

The devices acting in the production line, are provided with the embedded apps, enabling to send transactions to the validator nodes. Thus, each device is the root-of-trust of the data it produces, forming a distributed root-of-trust network. The provisioning is done, independently by each device’s owner, prior to the deployment of the hardware in the factory. The management of the access rights and authorizations is done through smart contracts.

The blockchain is a time-stamping system consisting of a sequence of blocks spaced out in time. The recording of new transactions in the ledger is performed at low rate. We want to show that with our implementation of HistoTrust, the security and privacy properties brought thanks to the use of the blockchain have no impact on the industrial process flow and on the rate of inferences of the embedded NN.

To carry out the performance measurements, we consider the processing time of a transaction and its recording in the ledger by using the Ethereum Ganache simulator configured in automining, i.e., the transaction is recorded as soon as it arrives, without any latency due to the consensus protocol between the network validator nodes. The processing time of a transaction from the computing of the hash of the dataset to the receipt of the registration proof is estimated at 156ms.

We also considered the rate of inferences of the NN, and determined the processing time of an inference at the output of the NN, given an image presented at the input. This is estimated at 12.3ms.

So, the highest input rate of the NN is 1 picture every 12.3ms. We then chose the following measurement points: 1 picture every 20ms, 30ms, 50ms, 100ms, 150ms, 200ms, 300ms. For each rate considered for the input data, we determined the number of inferences contained in a transaction recorded by the Ganache simulator.

The results are presented on the graph in Fig. 7. For each measurement point on the x-axis, we presented 5000 images as input to the NN, and averaged the number of inferences contained per transaction on the y-axis.

This graph illustrates that the security and privacy properties, presented in the following paragraph and ensured with the blockchain technology, are implemented without impacting the performance of the industrial application.

In 1990, Reason [23] introduces the swiss cheese model to analyze the causality of an incident and manage risks. The physical device, that embeds the NN, integrates several security layers to protect and detect attacks or malfunctioning, as depicted in Fig. 8.

The first layer is a physical protection that prevents access to the components embedded in the smart robot, and that remains physically damaged in case of intrusion. By this way, succeeding in a physical attack on the electronic components that support the NN is difficult and leaves marks. The second layer is the cyber protection against logical attacks. The use of secure hardware components like TPM and OP-TEE to protect the cryptocraphic keys and seeds, is the foundation of this protection. The third layer is the detection of intrusions or tampering. At this layer, secure boot and measurement are deployed to monitor the integrity of embedded firmware and software. The fourth layer concerns the traceability to be able to understand what happens when the previous layers are bypassed. A blockchain is used to register the traces as attestations of the logged data produced by the embedded apps.

The assets to protect are the business-relevant data for the stakeholders. It is the logged data including all the relevant data produced by the physical devices, which contributes to make decisions of the digital command sent to the actuators. This includes inferences produced by the embedded NN (see Fig. 9). The authenticity must be ensured, as well as integrity and completeness.

The traceability is a valuable service to understand the origin and sequence of the events, while the raw data produced remains confidential to its owner. In order to reduce the attack surface on the electronic board, the different protection layers of Fig. 8 integrate several countermeasures. The goal is to fulfill these security requirements:

The threat events are the tampering of the data produced, the production of fake or dysfunctional data, the spoofing of data or issuing devices, and the theft of data.

The main sources of risks come from the following profiles:

The main stakeholders, involved in the smart manufacturing use case, are:

Table 1 shows the role that each stakeholder can play.

The provider of the smart robot may be negligent in providing an unreliable device, poorly configured, or in which bugs remain. In the event of a litigation, he must provide the integrity of the data requested by the auditor. Thus, it is the provider’s responsibility to maintain the tamper-resistance and confidentiality of his data. As there are usually several suppliers of smart robots in a factory, they are potentially concurrent.

This may be an incentive to obtain confidential data from their concurrent for analysis to gain market share. The expert is responsible for the learning of the AI and the decision of the embedded NN. He must be able to explain how the heuristics are derived. The manufacturer is physically present in the factory and has access to the smart robots. He may take any profile of attacker in order to hide a problem for which he is responsible and pass the blame on to another stakeholder. An operator or a maintenance agent may make a human error, and possibly seek to cover it up by destroying elements.

The auditor’s mandate is in the legal field, which gives him legal accreditation and independence from other stakeholders.

The blockchain provides an immutable history, shared among all stakeholders, of all past events. Figures 10 and 11 show an extract of the ledger securing a succession of blocks including the attestation history. The attestations are kept ordered, timestamped and of integrity.

Each block includes a tree of recorded transactions, as shown in Fig. 12. The sender address authenticates the issuer device, while the contract address authenticates the recipient smart contract. The field data includes the fingerprint of the raw dataset produced by the issuing device at the given time, whereas the field gas indicates the computing power required to execute the targeted smart contract in the blockchain. This value is an indicator of the energy consumed to execute an instance of the smart contract.

Until the request of personal data, any stakeholder member of the ecosystem can achieve the verification. The first step consists to get the recorded attestations of the considered time interval in the shared ledger. Each attestation authenticates the issuer device, as well as the owner who has endorsed his devices.

Each device’s owner may be requested to provide the raw data associated to the recorded attestations. As this data is confidential, only an accredited and independent auditor is authorized to perform this task regulated by the legal.

The provided data must be complete and of integrity; otherwise, the accountability of the owner is engaged, with the suspicion of hiding a fraud. Each owner is responsible for keeping and protecting its logged raw data.

Once the completeness and the integrity of the attested data established, the analysis of the raw data is conducted, in particular the analysis of the AI behavior. Each owner is responsible for providing an explanation of the behavior of its embedded NN.

At this stage, the analysis relies on tools and methods the expert used to explain the behavior of the NN, and on human expertise. For example, the picture presented in Fig. 13 is labelled “9”. However, the inferences of the embedded NN recognize the digit “4” with a probability of 68%, the digit “7” at 16% and the digit “9” at 5%. With a school case and a labelled image, one knows that it’s a “9”. But the pictures acquired by the smart robot’s on-board cameras are not labelled. And, only the explainability of the learning model and human expertise can remove the doubt on the most likely pattern. In a factory, the smart robots are supervised by human operators. Thus, one can consider that if the inference does not return any heuristics above a certain threshold, e.g., 71%, the decision is the accountability of the human operator. On the other hand, when the error is obvious, for example, the NN recognizes a “3” with 95% certainty when it is a “0”, the human operator will not be solicited, which can potentially lead to an incident on the production line. This may be due to an adversarial attack, i.e., an attack on the NN affecting the cyber protection layer (see Fig. 13), and not detected by the embedded system. The traceability implemented with HistoTrust allows to discover the cause.