Certified Machine-Learning Models

The massive adoption of Machine Learning (ML) has deeply changed the internal structure, the design and the operation of software systems. ML has shifted the focus from code to data, especially in application areas where it is easier to collect samples that embody correct solutions to individual instances of a problem, than to design and code a deterministic algorithm solving it for all instances. There is an increasing awareness of the need to verify key non-functional properties of ML-based software applications like fairness and privacy. However, the traditional approach trying to verify these properties by code inspection is pointless, since ML models’ behavior mostly depends on the data and parameters used to train them. Classic software certification techniques cannot solve the issue as well. The Artificial Intelligence (AI) community has been working on the idea of preventing undesired behavior by controlling a priori the ML models’ training sets and parameters. In this paper, we take a different, online approach to ML verification, where novel behavioral monitoring techniques based on statistical testing are used to support a dynamic certification framework enforcing the desired properties on black-box ML models in operation. Our aim is to deliver a novel framework suitable for practical certification of distributed ML-powered applications in heavily regulated domains like transport, energy, healthcare, even when the certifying authority is not privy to the model training. To achieve this goal, we rely on three key ideas: (i) use test suites to define desired non-functional properties of ML models, (ii) Use statistical monitoring of ML models’ behavior at inference time to check that the desired behavioral properties are achieved, and (iii) compose monitors’ outcome within dynamic, virtual certificates for composite software applications.