The chair of RCL Group, Andrea Bondavalli, just finished his Keynote speech at The 41st International Symposium on Reliable Distributed Systems (SRDS 2022)

Keynote Details

Title: Dependability Challenges in Safety-Critical Systems: the adoption of Machine learning

Abstract: Machine Learning components in safety-critical applications can perform some complex tasks that would be unfeasible otherwise. However, they are also a weak point concerning safety assurance. We will illustrate two specific cases where ML must be incorporated in SCS with much care. One is related to the interactions between machine-learning components and other non-ML components and how they evolve with training of the former. We argue that it is theoretically possible that learning by the Neural Network may reduce the effectiveness of error checkers or safety monitors, creating a major complication for safety assurance. An example on automated driving is shown. Among the results, we observed that indeed improving the Controller could make the Safety Monitor less effective; to a limit where a training increment makes the Controller's own behavior safer but results in the vehicle to be less safe. The other one regards ML algorithms that perform binary classification as error, intrusion or failure detectors. They can be used in SCS provided that their performance complies with SCS safety requirements. However, the performance analysis of MLs relies on metrics that were not developed with safety in mind and consequently may not provide meaningful evidence to decide whether to incorporate a ML into a SCS. We analyze the distribution of misclassifications and thus show how to better assess the adequacy of a given ML.