I Simposio de Postgrado 2023. Ingeniería, ciencias e innovación

I SIMPOSIO 2023 DATA-DRIVEN DISTRIBUTION- FREE MODEL OBSOLESCENCE DETECTION AND DIAGNOSIS USING MUTUAL INFORMATION ABSTRACT Model obsolescence diagnostics (MOD), in other words, determining the moment and reasons why a model ceases to represent the phenomenon it is supposed to characterize, is an open problem in disciplines such as failure prognostics and machine learning (ML). MOD is an enabling technology in high- impact applications associated with real-time monitoring, such as alerting the existence of unwanted operational changes, system degradation, and the need to re-train ML-based models. We propose a decision scheme for detecting and diagnosing model obsolescenceusinganon-parametricmutual information (MI) estimator to determine when a system has changed and which components are responsible for those changes. Our core contributions are twofold. On the one hand, we demonstrate a formal linkage between (i) the problem of model obsolescence detection and (ii) independence testing between the model residual and its inputs. On the other hand, we analyze the empirical performance of a proposedMOD framework based on a novel characteristic vector, namely the “Residual Information Feature,”consistingofMIestimationsbetweenthe input features and model residuals. For our detection scheme, we prove strong consistency, exponential decision convergence (on the nominal case), and control of significance levels and power of the test. For our diagnosis scheme, we perform tests in a classification setting. Our results, as obtained with no restriction on systems nor variable distributions, show our method being broad enough to be applied in real-life scenarios, which other SOTA methods could not address due to their restrictions. Camilo Ramírez 1 *, Tomás Rojas 1 , Jorge Silva 1 , Marcos Orchard 1 , Ferhat Tamssaouet 2 1Information and Decision System Group, Dpto. Ingeniería Eléctrica, Universidad de Chile. 2PROMES - CNRS, Université de Perpignan. *Email: camilo.ramirez@ug.uchile.cl