American Statistical Association (ASA), Section on Statistical Learning and Data Science (SLDS)
May webinar: Stability of black-box algorithms

Recorded on May 31, 2023

Presenter: Dr. Rina Foygel Barber is a Professor in the Department of Statistics at the University of Chicago. Before starting at U of C, she was a NSF postdoctoral fellow during 2012-13 in the Department of Statistics at Stanford University, supervised by Emmanuel Candès. Rina's research interests are in developing and analyzing estimation, inference, and optimization tools for structured high-dimensional data problems such as sparse regression, sparse nonparametric models, and low-rank models. She works on developing methods for false discovery rate control in settings where undersampled data or misspecified models may be present, and for distribution-free inference in settings where the data distribution is unknown. She also collaborates on modeling and optimization problems in image reconstruction for medical imaging.

Abstract: Algorithmic stability is a framework for studying the properties of a model fitting algorithm, with many downstream implications for generalization, predictive inference, and other important statistical problems. Stability is often defined as the property that predictions on a new test point are not substantially altered by removing a single point at random from the training set. However, this stability property itself is an assumption that may not hold for highly complex predictive algorithms and/or nonsmooth data distributions. This talk will present two complementary views of this problem. In the first part, we show that it is impossible to infer the stability of an algorithm through "black-box testing", where we cannot study the algorithm theoretically but instead try to determine its stability properties by the behavior of the algorithm on various data sets, when data is limited. In the second part, we establish that bagging any black-box algorithm automatically ensures that stability holds, with no assumptions on the algorithm or the data.

For more information about or to join ASA SLDS, visit

https://community.amstat.org/slds/home
  / slds_asa  
https://www.amstat.org/