American Statistical Association (ASA), Section on Statistical Learning and Data Science (SLDS)
August webinar: Statistical methods for assessing the factual accuracy of large language models

Record: August 29, 2024

Presenter: Emmanuel Candès is the Barnum-Simons Chair in Mathematics and Statistics, professor of electrical engineering (by courtesy), and a member of the Institute of Computational and Mathematical Engineering at Stanford University.

Before arriving at Stanford, Candès was the Ronald and Maxine Linde Professor of Applied and Computational Mathematics at the California Institute of Technology.

His research interests are in computational harmonic analysis, statistics, information theory, signal processing and mathematical optimization with applications to the imaging sciences, scientific computing and inverse problems.

Dr. Candès has received numerous awards including the Alan T. Waterman Award from NSF, which is the highest honor bestowed by the National Science Foundation, and recognizes the achievements of early-career scientists. He has given over 60 plenary lectures at major international conferences. He was elected to the National Academy of Sciences and to the American Academy of Arts and Sciences in 2014.

Abstract: We develop new statistical methods for obtaining validity guarantees on the output of large language models (LLMs). These methods enhance conformal methods to filter out claims (hallucination removal) while providing a finite-sample guarantee on the error rate of what it being presented to the user. This error rate is adaptive in the sense that it depends on the prompt to preserve the utility of the output by not removing too many claims. We demonstrate performance on real-world examples.

This is joint work with John Cherian and Isacc Gibbs.

For more information about or to join ASA SLDS, visit

https://community.amstat.org/slds/home

https://www.amstat.org/