Random Forests as Statistical Procedures: Design, Variance, and Dependence
Variance and dependence in random forests, including the uncertainty that remains as the number of trees grows.
arXiv
Biostatistics & data science
Nathaniel O’Connell, PhD
Associate Professor
Department of Biostatistics and Data Science
Division of Public Health Science
Wake Forest University School of Medicine
My research spans the intersection of statistical inference with machine learning and prediction modeling, Bayesian adaptive trial design, and collaborative biomedical research. Years of applied work have shaped the questions behind my growing focus on statistical methodology and theory.
01 — Research & practice
Active work
My growing focus on methods and theory comes from years of collaborative biomedical research. Over the last several years, much of my time has been spent thinking through and developing clinical prediction models, often using machine learning. Through that work, I’ve become increasingly bothered by how disconnected prediction modeling can feel from the ideas about uncertainty and inference I rely on as a statistician. I want to know how much confidence to place in an individual prediction, how uncertain a model’s performance estimate from cross-validation is, and how much I can trust the variables a model selects. I’ve often struggled to find satisfactory answers in the statistical literature, especially ones I can use to guide decisions in practice. These are questions I’ve been thinking about for years, and much of my current research is an effort to answer them. As I move forward in my career, I plan to spend more of my time on theory and methods, particularly addressing these issues.
A Theory of Cross-Validation Variance
How the learning procedure, performance metric, and resampling design contribute to uncertainty in cross-validation estimates.
Manuscript in preparation
Estimation of Cross-Validation Variance via Procedure Aligned Synthetic Resampling
Procedure-aligned synthetic resampling (PASR) for estimating uncertainty in cross-validation performance estimates.
Manuscript in preparation
Bayesian adaptive dose optimization with repeated patient outcomes
Bayesian adaptive trial designs that use repeated patient outcomes while accounting for dependence within patients and changes over time.
Work in progress
02 — Blog
Out-of-Bag Thoughts
Technical posts and things I explore just for fun.
The first post is on its way.
Keep in touch
Get in touch
I welcome conversations about research and collaboration.
Students and mentorship
I encourage students interested in machine learning, statistical inference, or adaptive trial design to reach out. Although our department does not have a graduate program of its own, I welcome opportunities to mentor students in other programs, including through research collaborations and service on dissertation committees. If your interests overlap with mine, please email me with a brief introduction and a little about the questions you would like to explore.