Daniel Roy

Research Director

Associate Professor, Department of Statistical Sciences, Faculty of Arts & Science, University of Toronto

Associate Professor, Department of Computer and Mathematical Sciences, University of Toronto Scarborough

Canada CIFAR Artificial Intelligence Chair

Roy’s research in deep learning spans theory and practice. His contributions range from his pioneering work on empirically grounded statistical theory for deep learning, to state-of-the-art algorithms for neural network compression and data-parallel training. His experimental work has shed light on various deep learning phenomena, including neural network training dynamics and linear mode connectivity, while his recent theoretical work introduces simple but accurate mathematical models for deep neural networks at initialization.

Beyond his contributions to deep learning, Roy has made significant advances to the mathematical and statistical underpinnings of AI. His dissertation on probabilistic programming languages and computable probability theory was recognized by an MIT Sprowls Award. Roy recently resolved several open problems in statistical decision theory posed over 70 years ago, by exploiting the properties of infinitesimal numbers to expand the set of allowable Bayesian priors. His latest work, focussing on robust and adaptive decision making, has been recognized by multiple oral presentations at leading conferences and best poster awards.

Research Interests

  • Foundations of Machine Learning
  • Deep Learning
  • Sequential Decision Making
  • Statistical Learning Theory
  • Probabilistic Programming
  • Bayesian-Frequentist Interface
  • PAC-Bayes

Publications

Data-dependent PAC-Bayes priors via differential privacy

G. K. Dziugaite, and D. M. Roy

Advances in Neural Information Processing Systems 31 2018

On the computability of conditional probability

N. L. Ackerman, C. E. Freer, and D. M. Roy

Journal of the ACM 2019 66(3)

Entropy-SGD optimizes the prior of a PAC-Bayes bound: Generalization properties of Entropy-SGD and data-dependent priors

G. K. Dziugaite, and D. M. Roy

Proceedings of the 35th International Conference on Machine Learning (ICML) 2018

An estimator for the tail-index of graphex processes

Z. Naulet, E. Sharma, V. Veitch, and D. M. Roy

2017

Sampling and estimation for (sparse) exchangeable graphs

V. Veitch, and D. M. Roy

Ann. Statist. 2019 47(6):3274--3299

On extended admissible procedures and their nonstandard Bayes risk

Haosui Duanmu and Daniel M Roy

2021

Towards a Unified Information-Theoretic Framework for Generalization

Mahdi Haghifam and Gintare Karolina Dziugaite and Shay Moran and Dan Roy

2021

The future is log-Gaussian: ResNets and their infinite-depth-and-width limit at initialization

Mufan Li and Mihai Nica and Dan Roy

2021

Minimax Optimal Quantile and Semi-Adversarial Regret via Root-Logarithmic Regularizers

Jeffrey Negrea and Blair Bilodeau and Nicolò Campolongo and Francesco Orabona and Dan Roy

2021

Methods and Analysis of The First Competition in Predicting Generalization of Deep Learning

Yiding Jiang and Parth Natekar and Manik Sharma and Sumukh K Aithal and Dhruva Kashyap and Natarajan Subramanyam and Carlos Lassance and Daniel M Roy and Gintare Karolina Dziugaite and Suriya Gunasekar and Isabelle Guyon and Pierre Foret and Scott Yak and Hossein Mobahi and Behnam Neyshabur and Samy Bengio

2021