John Duchi's Webpage
John C Duchi
A little about me:
I am currently a PhD candidate in computer
science at Berkeley,
where I started in the fall of 2008. I work in the Statistical Artificial
Intelligence Lab (SAIL) under the joint supervision of Mike Jordan and
Martin
Wainwright. I obtained my master's degree (MA) in
statistics in Fall 2012.
I was initially supported by an NDSEG fellowship, and currently
I am supported by Facebook, who have generously
awarded me a Facebook
Fellowship. Before this, I was an undergrad and a masters
student at Stanford University working with Daphne Koller in her
research group, DAGS. I
was also a Resident Assistant in Cedro, an
all-freshman dorm at Stanford. I also spend some time at Google Research (once
upon a time I was also a software engineer there), where I had
(and continue to have) the great fortune to work with Yoram
Singer. Last, but certainly not least, I got married to my
wonderful wife Emily
in the summer of 2008.
Curriculum Vitae: [pdf]
Contact info: [Visit]
Recipe Book: [Draft]
Publications
Clicking the publication title will give an abstract and publication
information.
Journal Articles
The Generalization Ability of Online Algorithms for Dependent Data,
Alekh Agarwal
and John C. Duchi. IEEE Transactions on Information Theory (2013).
[pdf]
Ergodic Mirror Descent,
John C. Duchi, Alekh
Agarwal, Mikael
Johansson, Michael
I. Jordan. SIAM Journal on Optimization (SIOPT), 2012.
[pdf]
Randomized Smoothing for Stochastic Optimization,
John Duchi, Peter
L. Bartlett, and
Martin Wainwright.
SIAM Journal on Optimization (SIOPT), 2012.
[pdf]
Dual Averaging for Distributed Optimization: Convergence and Network
Scaling,
John Duchi, Alekh
Agarwal, and
Martin Wainwright. IEEE Transactions on Automatic Control
(March 2012).
[pdf]
Adaptive Subgradient Methods for Online Learning and Stochastic
Optimization,
John Duchi, Elad Hazan,
and
Yoram Singer. Journal of Machine Learning Research
(JMLR 2011).
[pdf]
Efficient
Online and Batch Learning using Forward Backward Splitting,
John Duchi and
Yoram Singer. Journal of Machine Learning Research (JMLR 2009)
and Neural Information Processing Systems (NIPS 2009).
[pdf]
Submitted/In Preparation
Local Privacy and Minimax Bounds: Sharp Rates for Probability
Estimation,
John C. Duchi,
Michael I. Jordan,
and Martin
J. Wainwright.
[pdf]
Divide and Conquer Kernel Ridge Regression:
A Distributed Algorithm with Minimax Optimal Rates,
Yuchen Zhang,
John C. Duchi, and
Martin J.
Wainwright. [pdf]
Local Privacy and Statistical Minimax Rates,
John C. Duchi,
Michael I. Jordan,
and Martin
J. Wainwright.
[pdf,
slides]
Privacy Aware Learning,
John C. Duchi,
Michael I. Jordan, and
Martin
J. Wainwright.
A short version of this appeared in Neural Information Processing
Systems (NIPS 2012).
[pdf]
Communication-Efficient Algorithms for Statistical Optimization,
Yuchen Zhang,
John C. Duchi, and
Martin Wainwright. A short version
of this appeared in Neural Information Processing
Systems (NIPS 2012).
[pdf]
Oracle Inequalities for Computationally Adaptive Model Selection,
Alekh Agarwal,
Peter L. Bartlett,
and John C. Duchi.
[pdf]
The Asymptotics of Ranking Algorithms,
John C. Duchi,
Lester Mackey,
Michael I. Jordan.
[pdf]
Conference Proceedings
Divide and Conquer Kernel Ridge Regression,
Yuchen Zhang,
John C. Duchi, and
Martin Wainwright. Conference on Learning
Theory (COLT 2013).
[pdf]
Privacy Aware Learning, John C. Duchi,
Michael I. Jordan,
and
Martin Wainwright. Neural Information Processing
Systems (NIPS 2012).
[pdf]
Communication-Efficient Algorithms for Statistical Optimization,
Yuchen Zhang,
John C. Duchi, and
Martin Wainwright. Neural Information Processing
Systems (NIPS 2012).
[pdf]
Finite Sample Convergence Rates of Zero-Order Stochastic Optimization
Methods, John C. Duchi,
Michael I. Jordan,
Martin Wainwright, and
Andre Wibisono.
Neural Information Processing
Systems (NIPS 2012).
[pdf]
Randomized Smoothing for (Parallel) Stochastic Optimization,
John Duchi, Peter
L. Bartlett, and
Martin Wainwright.
International Conference on Machine Learning (ICML 2012) .
Presented but not included in proceedings.
[pdf]
Distributed Delayed Stochastic Optimization,
Alekh
Agarwal and John Duchi. Neural Information Processing Systems
(NIPS 2011).
[pdf]
[Long pdf]
Ergodic Subgradient Descent, John Duchi
Alekh
Agarwal, Mikael
Johansson, Michael
I. Jordan. Allerton Conference on Communications, Control,
and Computing 2011.
[pdf]
Oracle Inequalities for Computationally Budgeted Model Selection,
Alekh Agarwal,
John Duchi, Peter
L. Bartlett, Clement Levrard. Conference on Learning Theory
(COLT 2011). [pdf]
Distributed Dual Averaging in Networks,
John Duchi, Alekh
Agarwal, and
Martin Wainwright. Neural Information Processing Systems
(NIPS 2010).
[pdf]
On the Consistency of Ranking Algorithms,
John Duchi, Lester
Mackey, and Michael
Jordan. International Conference on Machine Learning
(ICML 2010). [pdf]
Winner of best student paper award.
Adaptive Subgradient Methods for Online Learning and Stochastic
Optimization,
John Duchi, Elad Hazan,
and
Yoram Singer. Conference on Learning
Theory (COLT 2010).
[pdf]
Composite Objective Mirror Descent,
John Duchi,
Shai Shalev-Shwartz,
Yoram Singer,
Ambuj Tewari.
Conference on Learning Theory (COLT 2010).
[pdf]
Efficient Learning using Forward Backward Splitting,
John Duchi and
Yoram Singer.
Neural Information Processing Systems (NIPS 2009).
[pdf]
Boosting with Structural Sparsity,
John Duchi and
Yoram Singer. International Conference on Machine Learning
(ICML 2009).
[pdf]
[Long pdf]
Efficient Projections onto the L1-Ball for Learning in High
Dimensions, John Duchi,
Shai Shalev-Shwartz,
Yoram
Singer, and
Tushar
Chandra.
International Conference on Machine Learning (ICML 2008).
[pdf]
Constrained Approximate Maximum Entropy Learning of Markov
Random Fields, Varun Ganapathi, David Vickrey,
John Duchi, and
Daphne Koller.
Conference on Uncertainty in Artificial Intelligence (UAI 2008).
[pdf]
Projected Subgradient Methods for Learning Sparse Gaussians,
John Duchi,
Stephen Gould and
Daphne Koller.
Conference on Uncertainty in Artificial Intelligence (UAI 2008).
[pdf]
Using Combinatorial Optimization within Max-Product Belief
Propagation,
John Duchi,
Danny Tarlow,
Gal Elidan, and
Daphne Koller.
Neural Information Processing Systems (NIPS 2006).
[pdf]
Invited Talks
Composite Objective Optimization and Learning for Large Datasets,
Workshop on Massive Modern
Datasets, Stanford, CA, June 2010.
[slides pdf]
Classes I have TAed
Berkeley EE127a, Introduction to Optimization, Spring 2009,
taught by
Laurent El Ghaoui (Berkeley).
Stanford CS227, Reasoning
Methods in Artificial Intelligence, Spring 2006, Spring
2007, taught by Pandurang Nayak (Stanford).
Stanford CS228,
Probabilistic Models in Artificial Intelligence, Winter
2007, taught by Daphne
Koller.
A Few Class Papers, Potentially Useful Notes, and Scribing
Note that none of these are guaranteed in any way to be correct, so
forgive me if they are not. I just sometimes like to derive things
that I may find useful later.
Notes on concentration bounds and probability inequalities,
for fun.
[pdf]
Derivations for Linear Algebra and Optimization, for fun.
[pdf]
Notes on some matrix properties, for fun.
[pdf]
