John Duchi's Webpage

John C Duchi

A little about me: I am currently a PhD candidate in computer science at Berkeley, where I started in the fall of 2008. I work in the Statistical Artificial Intelligence Lab (SAIL) under the joint supervision of Mike Jordan and Martin Wainwright. I obtained my master's degree (MA) in statistics in Fall 2012. I was initially supported by an NDSEG fellowship, and currently I am supported by Facebook, who have generously awarded me a Facebook Fellowship. Before this, I was an undergrad and a masters student at Stanford University working with Daphne Koller in her research group, DAGS. I was also a Resident Assistant in Cedro, an all-freshman dorm at Stanford. I also spend some time at Google Research (once upon a time I was also a software engineer there), where I had (and continue to have) the great fortune to work with Yoram Singer. Last, but certainly not least, I got married to my wonderful wife Emily in the summer of 2008.

Curriculum Vitae: [pdf]

Contact info: [Visit]

Recipe Book: [Draft]

Publications

Clicking the publication title will give an abstract and publication information.

Journal Articles

The Generalization Ability of Online Algorithms for Dependent Data, Alekh Agarwal and John C. Duchi. IEEE Transactions on Information Theory (2013). [pdf]

Ergodic Mirror Descent, John C. Duchi, Alekh Agarwal, Mikael Johansson, Michael I. Jordan. SIAM Journal on Optimization (SIOPT), 2012. [pdf]

Randomized Smoothing for Stochastic Optimization, John Duchi, Peter L. Bartlett, and Martin Wainwright. SIAM Journal on Optimization (SIOPT), 2012. [pdf]

Dual Averaging for Distributed Optimization: Convergence and Network Scaling, John Duchi, Alekh Agarwal, and Martin Wainwright. IEEE Transactions on Automatic Control (March 2012). [pdf]

Adaptive Subgradient Methods for Online Learning and Stochastic Optimization, John Duchi, Elad Hazan, and Yoram Singer. Journal of Machine Learning Research (JMLR 2011). [pdf]

Efficient Online and Batch Learning using Forward Backward Splitting, John Duchi and Yoram Singer. Journal of Machine Learning Research (JMLR 2009) and Neural Information Processing Systems (NIPS 2009). [pdf]

Submitted/In Preparation

Local Privacy and Statistical Minimax Rates, John C. Duchi, Martin J. Wainwright, and Michael I. Jordan. [pdf, slides]

Privacy Aware Learning, John C. Duchi, Michael I. Jordan, and Martin J. Wainwright. A short version of this appeared in Neural Information Processing Systems (NIPS 2012). [pdf]

Communication-Efficient Algorithms for Statistical Optimization, Yuchen Zhang, John C. Duchi, and Martin Wainwright. A short version of this appeared in Neural Information Processing Systems (NIPS 2012). [pdf]

Oracle Inequalities for Computationally Adaptive Model Selection, Alekh Agarwal, Peter L. Bartlett, and John C. Duchi. [pdf]

The Asymptotics of Ranking Algorithms, John C. Duchi, Lester Mackey, Michael I. Jordan. [pdf]

Distributed Delayed Stochastic Optimization, Alekh Agarwal and John Duchi. [pdf]

Conference Proceedings

Divide and Conquer Kernel Ridge Regression, Yuchen Zhang, John C. Duchi, and Martin Wainwright. Conference on Learning Theory (COLT 2013). To appear.

Privacy Aware Learning, John C. Duchi, Michael I. Jordan, and Martin Wainwright. Neural Information Processing Systems (NIPS 2012). [pdf]

Communication-Efficient Algorithms for Statistical Optimization, Yuchen Zhang, John C. Duchi, and Martin Wainwright. Neural Information Processing Systems (NIPS 2012). [pdf]

Finite Sample Convergence Rates of Zero-Order Stochastic Optimization Methods, John C. Duchi, Michael I. Jordan, Martin Wainwright, and Andre Wibisono. Neural Information Processing Systems (NIPS 2012). [pdf]

Randomized Smoothing for (Parallel) Stochastic Optimization, John Duchi, Peter L. Bartlett, and Martin Wainwright. International Conference on Machine Learning (ICML 2012) . Presented but not included in proceedings. [pdf]

Distributed Delayed Stochastic Optimization, Alekh Agarwal and John Duchi. Neural Information Processing Systems (NIPS 2011). [pdf]

Ergodic Subgradient Descent, John Duchi Alekh Agarwal, Mikael Johansson, Michael I. Jordan. Allerton Conference on Communications, Control, and Computing 2011. [pdf]

Oracle Inequalities for Computationally Budgeted Model Selection, Alekh Agarwal, John Duchi, Peter L. Bartlett, Clement Levrard. Conference on Learning Theory (COLT 2011). [pdf]

Distributed Dual Averaging in Networks, John Duchi, Alekh Agarwal, and Martin Wainwright. Neural Information Processing Systems (NIPS 2010). [pdf]

On the Consistency of Ranking Algorithms, John Duchi, Lester Mackey, and Michael Jordan. International Conference on Machine Learning (ICML 2010). [pdf] Winner of best student paper award.

Adaptive Subgradient Methods for Online Learning and Stochastic Optimization, John Duchi, Elad Hazan, and Yoram Singer. Conference on Learning Theory (COLT 2010). [pdf]

Composite Objective Mirror Descent, John Duchi, Shai Shalev-Shwartz, Yoram Singer, Ambuj Tewari. Conference on Learning Theory (COLT 2010). [pdf]

Efficient Learning using Forward Backward Splitting, John Duchi and Yoram Singer. Neural Information Processing Systems (NIPS 2009). [pdf]

Boosting with Structural Sparsity, John Duchi and Yoram Singer. International Conference on Machine Learning (ICML 2009). [pdf] [Long pdf]

Efficient Projections onto the L1-Ball for Learning in High Dimensions, John Duchi, Shai Shalev-Shwartz, Yoram Singer, and Tushar Chandra. International Conference on Machine Learning (ICML 2008). [pdf]

Constrained Approximate Maximum Entropy Learning of Markov Random Fields, Varun Ganapathi, David Vickrey, John Duchi, and Daphne Koller. Conference on Uncertainty in Artificial Intelligence (UAI 2008). [pdf]

Projected Subgradient Methods for Learning Sparse Gaussians, John Duchi, Stephen Gould and Daphne Koller. Conference on Uncertainty in Artificial Intelligence (UAI 2008). [pdf]

Using Combinatorial Optimization within Max-Product Belief Propagation, John Duchi, Danny Tarlow, Gal Elidan, and Daphne Koller. Neural Information Processing Systems (NIPS 2006). [pdf]

Invited Talks

Composite Objective Optimization and Learning for Large Datasets, Workshop on Massive Modern Datasets, Stanford, CA, June 2010. [slides pdf]

Classes I have TAed

Berkeley EE127a, Introduction to Optimization, Spring 2009, taught by Laurent El Ghaoui (Berkeley).

Stanford CS227, Reasoning Methods in Artificial Intelligence, Spring 2006, Spring 2007, taught by Pandurang Nayak (Stanford).

Stanford CS228, Probabilistic Models in Artificial Intelligence, Winter 2007, taught by Daphne Koller.

A Few Class Papers, Potentially Useful Notes, and Scribing

Note that none of these are guaranteed in any way to be correct, so forgive me if they are not. I just sometimes like to derive things that I may find useful later.

Notes on concentration bounds and probability inequalities, for fun. [pdf]

Derivations for Linear Algebra and Optimization, for fun. [pdf]

Notes on some matrix properties, for fun. [pdf]