John Duchi's Webpage

John C Duchi

John Duchi

A little about me: (Just so you know) I am currently a PhD candidate in computer science at Berkeley, where I started in the fall of 2008. I work in the Statistical Artificial Intelligence Lab (SAIL) under the joint supervision of Mike Jordan and Martin Wainwright. I am currently supported by an NDSEG fellowship. Starting next year, I will be supported by Facebook, who have generously awarded me a Facebook Fellowship. Before this, I was an undergrad and a masters student at Stanford University working with Daphne Koller in her research group, DAGS. I was also a Resident Assistant in Cedro, an all-freshman dorm at Stanford. My little brother, Andrew Duchi, is a master's student at Stanford. I also spend some time at Google Research (once upon a time I was also a software engineer there), where I had (and continue to have) the great fortune to work with Yoram Singer. Last, but certainly not least, I got married to my wonderful wife Emily in the summer of 2008.

Workshop: I helped organize a workshop at NIPS 2010, Learning on Cores, Clusters, and Clouds.

Contact info: [Visit]

Recipe Book: [Draft]

Publications

Clicking the publication title will give an abstract and publication information.

Journal Articles

Dual Averaging for Distributed Optimization: Convergence and Network Scaling, John Duchi, Alekh Agarwal, and Martin Wainwright. IEEE Transactions on Automatic Control, to appear (January 2012). [pdf]

Adaptive Subgradient Methods for Online Learning and Stochastic Optimization, John Duchi, Elad Hazan, and Yoram Singer. Journal of Machine Learning Research, (JMLR 2011). [pdf]

Efficient Online and Batch Learning using Forward Backward Splitting, John Duchi and Yoram Singer. Journal of Machine Learning Research (JMLR 2009) and Neural Information Processing Systems (NIPS 2009). [pdf]

Submitted/In Preparation

The Generalization Ability of Online Algorithms for Dependent Data, Alekh Agarwal and John C. Duchi. [pdf]

Ergodic Subgradient Descent, John C. Duchi, Alekh Agarwal, Mikael Johansson, Michael I. Jordan. [pdf]

Distributed Delayed Stochastic Optimization, Alekh Agarwal and John Duchi. [pdf]

Randomized Smothing for Stochastic Optimization, John Duchi, Peter L. Bartlett, and Martin Wainwright. [pdf]

Conference Proceedings

Distributed Delayed Stochastic Optimization, Alekh Agarwal and John Duchi. Neural Information Processing Systems (NIPS 2011). [pdf]

Ergodic Subgradient Descent, John Duchi Alekh Agarwal, Mikael Johansson, Michael I. Jordan. Allerton Conference on Communications, Control, and Computing 2011. [pdf]

Oracle Inequalities for Computationally Budgeted Model Selection, Alekh Agarwal, John Duchi, Peter L. Bartlett, Clement Levrard. Conference on Learning Theory (COLT 2011). [pdf]

Distributed Dual Averaging in Networks, John Duchi, Alekh Agarwal, and Martin Wainwright. Neural Information Processing Systems (NIPS 2010). [pdf]

On the Consistency of Ranking Algorithms, John Duchi, Lester Mackey, and Michael Jordan. International Conference on Machine Learning (ICML 2010). [pdf] Winner of best student paper award.

Adaptive Subgradient Methods for Online Learning and Stochastic Optimization, John Duchi, Elad Hazan, and Yoram Singer. Conference on Learning Theory (COLT 2010). [pdf]

Composite Objective Mirror Descent, John Duchi, Shai Shalev-Shwartz, Yoram Singer, Ambuj Tewari. Conference on Learning Theory (COLT 2010). [pdf]

Efficient Learning using Forward Backward Splitting, John Duchi and Yoram Singer. Neural Information Processing Systems (NIPS 2009). [pdf]

Boosting with Structural Sparsity, John Duchi and Yoram Singer. International Conference on Machine Learning (ICML 2009). [pdf] [Long pdf]

Efficient Projections onto the L1-Ball for Learning in High Dimensions, John Duchi, Shai Shalev-Shwartz, Yoram Singer, and Tushar Chandra. International Conference on Machine Learning (ICML 2008). [pdf]

Constrained Approximate Maximum Entropy Learning of Markov Random Fields, Varun Ganapathi, David Vickrey, John Duchi, and Daphne Koller. Conference on Uncertainty in Artificial Intelligence (UAI 2008). [pdf]

Projected Subgradient Methods for Learning Sparse Gaussians, John Duchi, Stephen Gould and Daphne Koller. Conference on Uncertainty in Artificial Intelligence (UAI 2008). [pdf]

Using Combinatorial Optimization within Max-Product Belief Propagation, John Duchi, Danny Tarlow, Gal Elidan, and Daphne Koller. Neural Information Processing Systems (NIPS 2006). [pdf]

Invited Talks

Composite Objective Optimization and Learning for Large Datasets, Workshop on Massive Modern Datasets, Stanford, CA, June 2010. [slides pdf]

Classes I have TAed

Berkeley EE127a, Introduction to Optimization, Spring 2009, taught by Laurent El Ghaoui (Berkeley).

Stanford CS227, Reasoning Methods in Artificial Intelligence, Spring 2006, Spring 2007, taught by Pandurang Nayak (Stanford).

Stanford CS228, Probabilistic Models in Artificial Intelligence, Winter 2007, taught by Daphne Koller.

A Few Class Papers, Potentially Useful Notes, and Scribing

Note that none of these are guaranteed in any way to be correct, so forgive me if they are not. I just sometimes like to derive things that I may find useful later.

Notes on concentration bounds and probability inequalities, for fun. [pdf]

Derivations for Linear Algebra and Optimization, for fun. [pdf]

Notes on some matrix properties, for fun. [pdf]