Learning multiple online tasks with a global objective
Ofer Dekel
The Hebrew University of Jerusalem
Abstract
The simplicity and elegance of online learning make it a practical tool with
many useful applications. In practice, we are often faced with multiple online
prediction tasks in parallel. The naive approach to dealing with such
situations is to learn each task separately and independently of the others.
Can we do any better than this?
We present an online multitask learning framework where the multiple tasks
all contribute to a common goal and share the consequences of their
prediction mistakes. We present several new learning algorithms that take
advantage of this framework and learn the multiple tasks jointly. We prove
cumulative loss bounds that provide a guarantee on the worst-case performance
of our algorithms and elucidate the advantages of our approach over the
naive approach of learning each task separately.
Many real-world online prediction applications naturally fit within our
framework. We present several concrete examples, including a
multiclass-multilabel text categorization algorithm capable of processing
millions of examples in minutes, a surprising application of our technique to
online ordinal regression, and a randomized algorithm that dynamically allocates
memory to multiple kernel-based classifiers.
Joint work with Yoram Singer and Phil Long
Bio:
Ofer Dekel is a Ph.D student at The Hebrew University of Jerusalem, School of Computer Science and Engineering.
Maintained by:
Fei Sha