Enhanced Server Performance with StagedServer
James Larus
larus@microsoft.com
Microsoft Research
Servers--for the web, databases, mail, files, or many other resources--are
commonly structured as a collection of parallel tasks, each of which runs
code to process a request. Threads, processes, or event handlers underlie
these systems' software architecture because these mechanisms provide the
control independence and dynamic scheduling to hide high latency operations
such as I/O and communication. Unfortunately, many servers run poorly on
modern processors. Considerable empirical evidence shows that these systems
run at a fraction of processors' potential performance. In part, poor
performance is attributable to this software architecture, which frequently
switches between unrelated pieces of code, thereby destroying the program
locality that is a prerequisite for effective caches, TLB, and branch
predictors.
This work propose a software solution to this problem. Cohort Scheduling is
a technique that increases code and data locality by consecutively executing
logically related operations across different server requests. To support
this technique, we developed a programming model, called Staged Computation,
that offers an abstraction to group related operations and mechanisms to
implement this scheduling policy. These techniques have been implemented in
a library called StagedServer. Measurements on realistic systems show that
Cohort Scheduling can improve both processor performance and server
throughput. (Joint work with Michael Parkes.)