Freeze'nSense: Estimation of performance isolation in cloud environments

Modern computing hardware has a very good task parallelism, but resource contention between tasks remains high. This renders large fractions of CPU time wasted and leads to application interference. Even tasks running on dedicated CPU cores can still incur interference from other tasks, most notably because of the caches and other hardware components shared by more than one core. The level of interference depends on the nature of executed tasks and is difficult to predict. A customer who has been granted that his task will run as if it were alone (e.g., a CPU core dedicated to a virtual machine), indeed suffers from significant performance degradation due to the time spent waiting for resources occupied by other tasks. Measuring actual performance of a task or a virtual machine can be difficult. However, even more challenging is estimating what the performance of the task should be if it were running completely in isolation. In this paper, we present a measurement technique Freeze'nSense. It is based on the hardware performance counters and allows measuring actual performance of a task and estimating performance as if the task were in isolation, all during runtime. To estimate performance in isolation, the proposed technique performs a short-time freezing of the potentially interfering tasks. Freeze'nSense introduces lower than 1% overhead and is confirmed to provide accurate and reliable measurements. In practice, Freeze'nSense becomes a valuable tool helping to automatically identify tasks that suffer the most in a shared environment and move them to a distant core. The observed performance improvement can be as large as 80–100% for individual tasks, and scale up to 15–20% for the computing node.