Injecting Realistic Burstiness to a Traditional Client-Server Benchmark

Mi, Ningfang ; Casale, Giuliano ; Cherkasova, Ludmila ; Smirni, Evgenia

HP Development Company

关键词: Client-server benchmarks;    burstiness;    index of dispersion;    performance evaluation of self-managed systems.;   

RP-ID  :  HPL-2009-114

学科分类：计算机科学（综合）

美国|英语

来源: HP Labs

【 摘 要 】

The design of autonomic systems often relies on representative benchmarks for evaluating system performance and scalability. Despite the fact that experimental observations have established that burstiness is a common workload characteristic that has deleterious effects on user-perceived performance, existing client-server benchmarks do not provide mechanisms for injecting burstiness into the workload. In this paper, we introduce a new methodology for generating workloads that emulate the temporal surge phenomenon in a controllable way, thus provide a mechanism that enables testing and evaluation of client-server system performance under reproducible bursty workloads. This new methodology allows to inject different amounts of burstiness into the arrival stream using the index of dispersion, a single parameter that is as simple to use as a turnable knob. We exemplify the effectiveness of this new methodology by introducing a new module into the TPC-W, a benchmark that is routinely used for capacity planning of e-commerce systems. This new module injects burstiness into the arrival process of clients in a controllable manner, and hence, enables understanding system performance degradation due to burstiness. Detailed experimentation on a real system shows that this benchmark modification can stress the system under different degrees of burstiness, making a strong case for the usefulness of this modification for capacity planning of autonomic systems.