virtualization benchmark suites

[Anthony Liguori]

On 09/22/2011 04:47 PM, Michael D Day wrote:
> project-planning-bounces at ovirt.org wrote on 09/22/2011 05:34:38 PM:
>>
>> Does such benchmark ever existed (apart from kernel build)? :)
>> As you pointed Anthony, let the ovirt project first focus on simplify
>> the build process before we develop a competing benchmark suite to the
>> one that we base lots of our marketing over.
>>
>
> Yes, there are several distributed benchmarks that test different aspects of
> virtualization performance. SPECVirt tests server consolidation. It doesn't
> test memory over-commitment. In fact, I don't think anyone who publishes a
> SPECVirt score using a configuration that over-commits memory because it hurts
> the score. There are other benchmark suites that test HPC cluster-style
> workloads running on a hypervisor. And networking performance. These are just
> some of the ones we use within IBM. They are very useful. If we would have had
> better benchmark suites to test memory usage you would have decided to include
> the balloon driver in RHEL 5.4 instead of relying solely upon KSM. And if we
> had used a good distributed transactional benchmark like DayTrader we would
> have known early on that small-packet network performance is relatively poor.
> Just a couple of examples.
>
> If a benchmarking project does happen, it would need to be a self-sustaining
> project under the oVirt community umbrella. That means there would need to be
> interest in the form of independent developer resources.

DayTrader is a web application benchmark.  You provide it a profile and it will 
simulate a bunch of clients trying to connect to your web application.

It's geared toward J2EE developers as it provides to do performance testing of 
your J2EE application.

I think the way to think of this in the context of oVirt, is a suite of tools 
that can be used by an ISV/Customer to validate that they're deployment of oVirt 
is optimized appropriately.

I definitely think its worth having a perf BoF as part of the workshop (or 
perhaps the next workshop) to start discussing how we handle performance 
measurement/tuning in an oVirt world.

I agree though that we need to tackle things one at a time but it doesn't hurt 
to have an idea of where we want to be in the long term while we do that :-)

Regards,

Anthony Liguori

> And keep in mind I'm passing on customer feedback. (I happen to think its a
> good suggestion as well).
>
> Thanks,
>
> Mike
>
> Mike Day
> IBM Distinguished Engineer
> Chief Virtualization Architect, Open Systems Development
> Cell: +1 919 371-8786 | mdday at us.ibm.com
> http://code.ncultra.org
>