Sandra Benchmarks Clarification: compilers, optimisations

In light of the recent allegations regarding the Intel* compiler and its use in benchmarks, I am writing to you to clarify the benchmarks in Sandra you might be using in your articles:

- Sandra 2010 uses ONLY the Visual Studio 2008/SP1 compiler (15.xx) and MASM/YASM for the assembler code. We use blended optimisation optimised for speed (O2).

- Sandra 2009 used ONLY the Visual Studio 2005/SP1 (13.xx), Vista SDK (14.xx) compilers and MASM/YASM for the assembler code. We use blended optimisation optimised for speed (O2).

- We do not test for the manufacturer when choosing code paths; we do, however, optimise based on CPU model where appropriate (e.g. threads, buffer sizes, etc.).

- In general, there is only 1 code path for each technology (e.g. SSE2, SSE3, etc.), except where other characteristics yield serious sub-optimal performance: for example the multi-core benchmark has a path for inclusive L2/L3 cache (e.g. Intel) and exclusive L2/L3 cache (e.g. AMD). Such exceptions are detailed in the Help file.

- Sandra is not open-source, sadly we cannot do it, however we do provide the source code under mutual NDA to all major manufacturers working with us (AMD/ATI, Intel, VIA/Centaur, nVidia) and - in the spirit of full disclosure - we list the names of engineers who have worked on optimising the tests, press F1 for the Help file in any benchmark.

We would kindly ask - if you can - to urge manufacturers to provide us with datasheets (under NDA) for their hardware, you would probably not believe how hard it is to receive the information required.

- We do NOT charge any manufacturer for working with us, nor do we receive any financial, advertising or otherwise incentive from anyone (unlike some of our competitors). Thus we have nothing to gain whoever "wins", nor do we care.

We actively try to engage with manufacturers to find out whether our way of doing things makes sense and ask them to test in their labs to ensure the benchmarks work and make sense on all the hardware they have; naturally we cannot afford nor can we test each component on the market.

The Lite version contains ALL the benchmarks and has always been free, thus you readers do not need to pay anything to benchmark their systems.

- Except where we cannot, i.e. before a major new arch release - we buy ALL our hardware. We always ask whether the hardware should be returned and have returned at our cost such hardware. Any hardware not returned is actively used for regression testing and returned if asked.

I am puting together a public KB article, so if you have further questions please ask - and I will update both the article and Help file:

support.sisoftware.co.uk/knowledgebase.php?article=24