Huawei gets caught cheating on benchmarks

Chinese smartphone and networking giant Huawei has been caught out fiddling benchmark scores for its devices, and following delisting by Underwriters' Laboratories (UL) has announced it will make its performance-boosting benchmark mode available to all under manual control.

The smartphone industry is a cut-throat business. Manufacturers are constantly looking for an edge, and while much of the industrial design follows Apple's lead internally there are all kinds of tweaks manufacturers make to have their devices stand out: Many-core processors, scads of RAM, 'holographic' displays, and ultra-high performance aimed at power users, gamers, and those who just like to boast that their device gets more benchmark numbers than anyone else's.

For Huawei, though, the desire to top the benchmark leaderboards has led to some underhanded activity: a secret 'performance mode,' which overclocks the phones' processor outside usual thermal and power envelopes when one of a selected number of benchmark applications is detected. Having been caught out, Huawei's initial reaction was a schoolyard argument: Everyone else does it, the company claimed, so we have to do it too - which, given that rival companies including OnePlus have been pulled up for the same trickery in the past, isn't completely unbelievable.

Those behind the benchmarks, though, disagreed. Underwriters' Laboratories (UL), the testing organisation which acquired Futuremark, responded by delisting Huawei's devices from its 3DMark benchmark tables - and that led to a rapid conflab between the two on the best way forward.

'Huawei respects consumers’ right to choose what to do with their devices. Therefore, Huawei will provide users with open access to "Performance Mode" in EMUI 9.0, so that the user can choose when to use the maximum power of their device,' a joint statement from the two organisations explains. 'Huawei and UL have also discussed current common benchmark testing methodologies in general. UL and Huawei would like to participate in an industry movement to develop benchmarking standards that best serve the needs of manufacturers, press, and consumers.'

What's most interesting in Huawei's response, though, is its attempt to pin the blame not on a deliberate decision to add benchmark applications to a hidden accelerated performance mode but instead its devices' built-in artificial intelligence capabilities - as though the phone itself, and not Huawei's engineers, had made the decision to cheat in the benchmarks. 'Huawei explained that its smartphones use an artificial intelligent resource scheduling mechanism,' the statement reads. 'Because different scenarios have different resource needs, the latest Huawei handsets leverage innovative technologies such as artificial intelligence to optimise resource allocation in a way so that the hardware can demonstrate its capabilities to the fullest extent, while fulfilling user demands across all scenarios.'

UL has indicated that affected Huawei devices will not be restored to the benchmark tables until the company has rolled out the manual performance toggle update to its users.