IBM and MLCommons show how ubiquitous machine learning has become

This week IBM announced its latest Z series mainframe and MLCommons released its latest benchmark series. The two announcements had something in common – the acceleration of machine learning (ML) – which is catching on in everything from financial fraud detection in mainframe computers to wake word detection in home appliances.

While these two announcements were not directly related, they are part of a trend that shows how ubiquitous ML has become.

MLCommons brings standards to ML benchmarking

ML benchmarking is important because we often hear about ML performance in terms of TOPS – trillions of operations per second. Like MIPS (“millions of instructions per second” or “meaningless indication of processor speed”, depending on your perspective), TOPS is a theoretical number calculated from the architecture and not a measured rating based on running workloads. Therefore, TOPS can be a deceptive number as it does not take into account the impact of the software stack. Software is the most critical aspect of implementing ML and efficiency varies widely, which Nvidia has clearly demonstrated by improving the performance of its A100 platform by 50% in MLCommons benchmarks over the years.

The industry organization MLCommons was founded by a consortium of companies to create a standardized set of benchmarks along with a standardized testing methodology that allows different machine learning systems to be compared. MLCommons MLPerf benchmark suites include various benchmarks covering many popular ML workloads and scenarios. The MLPerf benchmarks address everything from the tiny microcontrollers used in consumer and IoT devices, to mobile devices like smartphones and PCs, to edge servers and data center-class server configurations. MLCommons supporters include Amazon, Arm, Baidu, Dell Technologies, Facebook, Google, Harvard, Intel, Lenovo, Microsoft, Nvidia, Stanford, and the University of Toronto.

MLCommons releases benchmark results in batches and has different release schedules for inference and training. The last announcement concerned version 2.0 of MLPerf Inference Suite for data center and edge servers, version 2.0 for MLPerf Mobile and version 0.7 for MLPerf Tiny for IoT devices.

To date, Nvidia has been the company that has had the most consistent set of submissions, scoring results on every iteration, in every benchmark test, and from multiple partners. Nvidia and its partners appear to have invested enormous resources in running and publishing all relevant MLCommons benchmarks. No other provider can keep up with this claim. Recent inference benchmark submissions include Nvidia Jetson Orin SoCs for edge servers and the Ampere-based A100 GPUs for data centers. Announced at GTC in Spring 2022, Nvidia’s data center GPU “Hopper” H100 came too late to be included in the latest MLCommons announcement, but we fully await Nvidia H100’s results in the next round to see.

Recently, Qualcomm and its partners released more data center MLPerf benchmarks for the company’s Cloud AI 100 platform and more mobile MLPerf benchmarks for Snapdragon processors. Qualcomm’s latest silicon has proven to be very power efficient in data center ML testing, which can give it an edge in power-constrained edge server applications.

Many of the submitters are system vendors using processors and accelerators from silicon vendors such as AMD, Andes, Ampere, Intel, Nvidia, Qualcomm and Samsung. But many of the AI ​​startups were missing. As one consulting firm, Krai, put it, “Potential submitters, particularly ML hardware startups, are understandably wary of dedicating valuable technical resources to optimizing industry benchmarks rather than actual customer workloads.” But then Krai countered theirs own objection with “MLPerf is the Olympics of ML optimization and benchmarking”. Still, many startups haven’t invested in creating MLCommons results for various reasons, and that’s disappointing. There are also not enough FPGA vendors participating in this round.

The MLPerf Tiny benchmark is designed for very low power applications such as B. for recognizing keywords, visual activation words, image classification and anomaly detection. In this case, we’re seeing results from a mix of small companies like Andes, Plumeria, and Syntiant, as well as established companies like Alibaba, Renesas, Silicon Labs, and STMicroeletronics.

IBM inserts AI acceleration into every transaction

Although IBM has not participated in MLCommons benchmarks, the company takes ML seriously. With its latest Z-series mainframe computer, the z16, IBM has added accelerators for ML inference and quantum-proof secure boot and cryptography. But mainframe systems have different customer requirements. With approximately 70% of banking transactions (by value basis) running on IBM mainframes, the company anticipates financial institutions’ needs for highly reliable transaction processing protection. Additionally, by adding ML acceleration to its CPU, IBM can offer per-transaction ML intelligence to detect fraudulent transactions.

MORE FROM FORBESIBM redesigns mainframe with new Telum processor

In an article I wrote in 2018, I said: “Indeed, the future hybrid cloud computing model will likely include classical computing, AI processing, and quantum computing. When it comes to understanding all three of these technologies, few companies can match IBM’s commitment and expertise.” And the latest developments in IBM’s quantum computing roadmap and the ML acceleration in the z16 show that IBM is on both fronts is leading.

MORE FROM FORBESDoes IBM have the quantum advantage?


Machine learning is important from tiny devices to mainframe computers. Accelerating this workload can be done on CPUs, GPUs, FPGAs, ASICs and even MCUs and is now a part of all future computing. These are two examples of how ML changes and improves over time.

Tirias Research follows and advises companies across the electronics ecosystem, from semiconductors to systems and sensors to the cloud. Tirias research team members have advised IBM, Nvidia, Qualcomm and other companies across the AI ​​ecosystem.

Leave a Reply

Your email address will not be published.