With the increasing complexity of AI applications, compilers and runtime systems are the drivers that customize workloads to their specific needs. In contrast to classical work, AI entails the trade-off of compute, memory, and accuracy amongst CPUs, GPUs, and accelerators. Combining automated profiling and manual tuning brings efficiency to the profiling process and makes it scalable and sustainable. In this growing arena, practitioners like Rajalakshmi Srinivasaraghavan have made distinct contributions that highlight the significance of this hands-on, hybrid approach.

The focus of the work of Rajalakshmi is to examine the different stages of an AI workload with the aim of discovering the computational bottlenecks and finding options that are unique to the available resources. Through observing the trends in throughput, latency and utilization of memory, she was able to come up with strategies that best suited workloads to the hardware. The result was a hybrid performance model, with various tasks dynamically path-based to CPUs, GPUs or accelerators, depending on the specifics of each. Not only did this method minimize inefficiencies, but it also made it more scalable in a vast range of inference problems. “AI optimization is less about raw power and more about aligning every task with its most natural execution path,” she added.