Week 10: Countering Runtime Variance

As we were making progress with our implementation of hardware acceleration of the Additive Number Theoretic Transform (ANTT) for the BINIUS proof system, we faced a very common issue in GPU programming – execution time variance. Despite achieving excellent peak performance with our CUDA implementations, we had trouble achieving consistent execution times.

Variance issues are known to occur in high-speed cryptographic systems frequently. In GPU architectures, the source of variance may lie in minor differences in memory access patterns or even in warp divergence leading to diverging thread blocks.

This week I was actively profiling our kernels to find potential hardware bottlenecks. Through careful optimization of our memory access patterns and using the L1 and L2 memory efficiently, we managed to achieve more consistent execution times. This is a very thorough process involving a lot of tweaking parameters and fine tuning. But now, our performance graphs look amazing.