The growing accessibility of artificial intelligence has led to widespread experimentation with deep learning models. As neural networks become increasingly common, efforts to optimize the training process have intensified, focusing on architectural improvements, better optimization algorithms, and learning rate scheduling techniques. Batch size schedulers have recently gained attention for their potential to improve convergence speed and hardware efficiency, analogous to the benefits provided by learning rate schedulers.

Increasing the batch size during training enables efficient hardware utilization. However, larger batch sizes are often associated with reduced generalization performance. To address this issue, adaptive strategies have been proposed that incrementally adjust batch size during training. These approaches aim to maintain model performance while benefiting from faster convergence and improved compute efficiency.

In this work, we analyze the performance implications of using batch size schedulers, both in terms of training time and generalization ability. We compare six different batch size schedulers against equivalent learning rate adaptation policies on image classification benchmarks. Our results show that batch size schedulers can reduce training time by up to 38% with a generalization drop of less than 0.7%.

Finally, we provide practical recommendations for integrating batch size schedulers into modern training pipelines and replacing learning rate schedulers.