Profiling Neural Network Partitioning Strategies for Inference across the Computing Continuum

Abstract

The increasing adoption of neural networks in edge applications has created a need for efficient inference across heterogeneous computing continuum environments. Partitioning a neural network across multiple computing nodes can improve resource utilization and enable inference on devices with limited computational capabilities, but the choice of partitioning strategy can significantly affect performance. This paper presents a profiling-based approach for evaluating neural network partitioning strategies across the computing continuum. We investigate the performance implications of different partitioning configurations by considering the computational and communication characteristics of the participating nodes. Through experimental evaluation, we analyze inference performance across heterogeneous computing environments and provide insights into the trade-offs associated with different neural network partitioning strategies.