Can I say that a trained neural network model with less parameters requires less resources during real world inference?

1

1

Let us imagine that we have two trained neural network models with different architectures (e.g., type of layers). The first model (a) uses 1D convolutional layers with fully-connected layers and has 10 million learnable prameters. The second model (b) does use 2d conv layer with and has only 1 million paramerts in total. Both model achieve equal scores on the same input data set.

Can I say that model b with less parameter is more favourable because it has less (trainable) prameters and for this reason it requires less resources (GPU-Power, Memory ...) if it would used in a real world scenario? Or does this only show that I is able to learn more effectively in constrast to model a? Are the required resource during real world inference also influenced by the layer types? How can I measure the resource utilization during inference

Just for information: I use tensorflow / keras for both models.

Thanks for your opinon.

user3352632

Posted 2021-01-16T16:29:11.380

Reputation: 430

3well memory and CPU are certainly a function of num of paremeters – Nikos M. – 2021-01-16T17:31:31.887

No answers