Tensorflow Lite model vs Tensorflow Model

0

I have explored edge computation for AI and I came across multiple library or framework which can help to convert model into lite format which is suitable for edge devices.

Tensorflow Lite will help us to convert Tensorflow Model into Tensorflow lite. Or Intel OpenVino will optimise the model for edge devices.

Questions

  1. If we have library to optimise the model for edge devices, after conversion will it cause any accuracy issue. performance will get increase but accuracy of that model will get decrease.
  2. If we have such technology to optimise model performance and wont get accuracy decrease then why people wont prefer tensorflow Lite model over tensorflow model.

Maheshwar Ligade

Posted 2019-12-18T07:15:41.730

Reputation: 766

Answers

0

I have explored the AI for edge devices. My findings for tflite model.

  1. TFLite is just the tool suite to convert TFModel into TFLite.
  2. TFLite optimizes the model for edge or embedded devices using quantization techniques.

Quantization dramatically reduces both the memory requirement and computational cost of using neural networks.

Answer in brief:

  1. When we optimize the model definitely they are faster to take inference but it will impact the accuracy. Quantization leads to bit accuracy loss in smaller networks.
  2. TFLite is increasing the model performance but we have to pay a cost in terms of accuracy that's why we have both TFLite for edge computation and TFModel as well.

Maheshwar Ligade

Posted 2019-12-18T07:15:41.730

Reputation: 766

2

This partly answer to question 1. There is no general rule concerning accuracy or size of the model. It depends on the training data and the processed data. The lightest is your model compared to the full accuracy model the less accurate it will be. I would run the lite model on test data and compare to the accuracy of the full model to get an exact measure of the difference.

Tensor flow has different options to save the "lite" model (optimized in size, latency, none and default).

The following mostly answer question 2.

Tensor flow lite is intented to provide the ability to use the model to on line predict only and load the model not to train the model.

On the other hand Tensor flow is used to build (train) the model off line.

If your edge platform support any of the binding language provided for TensorFlow (javascript, java/kotlin, C++, python) you can use Tensorflow for prediction. The accuracy or speed options you might have selected to create the model will not be affected whether you use Tensor Flow or Tensor Flow Lite. Typically Tensor flow lite can be used on mobile devices (Ios, Android). There are other supported target, see this link

pascal sautot

Posted 2019-12-18T07:15:41.730

Reputation: 125

Please read my question carefully. – Maheshwar Ligade – 2019-12-19T05:27:27.573

can you be more specific about your comment or rephase your question please I might be able to reformulate my answer – pascal sautot – 2019-12-19T09:26:11.247

I have two questions which are interrelated point 1 and point 2. – Maheshwar Ligade – 2019-12-19T10:47:11.567

By any chance could you check if that answered your question. – pascal sautot – 2020-01-31T14:05:12.870

I upvoted for your answer. Thank you for giving me a brief overview. – Maheshwar Ligade – 2020-02-03T05:21:48.517