Machine learning model bundled with a library vs. an API



I am thinking to "deploy" a machine learning model (in pickle it is sized 3 megabytes) and after discussing with my developer colleagues, they said it would be better if the model is packed as a python library instead of a microservice (like a rest API).

I wanted to ask what's your view on this: Pickled model packed in a library specifically meant for it vs. a rest API, pros and cons?

I was thinking that having it as a library could possibly be easier to use and wouldn't require to have to worry about deployment, web address etc.


Posted 2020-01-07T15:02:33.653

Reputation: 63



The advantages of deploying the model as a package is the code will be part of the monolith application and require no remote calls thus:

  • No additional operational demands
  • No external dependencies
  • Faster
  • Fewer security issues
  • Same uptime as the rest of the application

In general, most features stay in a single monolith application until there is are specific reasons to create separate services.

Brian Spiering

Posted 2020-01-07T15:02:33.653

Reputation: 10 864


Complexity, just look at this simple deplyoment from fast ai. you get a web app in a couple of simple steps. Where as for restapi you need to know your way around a lot more libraries. For example the cool fastapi (flask+more capabilities). Example app

Online Learning there is no possibility to update (not trivial atleast) your model in the pickeled file. Because you restapi code is modularised this can be modified.

Overhead restAPi is designed to handle larger number of requests, and scale more easily.

Modularity/Control You define exactly how you interect with the client, more control but more complexity when using restAPI

Noah Weber

Posted 2020-01-07T15:02:33.653

Reputation: 4 932

Thanks for sharing this. Never heard of fastapi, will have a look. But my question still holds: machine learning model packed as python library/package vs. rest API. I have seen some implementation of open source libraries on github that have pickle models as part of it. It would be interesting to see a contrast from people, when to go for this format and when to go for a rest API. – c1377554 – 2020-01-10T08:41:27.410