How can I use my laptop to implement analysis in Python but use a different fast server for computation of analysis?

0

I am working in a company but I am new in the field. We have a central server which is faster than my laptop (of course). So my goal is that I still use my laptop to do python analysis for machine learning but actual computation is happening in the central server so that the computing power is stronger. I am not considering any cloud service as we have a fast server computer.

But I am clueless in this yet. Can anyone advise me how to do this?

Thank you very much in advance.

junmouse

Posted 2019-05-09T07:03:37.770

Reputation: 33

You can work with a very small data on laptop for development (say just 20k samples) and check-in the code. On sever; run the same training with whole data. – Shamit Verma – 2019-05-09T11:26:53.023

This is a standard situation, everybody codes and genarates the pilot on their laptop and then sends it to the headnode of the cluster for farming to the slave nodes via a queuing system. The Jupyter notebook may be a good workaround – M__ – 2019-06-08T19:03:35.930

Answers

1

I have a similar set-up; I run both local and remote installations of Jupyter. On the server I have JupyterHub running, into which i can ssh. Locally I have Anaconda set up etc.

If you are meticulous in mirroring any custom libraries across the system (or even better package them for pip installs) this works quite well.

BAC83

Posted 2019-05-09T07:03:37.770

Reputation: 131