Why not to change the working directory on Microsoft Machine Learning Server

#datascience #microsoft #machinelearning #r

Some time ago I was developing an R service that used MicrosoftR, now called Machine Learning Server, which instead of creating a model as suggested in the documentation examples it just run R routines which operated over a data.table . Therefore, I needed to load my code on each call to the API service since you are not able to use snapshots but only when working with the R-Client (not on any web application).

The problem rises at the moment I tried to change the working directory. As I used relative paths for my different modules, for example:

source('/home/administrator/src/MyMainFile.R')

which in turn loaded some other modules, but as I say, through relative routes:

source('/utils.R')

I needed to set my base directory. As a consequence, after 3 or, luckily, 4 really slow executions, the web node crushed and the only log message I got was:

{ "Link": null, "Message": "c66b3558-9d60-4620-adba-1429fc5355ab", "ExceptionType": null } { "Link": null, "Message": "b16bdf62-1396-4be9-ad54-bd1ed59f0d35", "ExceptionType": null }

No one in MicrosoftR forums gave just only 1 answer. So after days of hitting my head against the wall (because I had deeply and thoroughly tested my code), I decided to just republish the service one line at a time and run it 3 or 4 times. My surprise was huge when only the first line setwd(‘my/base/directory’) was responsible for my desperation. It’s even worse when you see that this is not documented in Microsoft docs.

So the solution was simple:

Remove setwd()
Use absolute paths

Nevertheless, after that experience I decided to never work again with MLS without making a package, so from that moment on I use Rcpp and R6Classes .

Originally published at leolanchas.ml on April 14, 2018.