Where to publish replications of machine learning papers?

6

0

If I replicated the results of a machine learning paper, is there a website that I can publish my code to that collects informal replication attempts?

https://paperswithcode.com/ allows anyone to publish implementations, but I am looking for something similar that allows anyone to publish implementations along with replicated experimental results. Does such a website exist?

Jared Jacobsen

Posted 2020-05-14T14:50:59.230

Reputation: 63

Why isn't a code-hosting service, like Github, enough for your purposes? – nbro – 2020-05-14T19:36:21.083

The effort of replicating a paper would be somewhat wasted if the results were not easily discoverable. A database for linking papers to replications might be better for this than Github. – Jared Jacobsen – 2020-05-18T15:28:30.843

So, you mean something more sophisticated than Github that keeps also track of datasets? – nbro – 2020-05-18T15:31:48.830

Right, and which makes it easy for others to discover your replication and to integrate it with the results of other replication attempts. – Jared Jacobsen – 2020-05-18T16:37:10.297

Answers

1

I think you're looking for something like Comet.ml or IDSIA's sacred, but bear in mind that these tools can also be combined with code-hosting services such as Github.

nbro

Posted 2020-05-14T14:50:59.230

Reputation: 19 783

1

OpenReview (https://openreview.net/) may be what you're looking for. It's both a platform for ML conferences to organize their reviewing process as well as an arXiv-like preprint repository.

One major machine learning conference (ICLR) has been holding "reproducibility challenges," in which participants can submit replication reports of existing papers, but that probably only applies to papers submitted to that conference.

There is a general preprints section, which might be your best bet.

Disclaimer: I used to be on the OpenReview development team

user37097

Posted 2020-05-14T14:50:59.230

Reputation:

"I am looking for something similar that allows anyone to publish implementations along with replicated experimental results.". I am not sure how OpenReview provides this service of sharing code, so I am not sure why you think OR would be suited for the OP's needs. – nbro – 2020-05-15T21:39:43.027