This repository serves as the editorial venue for ML Reproducibility Challenge 2021 accepted papers. If your paper is accepted to the challenge, you must follow the instructions to submit your camera ready paper by May 4th, 2022, Wednesday EOD.
ReScience editorial process consists of porting the existing latex sources into ReScience template, and then adding necessary information relevant for final publication. Authors of RC2021 accepted papers are requested to submit their camera-ready papers as outlined in the steps below. The project structure of this repository is organized as:
- 2021/
<paper_citekey><-- OpenReview citekey of the accepted paper at RC2020. List of accepted papers inaccepted.bibopenreview<-- location for the source files submitted to openreviewjournal<-- location of ReScience C Template files
Some of you may have used Overleaf to write and submit your papers. We recommend to setup the Latex development pipeline locally in your system for full control over the editorial process. If you use a Mac, then use the following Homebrew formula:
brew install --cask mactexFor Ubuntu/Debian:
apt-get install texliveFor Windows: Download Tex Live from source.
Note: We also use Git LFS to upload and store the compiled pdfs. You need to run this once in your local terminal to setup LFS uploads: git lfs install.
- Fork this repository and clone it locally (Note: clone recursively using
--recurse-submodulesor--recursiveflag) - Find your
<paper_citekey>from accepted.bib. It is the key after@inproceedings. - In 2021 directory, setup your report folder using the
<paper_citekey>. For example, if your citekey isahmed2022re, then run the setup.sh with the argument:
cd 2021/
./setup.sh ahmed2022reThis will create the folder ahmed2022re and two subfolders, openreview and journal along with the boilerplate code.
- Test building your report using build_all.sh script. This assumes you have the correct texlive distributions installed.
cd 2021/
./build_all.shYou can find the logs in your terminal, as well as saved in 2021/<paper_citekey>/journal/build.log file.
- Open the compiled pdf. It will be located at:
2021/<paper_citekey>/journal/article.pdf.
Now we need to organize your contents a bit so as to make life easier for the editors! (:bulb: When setting up the folder in Step 1, we already add boilerplate for the following steps to help you understand the import process.)
-
Add the latex content of your paper in the
openreviewfolder of your paper citekey. (You can check your correct folder from accepted.bib) -
Use the
packages.texin youropenreviewfolder to add all your import lines in Latex. (Reference). Import this file in your main tex file. -
Convert your main content in a
content.texfile in theopenreviewfolder, and import it in yourmain.texfile. Feel free to re-organize your latex code in different files, as long as everything can be imported incontent.tex. Yourmain.texfile could look like following:\documentclass{article}[12pt] \input{packages.tex} \title{Your title} \author{Your authors} \begin{document} \maketitle \input{content.tex} \end{document}
-
[Optional] Test that the
openreviewcompilation is working. Since the contents ofopenreviewis isolated, you can test this in the Overleaf template directly. -
Now, head over to the
journalfolder in your paper folder, and edit thecontent.texsuch that it is importing the correct../openreview/content.texfile. Yourcontent.texfile could look like this:\input{../openreview/content.tex}
Note: It is possible to face compilation issues in imports if you have extra packages/commands. In such case, feel free to copy your main tex contents in the
content.texappropriately. -
Edit the
article.texfile to include yourpackages.texfile before\begin{document}. (Reference) -
If you have an appendix, import it in
article.texafter\printbibliographyline. (Reference)
In the journal folder you will find a metadata.yaml file. This file is crucial to fill correctly for the journal to publish your paper correctly. Here is a reference metadata.yaml file from our last iteration.
- Add the
titleof your paper. The paper title should start with[Re]. - Add the author information in
authors. This is a yaml list, so just copy paste the following block for all your authors:
- name: John Doe
orcid: <your ORCID id>
email: <your email>
affiliations:- Make sure to get your ORCID id.
- In the
affiliationssection, add the list of affiliations you have in your author list. At the same time, add thecodeof the affiliations in theauthorssection in a comma separated format. Add a*for contact author. - In the
codesection, provide theurlto your Github. - Again, in the
codesection, provide the link to Software Heritage Foundation archival link to your code inswh. To get your SWH identifier, provide the link to your code in Software Heritage Archive, which will queue it for archival. You can follow the progress of your request, and once the status changes to "succeeded", you can click on the URL link to arrive at the archived page of your code. From this page, click on the "Permalinks" side-button, and copy the code beginning withswh. Place this code in the yaml file underswh. (Reference]) - In the
replicationsection, add the necessary information about the paper you reproduced. - Provide your abstract in the
abstractfield (including double quotes) domainshould contain "ML Reproducibility Challenge 2021"- In
reviewsection, paste the url to the OpenReview forum of your paper. - Leave the fields
contributors,dates,articleandjournalfields blank, as those will be populated by the Area Chairs / ReScience editors.
💡 You can test your metadata is correct by running this Python Script:
cd 2021/
python check_yaml.pyTo submit your report, open a Pull Request (PR) to main branch. Our Area Chairs will then review and accept the submission, and proceed towards obtaining the journal metadata. Even if you have completed the report partially due to compilation issues, we strongly recommend to open your Pull Requests early to 1. get quicker help from our Area Chairs, and 2. take advantage of Github Actions for auto compilation.
This repository is also setup to use Github Actions to automatically compile the reports. After you submit your Pull Request, the CI will run the compilation step, check for errors, and then upload the compiled pdfs as "Artifacts" in the workflow run for the commit.
While compiling you might face latex issues / errors. In such events, first inspect the log carefully for any offending blocks. If you are unsure, submit your PR anyway, and open an issue linking to your PR, where you attach the logs. To capture the log, pipe the make output to a file and add it to the PR (make > build.log). Our AC's will gladly help to you fix the issues! Also, do consult the codebase of our last iteration or previously closed issues to narrow down any specific latex issues you are facing.
- Please do not import
xcolorin yourpackages.tex, as rescience.cls already imports this package - Please remove the import of
fontencfrom yourpackages.tex. ReScience uses custom fonts, which breaks when fontenc is loaded on top of it. (Check #4 for more discussion)
Files and latex sources for MLRC 2020 edition is now moved to 2020 folder.
- Koustuv Sinha (@koustuvsinha) Mail
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent