zafirshi / hf-daily-paper-newsletter

Daily Newsletter for 🤗 Daily Papers

This repository sends out a newsletter at 0 20 * * * collected from 🤗 Daily Papers to the subscribers. If you want to subscribe, please join the newsletter group.

Everything is automated via GitHub Action, so no paid maintanance at all.

How this works?

This whole project works in 2 steps broadly in the perspective of GitHub Action workflow.

1. collect papers

   1. Download the metadat of daily papers from 🤗 Daily Papers API Endpoint. The target date is the yesterday since it is unclear when the full list of daily paper is fully updated. The target date is dynamically calculated via date command.
   2. Run go run main.go parse command to (1) parse the downloaded the metadata file (JSON), (2) generate categories/tags of each paper via Gemini API, and (3) generate reformatted yaml file for each paper which is suitable for newsletter program. Each yaml file will be stored inside the current directory.

2. send newsletter

   1. Grasp all the yaml files from the current directory.
   2. Fill in the email HTML template with the contents from the yaml files.
   3. Archiving each papers' yaml file under the archive directory
   4. Generate markdown files based on the existing categories/tags. Add link of the corresponding archived paper yaml file to each markdown file.
   5. Send out email to the receivers which are specified in the config.yaml file.
   6. Clean up the current directory.

Experimentally, summary of each paper is generated by Upstage 's SOLAR-10.7B-Instruct model. Since abstract is too long to be contained in an email, abstracts needed to be shrinken.

Appearance

Todos

• Auto Translation to other languages (first target -> Korean)