sinedied / devto-cli

Dev.to authoring CLI to create and publish markdown files as articles, using assets hosted on GitHub.

$ npx @sinedied/card.

┌────────────────────────────────────────────────────────────┐
│                                                            │
│   Yohan Lasorsa / @sinedied                        【ツ】   │
│                                                            │
│       Work: Senior Cloud Developer Advocate @ Microsoft    │
│     GitHub: https://github.com/sinedied                    │
│    Twitter: https://twitter.com/sinedied                   │
│     Dev.to: https://dev.to/sinedied                        │
│   LinkedIn: https://linkedin.com/in/yohanlasorsa/          │
│                                                            │
│       Card: $ npx @sinedied/card                           │
│                                                            │
└────────────────────────────────────────────────────────────┘

devto-cli's Issues

Update failed: HTTPError: Response code 429 (Too Many Requests)

Hello, many thanks for this wonderful tool 🥰

I have this error when I push multiple (at least 5) articles at once :

Update failed: HTTPError: Response code 429 (Too Many Requests)

So when I update a minor thing in every articles, I fall back to this :

for file in *.md; do sleep 2 && dev push $file; done

Would it be possible to limit requests to dev.to ?

devto init creates broken .github/workflows/publish.yml file

Publish.yml created by init command creates contains values not used by the action:

Unexpected input(s) 'repository', 'files_glob', valid inputs are ['devto_key', 'github_token', 'files', 'conventional_commits']

Template repository has correct format: https://github.com/sinedied/publish-devto-template/blob/master/.github/workflows/publish.yml

Github CI tries to commit when there is nothing to commit

The publish.yml workflow tries to make a git commit even when it appears like there is nothing to commit. This appears to happens when I've published the posts from my local and the CI updates posts.

Run sinedied/publish-devto@v2
/usr/bin/docker run --name db2d7b541e79b11954185a9e3aef5ffeb8abc_8c4255 --label 7db2d7 --workdir /github/workspace 
......
Found 7 article(s)
- Retrieving articles from dev.to…
......
/usr/bin/git add posts/2015/....
/usr/bin/git -c user.name="dev.to bot" -c user.email="[email protected]" commit -m chore: update published articles [skip ci]
On branch main
Your branch is up to date with 'origin/main'.

nothing to commit, working tree clean
Error: Error: Cannot commit changes: The process '/usr/bin/git' failed with exit code 1

and the publish.yml file I have:

# This workflow will push and update articles on dev.to each time a new
# commit is pushed to the main branch.
#
# To make it work, you need a dev.to API key, see:
# https://docs.dev.to/api/#section/Authentication/api_key
#
# Once you have generated the token, you need to add them as a secret in your
# GitHub repository:
# - DEVTO_TOKEN: your dev.to API key
# See https://help.github.com/en/actions/configuring-and-managing-workflows/creating-and-storing-encrypted-secrets#creating-encrypted-secrets
# for more information about GitHub secrets.

name: publish
on:
  push:
    branches: [main]

jobs:
  publish:
    runs-on: ubuntu-latest

    steps:
    - uses: actions/checkout@v2
    - uses: actions/setup-node@v1
      with:
        node-version: 14

    - name: Publish articles on dev.to
      uses: sinedied/publish-devto@v2
      with:
        # Your dev.to personal API key to publish and update articles.
        # See https://docs.dev.to/api/#section/Authentication/api_key
        devto_key: ${{ secrets.DEVTO_TOKEN }}
        # Your GitHub personal access token, used to create commits for updated files.
        # If you have a protected branch, you need to create a commit with the
        github_token: ${{ secrets.GITHUB_TOKEN }}
        # (Optional) The files to publish. Default is "posts/**/*.md"
        files: 'posts/**/*.md'
        # (Optional) The git branch to use. Default is 'main'.
        branch: main
        # (Optional) Use conventional commit messages. Default is false.
        # See https://www.conventionalcommits.org. 
        conventional_commits: true

A mechanism for associating a post to a Dev.to Organization

It appears that by default posts get created as Personal. I was wondering if there was a setting to have it default to an Organization I'm part of.
You can manually edit it in Dev.to later, and all seems well with the sync later on anyways, so it isn't a blocker at the moment.

Also, I tried to get it working when the repo was private, and local assets weren't accesible so it would fail. The only options are to either have the repo public or manually upload the content somewhere public correct?

Crashes on colon

I was testing with the command: devto init --pull -t <token> but it crashes when the article title has colons, like this image:

I did some research and it seems that this problem happens with Gray Matter. Specifically in line 54 of article.js

devto-cli/lib/article.js

Line 54 in 1d447d9

const article = matter(data.body_markdown);

For some reason, it is not treating correctly the colon, so I tried to put this regex: data.body_markdown.replace(/(?<=title\:\s)(.*)?/gmi, '$1'), which puts every text inside quotes in title: , this way js-yaml can parse it right, however, it also did not work.

I tried to replicate the issue in this Repl (feel free to test it) and it worked, so I believe it's something related to how the articles are being sent to the getArticleFromRemoteData function.

Sorry I couldn't solve this

[Placeholder] assets

Allow local structure for posts

I published a couple of old posts to Dev.to and the process was very smooth. I have markdown file structure with year to allow some structure within the posts lists locally.

posts/
   2015/*.md
   2017/*.md

However when I pull the posts back from dev.to
dev i -p -s --verbose
the tool does not find the old posts in the subfolders but copies them directly under posts/

Am I missing some configuration option, some flag maybe?

To me this looks like the filename is converted from the front matter title on pull. While it works just fine when publishing pulling does not consider all local files (my existing post files in subfolders) while pushing out works.

Concrete example:

1. write post to file posts/2022/example.md with title: Example, commit to git
2. publish it with dev p posts/2022/example.md
3. pull all posts with dev i -p -s
4. the newly created post (and all other posts) appears now also in the file posts/example.md

devto Found 0 remote article : Error: Cannot find published article on dev.to

@sinedied first of all this look like a very interesting tool (although it has not worked for me yet and hence reaching out to you for some help)

I tried to use both your Github Action as well as the standalone CLI utility, both did not work.

So I am trying to publish posts/blog-m1.md which is a fresh new blog on Dev.to using your CLI. It seems like the CLI is first trying to retrieve articles from dev.to (which i am not sure why it does that) and later trying to push and fails with error Found 0 remote articles

The details logs are here, i would appreciate your help @sinedied if you can help me get around this error

$ dev push posts/blog-1.md --token xxxxxx --verbose
  push files: [ 'posts/blog-1.md' ] +0ms
  push options: {
  push   devtoKey: 'xxx',
  push   repo: undefined,
  push   branch: undefined,
  push   dryRun: false,
  push   reconcile: false,
  push   checkImages: true
  push } +3ms
  repo Repository found in git origin: ksingh7/rhd-blogs +0ms
  push repository: { user: 'ksingh7', name: 'rhd-blogs' } +48ms
  push branch: main +18ms
Found 1 article(s)
  push Retrieving articles from dev.to… +17ms
  devto Requesting articles (page 1) +0ms
  devto Found 0 remote article(s) +540ms
  push Retrieved 0 article(s) +540ms
  push Pushing articles to dev.to… +0ms
Error: Cannot find published article on dev.to: Mastering Machine Learning
Push failed

Tune `relativeImageRegex` used to find and update relative Image URL's

Hi @sinedied ! Thanks so much for coming up with this library.

I would request to look into tuning regex relativeImageRegex used in function updateRelativeImageUrls to update relative image URLs in a markdown with their absolute paths, in order to support more image link formats.

For example, this increasingly popular gist on a "how-to" for image resizing in GitHub markdown:
https://gist.github.com/uupaa/f77d2bcf4dc7a294d109

I personally deploy my blog articles using Jekyll and GitHub Pages, so as mentioned in the same article, we might see this syntax in the wild in .md files used w/ Jekyll:

![alt](image.png){: width="50%"}

I have tested it extensively on my local. I believe I was able to update the regex to include an optional capturing group for the height and width modifier, such as {: height=200px } for example.

This is my updated regex I have tested with:

!\[(.*)]\((?!.*?:\/\/)([^ ]*?) *?( (?:'.*'|".*"))? *?\)( *\{ *\:? *(?:width|height) *[:=] *["'0-9%A-Za-z]+ *;? *})?

Snippet of relevant parts for while loop replacement I'm using in code:

    while ((match = relativeImageRegex.exec(article.content))) {
        const [
            link,
            alt = '',
            imagePath,
            title = '',
            sizeModifier = '',
        ] = match;

        if (imagePath) {
            // const fullPath = getFullImagePath(basePath, imagePath);
            const newLink = `![${alt}](${getResourceUrl(repository, branch)}${imagePath.replace(/^\/+/g, '')}${title})`;

            content = content.replace(link, newLink);
            if (sizeModifier)
                content = content.replace(sizeModifier, '');
        }
    }

Regex Playground can be found here:
https://regex101.com/r/r6HJT8/1

Thanks,
Ritvik

initGitRepository() harcoded to create main branch even instead of branch injected with dev init --branch

867d905 added feature to create main branch instead of master when initing directory. initGitRepository() method probably should create a branch requested by user with --branch flag or with interactive prompt.

Error: Subject parameter value width cannot be greater than the container width

Hello, thank you for this great tool 😊

I have this error from today when I push :

Error: Subject parameter value width cannot be greater than the container width

But the articles seems to be updated just fine, so quite ignoring it for now.

Verbose logs :

articles git:(30-elk) ✗ dev --verbose push 24_03_GITLAB-majestic-single-server-runner.md
  push files: [ '24_03_GITLAB-majestic-single-server-runner.md' ] +0ms
  push options: {
  push   devtoKey: 'XXX',
  push   repo: undefined,
  push   branch: undefined,
  push   dryRun: false,
  push   reconcile: false,
  push   checkImages: true
  push } +1ms
  push repository: { user: 'bcouetil', name: 'articles' } +1ms
  push branch: main +0ms
Found 1 article(s)
  push Retrieving articles from dev.to… +9ms
  devto Requesting articles (page 1) +0ms
  devto Found 14 remote article(s) +498ms
  push Retrieved 14 article(s) +507ms
  push Pushing articles to dev.to… +0ms
  article Found 9 image(s) to check for "🦊 GitLab CI : The Majestic Single Server Runner" +0ms
  article Checking image "https://raw.githubusercontent.com/bcouetil/articles/main/images/buckle-up-dorothy.jpg"… +0ms
  article Checking image "https://res.cloudinary.com/practicaldev/image/fetch/s--asKSgsVB--/c_imagga_scale,f_auto,fl_progressive,h_420,q_auto,w_1000/https://thepracticaldev.s3.amazonaws.com/i/9i6bs4g6cx05jeagfhum.png"… +2ms
  article Checking image "https://raw.githubusercontent.com/bcouetil/articles/main/images/gitlab/beefy-fox-2.jpg"… +1ms
  article Checking image "https://raw.githubusercontent.com/bcouetil/articles/main/images/gitlab/bench-full-pipeline.jpg"… +0ms
  article Checking image "https://raw.githubusercontent.com/bcouetil/articles/main/images/gitlab/bench-build-pipeline.jpg"… +1ms
  article Checking image "https://raw.githubusercontent.com/bcouetil/articles/main/images/gitlab/beefy-fox-3.jpg"… +112ms
  article Checking image "https://raw.githubusercontent.com/bcouetil/articles/main/images/gitlab/bench-results.jpg"… +6ms
  article Checking image "https://raw.githubusercontent.com/bcouetil/articles/main/images/gitlab/beefy-fox-4.jpg"… +14ms
  article Checking image "https://raw.githubusercontent.com/bcouetil/articles/main/images/gitlab/beefy-fox-1.jpg"… +4ms
  push Article "🦊 GitLab CI : The Majestic Single Server Runner" has pending changes +1s
  article Saved article "🦊 GitLab CI : The Majestic Single Server Runner" to file "24_03_GITLAB-majestic-single-server-runner.md" +860ms

Error: Subject parameter value width cannot be greater than the container width.
Push failed