andstor / copycat-action

:copyright::octocat: GitHub Action for copying files to other repositories

COPYCAT

GitHub Action for copying files to other repositories.

This is a GitHub action to copy files from your repository to another external repository. It is also possible to copy files from/to repository Wikis.

This action runs in a Docker container and therefore only supports Linux.

Usage

The following example workflow step will copy all files from the repository running the action, to a folder named backup in the destination repo copycat-action. If the files already exist at the destination repo, they will be overwritten.

- name: Copy
  uses: andstor/copycat-action@v3
  with:
    personal_token: ${{ secrets.PERSONAL_TOKEN }}
    src_path: /.
    dst_path: /backup/
    dst_owner: andstor
    dst_repo_name: copycat-action

Options ⚙️

The following input variable options can/must be configured:

Input variable	Necessity	Description	Default
src_path	Required	The source path to the file(s) or folder(s) to copy from. For example /. or path/to/home.md.
dst_path	Optional	The destination path to copy the file(s) or folder(s) to. For example /wiki/ or path/to/index.md.	src_path
dst_owner	Required	The name of the owner of the repository to push to. For example andstor.
dst_repo_name	Required	The name of the repository to push to. For example copycat-action.
src_branch	Optional	The name of the branch in source repository to clone from.	master
dst_branch	Optional	The name of the branch in the destination repository to push to. If the branch doesn't exists, the branch will be created based on the default branch.	master
clean	Optional	Set to true if the dst_path should be emptied before copying.	false
file_filter	Optional	A simple pattern for filtering files to be copied. Acts on file basename. For example *.sh.
filter	Optional	A glob pattern for filtering files to be copied. Acts on file paths. For example **/!(*.*).
exclude	Optional	A glob pattern for excluding paths. For example */tests/*.
src_wiki	Optional	Set to true if the source repository you want to copy from is the GitHub Wiki.	false
dst_wiki	Optional	Set to true if the destination repository you want to copy from is the GitHub Wiki.	false
commit_message	Optional	A custom git commit message.
username	Optional	The GitHub username to associate commits made by this GitHub action.	GITHUB_ACTOR
email	Optional	The email used for associating commits made by this GitHub action.	GITHUB_ACTOR @users.noreply.github.com

Secrets

• personal_token: (required) GitHub Private Access Token used for the clone/push operations. To create it follow the GitHub Documentation.

Filtering

Copycat provides several ways of filtering which files you want to copy.
All three types of filtering can be applied simultaneously.

Input variables

file_filter

The file_filterinput variable allows you to filter file basenames. Hence, you can for example only copy all text files by setting file_filterto *.txt. The variable only accepts simple patterns.

filter

The filter input variable provides extensive globbing support. It also supports extended globbing and globstar. The globbing applies to file paths. So, to for example match all files that doesn't have a file extention, the pattern could look like **/!(*.*).

exclude

The exclude input variable can be used to exclude certain paths. It will apply to the file paths. One are for example able to exclude certain deirectory names. Setting exclude to */tests/* results in only copying files that don't lie inside a folder named tests (both directly and indirectly). Here, a file with for example the path foo/tests/bar/baz.txt would not be copied over.

Examples

Copy wiki files to external repo

This workflow configuration will copy all files from the repository's wiki to a folder named wiki in the destination repo andstor.github.io.

This can for example be used to merge several wikies of an organisation, and display them on a custom GitHub Pages site. The Jekyll theme Paper has support for this.

name: Copy
on: gollum
jobs:
  copy:
    runs-on: ubuntu-latest
    steps:
    - name: Copycat
      uses: andstor/copycat-action@v3
      with:
        personal_token: ${{ secrets.PERSONAL_TOKEN }}
        src_path: /.
        dst_path: /wiki/
        dst_owner: andstor
        dst_repo_name: andstor.github.io
        dst_branch: master
        src_branch: master
        src_wiki: true
        dst_wiki: false
        username: nutsbot
        email: [email protected]

Author

The Copycat GitHub action is written by André Storhaug [email protected]

License

This project is licensed under the MIT License - see the LICENSE file for details.

copycat-action's Issues

[FEATURE] dst_repo_owner and dst_repo_name should be merged in dst_repo

dst_repo should take a destination repository in the form of "owner/repo_name" so that this action can be used both by specifying owner and name separately (with a / in between) and by specifying directly owner and repo_name together, for example by using ${{ github.repository }}

This is particularly needed since github does not provide a way to access repo name (such as github.repository.name) despite the fact that they provide github.repository_owner and github.repository

Can´t exclude multiple folders.

Can't seem to set multiple paths as exceptions.
This may just be the expected behaviour of the action still, I figured I'd place an issue just in case I'm missing something.
The docs implied that we could do so and I think the glob pattern is correct for excluding everything under controller and view (these are not root folders).

I am using this config:

on:
  push:
    branches: [ production ]
jobs:
  copy:
    runs-on: ubuntu-latest
    steps:
    - name: Copycat
      uses: andstor/copycat-action@v3
      with:
        personal_token: ${{ secrets.GH_TOKEN }}
        src_path: /admin
        dst_path: /opver3/
        dst_owner: josesoaresif
        dst_repo_name: test01
        dst_branch: main
        src_branch: production
        commit_message: "tests: test commit message"
        username: josesoaresif
        email: [email protected]
        clean: true
        exclude: "**/{controller,view}/**/*"

works great if you change the exclude to a single folder:
exclude: "/controller//*"

Copy files to a different branch of the same repository

Based on the documentation it seems this would be possible.

This doesn't seem to work though. Any suggestions? Each push I'd like to copy the README_files from the master to the gh-pages branch.

Cleaning a directory

I would like to use clean only in a specific folder and keep the rest of the file structure intact.

Actual behaviour:
Is cleaning al my repo and copy just the src_path to dst_path

This is my code

name: Copy-New  
on: push  
jobs:  
  copy:
    runs-on: ubuntu-latest
    steps:
    - name: Copy
      uses: andstor/copycat-action@v3
      with:
        personal_token: ${{ secrets.copy_new }}
        src_path: icn_export/
        dst_path: ./
        dst_owner: yceballost
        dst_repo_name: yceballost.github.io
        clean: true

Thanks

##[error]Container action is only supported on Linux

I'm getting the following error when I try to use this action.
##[error]Container action is only supported on Linux

I assume this is because my GitHub agent is running Windows.

Is there any plans to make this Windows friendly?

Add usage description in README.md file

Removes files in the Destination repository.

I expect the GitHub action to sync directory in the source repository with the directory in the target repository. Whether I add a new file, update an existing file or delete a file.

copycat-action GitHub action works fine if I add a new file or update an existing file. However, when I delete a file (in the source repository), it does not delete the file in the target repository.

How can I truly sync both the directories in the source and target repository? Thanks.

Copy each `commit_message` from source folder's git history

Hi! I know this package has a commit_message option. But I'd like to have that option set automatically based on each commit on the src repo. AFAIK, this could be a little bit tricky to implement since I think this action just copies all the content in the src repository to the destination repo, without taking into account each commit's message content.

[FEATURE REQUEST] Copy from GitHub Action Workspace to the repo's branch

Is there any way that we can copy files/folders from the the GitHub Workspace into one of the repo's branches?

How to use env variables in COMMIT_MESSAGE?

I tried this:

jobs:
  copy:
    runs-on: ubuntu-latest
    steps:
      - name: Copycat
        uses: andstor/copycat-action@v3
        with:
          personal_token: ${{ secrets.PERSONAL_TOKEN }}
          src_path: workflow-templates/extension-test.yml
          dst_path: .github/workflows/test.yml
          dst_owner: lens0021
          dst_repo_name: Sanctions
          dst_branch: master
          commit_message: Update ".github/workflows/test.yml" from ${{ env.GITHUB_REPOSITORY }} (https://github.com/${{ env.GITHUB_REPOSITORY }}/commit/${{ env.GITHUB_SHA }})

(results Update ".github/workflows/test.yml" from (https://github.com//commit/))

and this:

commit_message: |
  Update ".github/workflows/test.yml"

  from "${{ env.GITHUB_REPOSITORY }}" (https://github.com/${{ env.GITHUB_REPOSITORY }}/commit/${{ env.GITHUB_SHA }}")

(results the same)

and this:

commit_message: |
  Update ".github/workflows/test.yml"

  from "${{ GITHUB_REPOSITORY }}" (https://github.com/${{ GITHUB_REPOSITORY }}/commit/${{ env.GITHUB_SHA }}")

(results "The workflow is not valid. .github/workflows/sync-extension-text.yml (Line: 26, Col: 27): Unrecognized named-value: 'GITHUB_REPOSITORY'. Located at position 1 within expression: GITHUB_REPOSITORY")

and this:

commit_message: |
  Update ".github/workflows/test.yml"

  from ${GITHUB_REPOSITORY} (https://github.com/${GITHUB_REPOSITORY}/commit/${GITHUB_SHA})

(results Update ".github/workflows/test.yml"
from ${GITHUB_REPOSITORY} (https://github.com/${GITHUB_REPOSITORY}/commit/${GITHUB_SHA}))

commit_message: |
  Update ".github/workflows/test.yml"

  from "${GITHUB_REPOSITORY}" (https://github.com/${GITHUB_REPOSITORY}/commit/${GITHUB_SHA})")

(results: Update ".github/workflows/test.yml"

from "${GITHUB_REPOSITORY}" https://github.com/"${GITHUB_REPOSITORY}"/commit/"${GITHUB_SHA}")

commit_message: |
  Update \".github/workflows/test.yml\"

  from ${{ env.GITHUB_REPOSITORY }} (https://github.com/${{ env.GITHUB_REPOSITORY }}/commit/${{ env.GITHUB_SHA }})

(results: Update \".github/workflows/test.yml\"

from (https://github.com//commit/))

or this:

commit_message: Update ".github/workflows/test.yml"\n\nfrom ${{ env.GITHUB_REPOSITORY }} (https://github.com/${{ env.GITHUB_REPOSITORY }}/commit/${{ env.GITHUB_SHA }})

(results: Update ".github/workflows/test.yml"\n\nfrom (https://github.com//commit/))

What is the certain way to pass environment variables?

Deleting contents of parent folder

Hi there,

Thank you for the great action, I use it quite a bit.

I'm having problems with the deletion of particular folders when merging into a repo.
The target repository has a directory structure like so:

content
│
└───client-1
│   └───project-1
│       │   index.html
│       │   ...
│   └───project-2
│       │   index.html
│       │   ...
└───client-2
│   └───project-1
│       │   index.html
│       │   ...

I copy the source of public (source repo) to the client's folder. However, if there are multiple projects, it will delete them all.
For example, if project-1 already resides in the client-1 folder and project-2 gets copied, it will delete project-1.

I'm using the clean flag, I'm not sure if that has anything to do with it? Although I do want to remove the project folder before copying.

      # Step 7: Copies the public folder to the staging repo.
      - name: Copy
        uses: andstor/copycat-action@v3
        with:
          personal_token: ${{ secrets.GIT_TOKEN }}
          src_path: public/.
          src_branch: staging
          dst_path: /content/client-1/example-project
          dst_owner: xxxx
          dst_repo_name: xxxx
          clean: true
          commit_message: Update /content/client-1/example-project

Thanks for your help in advance.

Add ENV variable for user.name and user.email

By making the user.name and user.email configurable, one might set a bot account as the author of the commits.

REPOSITORY IS DEAD?

I Created a Competitor called Git Get Cloned Action

Why?

• code lacked documentation
• I realized most of the code is procedural and kept it that way
• there were several features missing from copycat that I wanted to add and felt from scratch with better documentation was the way to go
• development has stalled at 2021

Action terminates successfully despite failure

I was using the copycat action with the wrong token, that didn't had needed rights. The copying failed (file wasn't copied to the other repo), but the action terminated successfully. I would have expected the action to fail if the token hasn't got the rights needed.
I don't know if this is a bug, just wanted to leave this here, feel free to close this issue if this isn't of interest for you.

How to include LFS files?

I have repos with LFS documents, how do include those files?

Personal Access Token in a Organization Secret "PERSONAL_TOKEN"

If you are planning to implement the CopyCat GitHub Action in your GitHub Organization, be aware that one of the Org Admins should create a Personal Access Token in their GitHub account > Settings > Developer Settings > Personal access tokens at https://github.com/settings/tokens and give the PAT admin:org privileges. Then take that key and create a Org Secret called "PERSONAL_TOKEN"

Org Secrets do not implicitly have permissions associated with them. The organization administrator has permissions, not the org secret.

I spent quite a few hours trying to understand why the CopyCat GHA didn't copy files between my organization repos. I was trying to incorrectly use a Org Secret called "PERSONAL_TOKEN" which didn't have permissions.

I would be willing to submit a PR to improve the CopyCat README so my future self will know how to set up this GHA in GitHub Orgs.

Add option to copy just folder contents instead of folder AND contents

In my use case I am using GitHub actions to build my Hugo site and would like to use Copycat Action to copy the compiled site directory's contents into the root of my GitHub pages repo. I have Copycat Action working, but it's copying the folder AND it's contents. It would be great if you could add an option to copy just the contents of a folder and not the folder itself. If you know a good workaround I'd also welcome that. I'm pretty new to GitHub Actions but it would definitely help out tremendously.

Creating a new Branch and making a MR to Master?

I've stumbled upon this action which seems to be exactly what I want. I'm working on a Project that uses ReactJS frontend + RoR backend. To keep things separate, we are using 2 separate repositories.
What I'm trying to do with CopyCat is automating the code transfer from frontend -> backend.

It seems to work as intended, but the problem is that CopyCat can't create a new branch on the target repository. This would make things much better for everyone. Maybe even make it create an automatic merge request?

i want to add a param src_repo_name

when i use workflow_call .the src repo is not i want. thinks!

Automatically detecting the default branch name.

Automatically detecting the default branch name if the src_branchand/ordst_branch` names are not provided by user.

Background:
A while back, GitHub started naming the master branch "main" by default. Since copycat-action looks for a "master" branch by default, you have to explicitly provide branch names if you are using Github's new naming convention "main" for your default branch. Obviously if you are already using custom branch names, you have to explicitly specify them.

Instead of assuming each default branch name is always "master", we need to automatically detect the name to explicitly use it to clone the src and dst repos.

I believe below is a reliable way to get the default branch name.

git remote show upstream | grep "HEAD branch" | sed 's/.*: //'

I will work on a PR to test this out.

Feature proposal: --no-clobber option for the copy command

In our usecase, we would like to have an option to NOT overwrite the already existing files.
I would like to propose a new action option, called no_clobber: false(default)/true.
The interpretation of it would be: do not overwrite an existing file
If you are also interested in such feature, than I could start working on the PR.

Exclude pattern not working

Hi! Firstly, thanks for this awesome tool.

I was trying to not get any of the files under the seeds dir to the mirror repo, or the README.md. I came up with this glob pattern that (should) work
{**/seeds/*,README.md}
Do you know why it isn't working?

fatal: could not read Password for 'https://***@github.com': No such device or address

Hello, I cannot get it to work. What I am trying to accomplish is copying the changelog from one repo into another, within an organization. I have ensured that my PAT does not have a space in it, and I have tried both ${{ secrets.GITHUB_TOKEN }}, and my PAT with all permissions granted.

Any ideas?

Add option to delete all before push

I often have issue, if I rename, move folder, or delete a file. It's not removed from where I copy them to.
Is there a solution for this already? Else maybe a simple "clean" option could be added which deletes the folder first?

Add ability to add filter for SRC_PATH files

It should be possible to only copy files passing a certain filter.

Determine how to handle potential merge conflicts

[Bug] Not copying directories containing spaces

As title says.
Also, I tried to use exclude to avoid copying .github/workflows/ but that also doesn't seem to work.

Otherwise this is an awesome action!

Exclude not working for me

Hello,

For some reason I am not able to get the exclude to work. It is copying the wofklow action itself in to the destination repository. I am sure I am just not doing the globs right. Any help on my config would help:

  - name: Copy
    uses: andstor/copycat-action@v3
    with:
      personal_token: ${{ secrets.PERSONAL_TOKEN }}
      src_path: /.
      dst_path: /site1/
      dst_owner: woodman231
      dst_repo_name: woodman231.github.io
      exclude: /.github/*
      clean: true

I even tried exclude: */.github/* but then it tells me that is an invalid token and won't allow me to save the yml file.

Permission error

Hi!

I don't understand what is happening when I try to use copycat. This error appears

fatal: could not read Password for 'https://github.com': No such device or address

This is my code

name: Copy
on: push
jobs:
  copy:
    runs-on: ubuntu-latest
    steps:
    - name: Copy
      uses: andstor/copycat-action@v3
      with:
        personal_token: ${{ secrets.PERSONAL_TOKEN }}
        src_path: icn_export/
        dst_path: /icn_export/
        dst_owner: yceballost
        dst_repo_name: yceballost.github.io

I did a personal token with a lot of permissions but doesn't work.

Thanks

Can we use github.token or secrets.github_token instead of personal access token

Hi ,

Can we use github.token or secrets.github_token instead of personal access token ?

Regards,

how to ignore some directories?

I have this job here
:

  copy_packages:
    runs-on: ubuntu-latest
    steps:
      - name: Copy config/packages/*
        uses: andstor/copycat-action@v2
        with:
          personal_token: ${{ secrets.COPY_DIST_TOKEN }}
          src_path: /config/packages
          src_filter: '!/test/*|!/dev/*'
          dst_path: /config
          dst_owner: zikula
          dst_repo_name: distribution

I am trying to get the copy to ignore /config/packages/dev and /config/packages/test but clearly the glob pattern is wrong because it isn't working. Can you help?

Can't get it working

Hello,

we (@craigh and me) are trying to use this action, but weren't successful yet.

First we had some errors like fatal: could not read Password for 'https://***@github.com': No such device or address which was probably caused by the token had not being setup correctly.

The current state is:

1. I created a personal access token (at my account) and added it as secret to the repository which runs the action (owned by an organisation).
2. The workflow looks like this: https://github.com/zikula/core/blob/master/.github/workflows/copy-to-dist.yml
3. The action execution can be seen here: https://github.com/zikula/core/runs/492015952
4. Current error message: fatal: unable to access 'https://': Could not resolve host: info Cloning 'zikula/core' failed.

We are a bit lost here and can only do "try and error". Can you give us an advice?

Overriding COMMIT_MESSAGE

Please consider adding an option to specify different COMMIT_MESSAGE in configuration

Cannot find newly created folder called build

I run scripts npm install and npm run build and a new folder called build would be created in the github action like this :
steps: - uses: actions/checkout@v2 - name: Use Node.js ${{ matrix.node-version }} uses: actions/setup-node@v2 with: node-version: ${{ matrix.node-version }} cache: "npm" - run: npm install - run: CI=false npm run build - name: Copy uses: andstor/copycat-action@v3 with: personal_token: ${{ secrets.PERSONAL_TOKEN }} src_path: /build dst_owner: name dst_repo_name: page clean: true
But it doesn't recognize the newly created folder and says it doesn't exist .