download-directory / download-directory.github.io

Web App: Download just a sub directory from a GitHub repo.

download-directory.github.io

Download just a sub directory from a GitHub repo

GitHub doesn’t let you download a single folder from a repo, which might be necessary when you just need a few files from a large repository.

This tool will handle the download of all the files in a directory, in a single click, after you entered your token.

The download starts automatically when you visit pass the link to the GitHub directory as url parameter, like:

download-directory.github.io?url=https://github.com/mrdoob/three.js/tree/dev/build

This is an alternative to the existing GitZip and DownGit but without the cruft.

Related

• list-github-dir-content - List all the files in a GitHub repo’s directory
• Refined GitHub - Browser extension that adds a link to this app to GitHub (and much more)

License

MIT © Federico Brigante

download-directory.github.io's Issues

Token through query param is deprecated

Hi,
Github just sent me this, I thought you should be aware of that.

Thanks for your amazing work.

Hi @mathieutu,

On March 29th, 2020 at 17:24 (UTC) your personal access token (Download GitHub directory) using Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:75.0) Gecko/20100101 Firefox/75.0 was used as part of a query parameter to access an endpoint through the GitHub API:

https://api.github.com/repositories/****/git/trees/master

Please use the Authorization HTTP header instead, as using the access_token query parameter is deprecated. If this token is being used by an app you don't have control over, be aware that it may stop working as a result of this deprecation.

Depending on your API usage, we'll be sending you this email reminder on a monthly basis for each token and User-Agent used in API calls made on your behalf.
Just one URL that was accessed with a token and User-Agent combination will be listed in the email reminder, not all.

Visit https://developer.github.com/changes/2020-02-10-deprecating-auth-through-query-param for more information about suggested workarounds and removal dates.

Thanks,
The GitHub Team

Filenames starting with "ad" are blocked by ad-blockers

This one has 261 files and it stops at 260: https://download-directory.github.io/?url=https://github.com/cjdowner/cryptocurrency-icons/tree/master/svg/color

Unable to view page

Maybe JavaScript file is missing.

Filenames with spaces

After regenerating a new token for download-directory, I tried to download a directory from a repository, but it seems to download any files.

When I try to unzip the file, since there wasn't files inside, it shows a warning

I checked multiple times: 😂there were files on the repo!

• Is there any way to solve this?
• Is this issue caused by a wrong setup of Github token?
• Is this even a valid issue, or am I just doing it wrong?

Support for GitHub Enterprise

I'd love to download directories in my GHE instance. This should be doable!

Add `url` form field on page

Currently it's only available via url parameter, but getting it there should be easy:

1. Append this to line 10 in the JS file

<form><input name="url" placeholder="URL to folder on GitHub.com"></form>

2. Add CSS to make it smaller

GitHub API format update

Behind GitHub’s new authentication token formats

download-directory didn't identify the new format API

Repeating a download after invalidating the originally used token

I (foolishly) didn't keep a note of my first ever token, which I used at https://download-directory.github.io/?url=https://github.com/QWERTYUIOPYOZO/ReFont/tree/master/src in Waterfox. The original token no longer exists.

Now whenever I aim to repeat that download, I get just this:

– not progressing to a download.

When I click token, a more recently generated token is not accepted (AFAIR the bullets, in the field, appear red).

As a workaround I can visit https://download-directory.github.io/?url=https://github.com/QWERTYUIOPYOZO/ReFont/tree/master/src in Firefox but I'm puzzling about how I can get it to work, again, in Waterfox.

Any suggestions?

(I already toyed with removal of cookies.)

TIA

Serve tar.gz from server

please in tar.gz and direct download not webpage loading

for use with wget and auto generate linux packaging

zip format is not compatibility for linux packaging

tank you

Use a smaller archiver

https://github.com/Stuk/jszip is huge (170KB) so maybe try https://github.com/beatgammit/tar-js?

Edit: I tried tar-js but it takes a loooot longer to build the archive.

Tested code

	const requests = await Promise.all(files.map(async path => {
		const response = await fetch(`https://raw.githubusercontent.com/${repo}/${branch}/${path}`);
		const blob = await response.arrayBuffer();

		downloaded++;
		updateStatus(files.length, downloaded);
		console.log('Downloaded:', path);

		return {path, blob};
	}));
	console.log('Downloaded', files.length, 'files');

	const tar = new Tar();
	for (const file of requests) {
		tar.append(file.path.replace(dir + '/', ''), new Uint8Array(file.blob));
	}

	saveFile(tar, `${repo} ${dir}.tar`.replace(/\//, '-'), () => {
		updateStatus(files.length, downloaded, true);
		console.log('Done!');
	});

Edit: maybe https://github.com/thejoshwolfe/yazl

Action Required: Fix Renovate Configuration

There is an error with this repository's Renovate configuration that needs to be fixed. As a precaution, Renovate will stop PRs until it is resolved.

Error type: Cannot find preset's package (github>fregante/.github:renovate-preset)

"No files to download" on large repos

Download using this link: https://github.com/phracker/MacOSX-SDKs/tree/master/MacOSX10.14.sdk

It prompts: No files to download

zipfile contains folder of parent level

I wanted to download folder waynes_world from https://github.com/j2labs/zmq_examples. The created zip-file also contained the complete folder waynes_world_2, which is not a content of folder waynes_world.

Git LFS support

https://github.com/jgarside1678/T9/blob/master/Content/Units/Alliance/Animations/MageAnimationSet/Animations/InPlace/Mage_Beam_2h_End_IP.uasset

uasset files are empty

Download of subfolder contains parent folder

When using this website to download the folder AikoMidori/SteamSkins/tree/main/Metro/Metro-JS/, the resulting zip has the structure of Metro/Metro-JS/, whereas I would expect it to just have Metro-JS in the root.

Is this intended behavior or a bug? If this is intended behavior, could there be some way to change this with an argument passed?

Allow anonymous usage

No need to ask user for token if repository is public.

Doesn't work for branches with slashes

It tries to GET https://api.github.com/repos/PaperMC/Paper/git/trees/feature?recursive=1 which returns 404 (Not Found), while I initiated the download from his link https://github.com/PaperMC/Paper/tree/feature/mcp/mcp

RangeError: offset is out of bounds

After outputting in the console "Zipping X files…"

If trows this exceptions and does not download anything:

UZIP.js:187 Uncaught (in promise) RangeError: offset is out of bounds
    at Uint8Array.set (<anonymous>)
    at f._writeHeader (UZIP.js:187:12)
    at f.encode (UZIP.js:136:12)
    at index.js:192:26

What is the token?

Why do I need a token for downloading from the public repos?

Handle API errors

404s are stuck on Retrieving directory info…

Failed to Load Resource: ERR_INSUFFICIENT_RESOURCES

The directory download fails for very large folders with error net::ERR_INSUFFICIENT_RESOURCES

File System Access API support

We can download directories without zipping using the File System Access API.

I'd like to implement this feature. 😆

Console errors on load

It still works, but I think the service worker is broken.

[Log]  (download-directory.github.io.9f7dde0d.js, line 31)
[Error] ReferenceError: Can't find variable: parcelRequire
	(anonymous function) (download-directory.github.io.9f7dde0d.js:1:1003)
	Module Code (download-directory.github.io.9f7dde0d.js:1:1024)
	evaluate
	moduleEvaluation
	(anonymous function)
	promiseReactionJob
[Info] Successfully preconnected to https://fonts.gstatic.com/
[Error] TypeError: Type error
	URL (download-directory.github.io.9f7dde0d.js:31:3505)
	(anonymous function) (download-directory.github.io.9f7dde0d.js:31:3505)
[Error] Failed to load resource: the server responded with a status of 404 () (favicon.ico, line 0)

Getting error "Some files were blocked from downloading, try to disable any ad blockers and refresh the page."

using example from the webpage e.g.

https://download-directory.github.io/?url=https://github.com/mrdoob/three.js/tree/dev/build

Or my own

https://download-directory.github.io/?url=https%3A%2F%2Fgithub.com%2FGMOD%2Fjbrowse-components%2Ftree%2Fmain%2Fdemos%2Fjbrowse-react-linear-genome-view-vite

error in private tab, no adblocker extensions enabled afaik

Can you **** the token?

I think it would be best to hide the token after putting one in

CRLF in files are being converted to LF only

Description:

When attempting to download a folder from the website, all text files with CRLF line endings are being automatically converted to LF, causing line-ending inconsistencies in the downloaded files.

Steps to Reproduce:

Download a folder using the web tool
Navigate to a folder containing text files with CRLF line endings.
Check the line endings.

Expected Behavior:

The downloaded files should retain their original CRLF line endings as they exist in the repository.

Actual Behavior:

After downloading, the line endings in text files are converted to LF, deviating from the original CRLF line endings.

Additional Information:

This issue is affecting all contributors who download folders on github repositories.
It impacts the consistency of line endings in text files, which can be problematic for cross-platform development.

Workarounds:

Manually converting line endings back to CRLF using a text editor like Notepad++ or Visual Studio Code with the "Change End Of Line Sequence" feature.

Possible Cause:

This issue might be related to Git configuration settings or Git clients auto-converting line endings on Windows systems.

Reproducibility:

This issue is reproducible consistently when downloading folders containing text files with CRLF line endings.

Support private repos

1. token instructions don't include private repos
2. if enabled, files are still fetched without token, so they 404

Cannot download when filename contains hash (#)

Is URI encode missing?

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

• Update all dependencies (actions/checkout, crazy-max/ghaction-github-pages, p-map, p-retry, xo)

Detected dependencies

github-actions

.github/workflows/ci.yml

• actions/checkout v3

.github/workflows/deployment.yml

• actions/checkout v3
• crazy-max/ghaction-github-pages v3

npm

package.json

• jszip ^3.10.0
• list-github-dir-content ^3.0.0
• p-map ^5.5.0
• p-retry ^5.1.1
• save-file ^2.3.1
• buffer ^6.0.3
• parcel ^2.6.2
• parcel-resolver-ignore ^2.1.3
• process ^0.11.10
• xo ^0.50.0

---

• Check this box to trigger a request for Renovate to run again on this repository

Can we re-use your code for our open-source community?

Hey @fregante, fantastic work!

I help manage an open-source community of over 1,300 active contributors with pretty large repos.

We'd love to give our community the ability to download only the subfolders they need, but we'd prefer to not overload your site with extra traffic.

I saw that your code is MIT-licensed, but I wanted to ask for your permission to use your code in our community website -- while making sure to give you credit, of course.

If so, do you have any recommendations on how to get started?

Add MIT License file

Great job on this repo! Would it be possible to add an MIT license file along with a link to it in the README? Here's an informative blog post: https://fossa.com/blog/open-source-licenses-101-mit-license/

Redirect to native zip URL when pasting a link to a branch, commit or tag

For example:

https://github.com/fregante/webext-domain-permission-toggle
https://github.com/fregante/webext-domain-permission-toggle/tree/main
https://github.com/fregante/webext-domain-permission-toggle/tree/v3.0.0
https://github.com/fregante/webext-domain-permission-toggle/tree/e125b40129e53342bc6ac07a1afc29696da347a4

Should just point to

https://github.com/fregante/webext-domain-permission-toggle/archive/refs/heads/main.zip
https://github.com/fregante/webext-domain-permission-toggle/archive/refs/tags/v3.0.0.zip
https://github.com/fregante/webext-domain-permission-toggle/archive/e125b40129e53342bc6ac07a1afc29696da347a4.zip

Silently fails if token is missing

Example: https://download-directory.github.io/?url=https://github.com/phracker/MacOSX-SDKs/tree/master/MacOSX10.1.5.sdk

It used to ask for a token but I think this was recently disabled. Now it just doesn't do anything 😞