qoomon / aws-s3-bucket-browser

Single page application to browse AWS S3 bucket content

AWS S3 Bucket Browser

Single HTML file to browse AWS S3 buckets

Demo

Features

• Compatible Providers:
• List all files in a table view
• Treat and display / in keys as folders
• Render preview for Makrdown files
• Show Install button for manifest.plist on iOS devices

Installation

Self-Hosted

• Just download index.html and upload it to your bucket.
  • Adjust config within index.html if needed, e.g.

    const config = {
      title: 'Bucket Browser', // prefix value with `HTML> ` to render as html, see subtitle
      subtitle: 'HTML>made with ♥ by <b><a href="https://qoo.monster">qoomon</a></b>', // prefix value with `HTML> ` to render as html
      logo: 'https://qoomon.github.io/aws-s3-bucket-browser/logo.png',
      favicon: 'https://qoomon.github.io/aws-s3-bucket-browser/favicon.ico',
      primaryColor: '#167df0',
      
      bucketUrl: undefined,
      // If bucketUrl is undefined, this script tries to determine bucket Rest API URL from this file location itself.
      //   This will only work for locations like these
      //   * https://s3.BUCKET-REGION.amazonaws.com/BUCKET-NAME/index.html
      //   * http://BUCKET-NAME.s3-website-BUCKET-REGION.amazonaws.com/index.html
      //   * https://storage.googleapis.com/BUCKET-NAME/index.html
      //   * https://BUCKET-NAME.s3-web.BUCKET-REGION.cloud-object-storage.appdomain.cloud/
      //   * https://BUCKET-NAME.BUCKET-REGION.digitaloceanspaces.com
      //   * https://BUCKET-NAME.BUCKET-REGION.cdn.digitaloceanspaces.com
      // If bucketUrl is set manually, ensure this is the bucket Rest API URL, e.g.
      //   * https://s3.BUCKET-REGION.amazonaws.com/BUCKET-NAME
      //   * https://storage.googleapis.com/BUCKET-NAME
      //   The URL should return an XML document with <ListBucketResult> as root element.
      rootPrefix: undefined, // e.g. 'subfolder/'
      keyExcludePatterns: [/^index\.html$/], // matches againt object key relative to rootPrefix
      pageSize: 50,
      
      bucketMaskUrl: undefined, 
      // If bucketMaskUrl is set file urls will be changed from ${bucketUrl}/${file} to ${bucketMaskUrl}/${file}
      //   bucketMaskUrl: 'https://example.org'
      //     => https://example.org/foo/bar.txt 
      //   bucketMaskUrl: document.location.origin
      //     => ${document.location.origin}/foo/bar.txt 
      
      defaultOrder: 'name-asc' // (name|size|dateModified)-(asc|desc)
    }

• ⚠️ Ensure Bucket Permissions

  • Go to https://s3.console.aws.amazon.com/s3/buckets/<YOUR BUCKET NAME>/?tab=permissions
  • Grant public read permission by Access Control List or Bucket Policy
    • Access Control List
      • Enable List objects for Everyone
    • Bucket Policy

      {
          "Version": "2012-10-17",
          "Statement": [
              {
                  "Sid": "PublicRead",
                  "Principal": "*",
                  "Effect": "Allow",
                  "Action": [
                      "s3:ListBucket",
                      "s3:GetObject"
                  ],
                  "Resource": [
                      "arn:aws:s3:::<YOUR BUCKET NAME>",
                      "arn:aws:s3:::<YOUR BUCKET NAME>/*"
                  ]
              }
          ]
      }

• ⚠️ Ensure Bucket CORS

  • Depending on your setup you may need need to ensure following CORS Configuration
  • Go to https://s3.console.aws.amazon.com/s3/buckets/<YOUR BUCKET NAME>/?tab=permissions
  • Grant Cross Origin Access by following CORS Configuration, replace http://www.example.com by your address of bucket browser index.html
    • e.g http://example.s3-website-eu-central-1.amazonaws.com/index.html

    [
      {
          "AllowedHeaders": [
              "*"
          ],
          "AllowedMethods": [
              "GET"
          ],
          "AllowedOrigins": [
              "http://www.example.com"
          ],
          "ExposeHeaders": [
              "x-amz-server-side-encryption",
              "x-amz-request-id",
              "x-amz-id-2"
          ],
          "MaxAgeSeconds": 3000
      }
    ]

• Open <YOUR BUCKET URL>/index.html in your browser.

Hosted

• ⚠️ Ensure Bucket Permissions

  • see Self-Hosted

• ⚠️ Ensure Bucket CORS

  • see Self-Hosted
• Open hosted index.html in your browser and provide bucket url as bucket request parameter
  • ${INDEX_FILE_LOCATION}?bucket=${S3_BUCKET_URL}
  • e.g. https://qoomon.github.io/aws-s3-bucket-browser/index.html?bucket=https://s3.eu-west-1.amazonaws.com/data.openspending.org

CloudFront Setup

If you use CloudFront in upfront of your S3 bucket ensure following CloudFront settings.

• Allowed/Cached HTTP Methods: GET, HEAD, OPTIONS
• Cached Based on Selected Headers: Whitelist
  • Access-Control-Request-Headers
  • Access-Control-Request-Method
  • Origin
• Query String Forwarding and Caching: Forward all

IBM Cloud Object Storage Setup

IBM Cloud Object storage only supports virtual host-style addressing, i.e. https://<bucket-name>s3-web.<region>.cloud-object-storage.appdomain.cloud/ for static website hosting. Otherwise follow the instructions in this tutorial to configure your bucket. In addition, you may need to configure CORS for your bucket.

<CORSConfiguration>
  <CORSRule>
      <AllowedOrigin>*</AllowedOrigin>
      <AllowedMethod>GET</AllowedMethod>
      <AllowedHeader>*</AllowedHeader>
  </CORSRule>
</CORSConfiguration>

aws-s3-bucket-browser's Issues

license?

what type of license does this repo use?

can it be used for commercial use?

Page shows but no listing and no errors in the console

I have a bucket with a sub folder, I've specified the bucketUrl as:

http://bucketname.s3.amazonaws.com/?prefix=subfolder/index.html

The S3 Bucket Browser page loads but nothing is listed. There are no errors in the console. If I go to:

https://bucketname.s3.amazonaws.com/?prefix=subfolder
I do get a ListBucketResult with the contents shown.

I've tried other combinations but can't get the page to load or I get no listing. Any suggestions?

Feature request - specify default ordering

In the config section ability to choose which column to order on and ordering direction so this is presented upon loading index.html without having to click on the column to obtain desired ordering.

index.html isn't being excluded

Using the current index.html, the keyExcludePatterns: [/^index.html$/] doesn't appear to be working. I'm getting the index.html in the listing of objects.

when in click in index.html inside a subfolder the file is downloaded instead of just serving it

first of all: awesome job, love what you have done here.

the way i use it is to show a list of folder where each folder have various reports from a build process (karma tests, junit, coverage..etc). navigation in and out is great, no problems there

everything works great until i get inside one of the leaf folder that has the html report

here's one example so you can see what i'm talking about

when i click on index.html the file is downloaded...but..if i right click, copy the link and paste that into a different tab the file is served and i can see the rendered html.

i'm purely guessing this is a content type problem...any ideas how to fix it?

bucket directory listing is not working

in index.html file after replaced the following line also

bucketUrl: 'http://bucket-name.s3-website-eu-west-1.amazonaws.com'

please help on where i am missing it.

not working url

http://cds.p6v7v3q9.hwcdn.net/

rootPrefix folder is visible on every page. It should be hidden.

rootPrefix folder is visible on every screen on the top. Need to hide it on every page.

It confuses users and they treats it as subfolder. When we click on it nothing shows up.

Header name is `Access-Control-Request-Method`

You used the plural (Access-Control-Request-Methods) in README.md.

Support for Virtual Style Paths on Open Hosted Implementation over HTTPS

First, thanks for developing this project. I spent hours searching for something that does what yours does, and it seems to be the only currently working solution. The slick implementation only increases my awe. 🥇

I do have an edge concern: Amazon will be deprecating path-style requests come this October, which seems to at least partially break the open hosted implementation for your tool.

Their preferred request format, Virtual Hosted-Style, botches support for HTTPS. Wildcard SSL certs can only extend up one subdomain level, which disallows support for multisegmented bucket names (such as alpha.bravo.s3.us-east-1.amazonaws.com). The warning can be bypassed on internally hosted (by adding an exception), but open hosted throws a CORS error (which I believe to be symptomatic of the SSL certificate mismatch).

Open hosted implementation also can't bypass by passing an HTTP request, as that will trigger a mixed content block that can't be overridden.

From previous attempts to find a solution, I found that tunneling through CloudFront (AWS CDN solution) restores the ability to request via HTTPS, with seemingly the same XML file returned. However, when parsed the folders disappear, only leaving the top level files displayed.

I've linked examples utilizing your demo to show the difference. If you can shed any light on what's happening, that would be awesome.

Path-Style Request (working properly)

Virtual Hosted-Style (missing all directories)

Apologies in advance for this novel of an issue report, and thanks in advance for your consideration.

Add search bar at the top which can help users to search files quickly

Could you add search bar at the top which can help users to search files quickly ?

ListBucketResponse error rendering on index.html

I'm trying to implement your bucket viewer, but am hitting the error at https://github.com/qoomon/aws-s3-bucket-browser/blob/master/index.html#L313. Do you have any insights as to what I might be doing wrong? My index page is at https://mothur.s3.us-east-2.amazonaws.com/index.html.

first time visit does not browse into given path prefix provided in url

When entering an url like https://s3-bukcet.eu-west-1.amazonaws.com/#myFolder/second-one/ it shows the home folder https://s3-bukcet.eu-west-1.amazonaws.com/# instead of the contents of https://s3-bukcet.eu-west-1.amazonaws.com/#myFolder/second-one/ .

Here is the patch: just adding a new line and changing another:

index.html.patch.txt

Thanks for the tool!

Feature Request - Render Markdown File(s)

Hello, @qoomon! Great job making this script!

I've forked your repository and added my personal setup to list all of my bucket files on my S3-compatible storage. So far, everything work as expected.

But I have an idea: Rendering markdown on the site.

For example, we have README.md. We could render it after the file list. It is very useful for people that wants viewable documentation.

I found a script related to this:

• https://zerodevx.github.io/zero-md/

Based on this StackOverflow answer: https://stackoverflow.com/a/60363693/13293193

Thanks!

Feature Request: Search Behavior

First, I really love this interface. Nice clean and functional. I am noticing with the search however that it does not find directories (perhaps files as well) when you search for a word or words, like a keyword or phrase, that is in the center or end of the directory. It only will return the results of the search if you get the spelling/capitalization exactly correct from left to right. For example:

First Word_Second Word_Third/

if you search for Word here you do not get results. If you search for Third you do not get results. If you search any word after "First" you do not get results. You only get results if the words are typed correctly in order from left to right.

I am requesting a more fluid search behavior that returns results based on the word(s) regardless of capitalization and location in a name if that makes sense.

%2F delimeters in href

I'm getting %2F delimeters generated in the path to the file, like:

root%2Fanotherfolder%2Ffile.jpeg instead of
root/anotherfolder/file.jpeg

This is making the browser save the file as "root_anotherfolder_file.jpeg" filename instead of "file.jpeg" filename.

Can this be controlled somewhere? I saw another site with an older version, where it's as what i'm trying to get, but seems latest version is different? or am i missing something?

A way to change {bucket].s3-website.us-east-1. to just the site/domain we are hosting?

This might be out of scope of this project, but i was wondering if there was a way to access the index by going to say...
dk.toastednet.org/CM400A instead of http://dk.toastednet.org.s3-website-us-east-1.amazonaws.com/CM400A/#

Would that be a DNS record of some kind?

Failed to fetch error is coming

I have downloaded index.html and placed in my bucket.

My Bucket policy

{
    "Version": "2008-10-17",
    "Statement": [
        {
            "Sid": "PublicRead",
            "Effect": "Allow",
            "Principal": "*",
            "Action": [
                "s3:GetObject",
                "s3:ListBucket"
            ],
            "Resource": [
                "arn:aws:s3:::cnl-test-poc/*",
                "arn:aws:s3:::cnl-test-poc"
            ]
        }
    ]
}

CORS POLICY

[
    {
        "AllowedHeaders": [
            "*"
        ],
        "AllowedMethods": [
            "GET"
        ],
        "AllowedOrigins": [
            "http://cnl-test-poc.s3-website-us-east-1.amazonaws.com/"
        ],
        "ExposeHeaders": [
            "x-amz-server-side-encryption",
            "x-amz-request-id",
            "x-amz-id-2"
        ],
        "MaxAgeSeconds": 3000
    }
]

my s3 bucket url is

http://cnl-test-poc.s3-website-us-east-1.amazonaws.com/

if i access ,i am getting failed to fetch error.Help on this

Feature Request: Compatibility with Cloudflare R2

Cloudflare R2 is a great object storage platform and utilizes the same S3 API. Would be great to see this functionality added!

Directory structure not being recognized from XML keys

Hello, first of all thank you for creating this incredible tool! I'm running into a bit of an issue when my S3 bucket objects are being listed and hopefully you'll be able to help me.

I'm running a private S3 origin bucket for Cloudfront, and I can go to my designated URL and get the XML layout of all of the objects within, which should include directories such as 'HCU' and 'OWASPJuiceShop'. Note that some aspects were blurred for privacy reasons:

However, when I check the index.html page to see the reflected contents, this directory structure appears to be missing:

All I've modified in the example index.html you provide is my bucket URL (https://zap-results.com), and my search through the rest of the style changes doesn't seem to indicate that anything would remove these directory headings - any help would be greatly appreciated!

Question: ListObjects / ListObjectsV2

Hi qoomon,

Sorry for contacting you this way, but maybe you have more experience with this I hope. Was trying your Bucket Browser and got the Failed to fetch because or CORS, that's fine because that's only in the browser.

I was trying to implement something related to Amazon S3 and got stuck.
KoalaBear84/OpenDirectoryDownloader#132

The issue is that maybe the marker parameter isn't supported on some hosts?

If you have any experience or info about this, thanks!

Doesn't work with mobile versions of browsers

Getting a blank page on Chrome for Android, we are trying to use this to host APK builds for sideloading on to test devices from our object storage

Is this supposed to work on mobile devices?

CloudFront support

Hi and thank you for this project.

I have set up it on S3 and it works.
But I want to use CloudFront. I have configured CF as you discribe it in README.md.

When I open CF url it shows me an error: List bucket response does not contain <ListBucketResult> tag!

According to message in footer its tries to use CF url instead of real bucket url.

And the same situation when I use my domain from route 53.

Maybe I did somting wrong..

Improved directory listing experience with CloudFront Functions

In a self-hosted scenario, it is a bit off that one has to expose the S3 website (to host index.html) and S3 REST API endpoints (to get the XML listing), then configure CORS rules accordingly. A much cleaner solution can be accomplished if everything is behind CloudFront (this is also the only way to use your own domain with valid TLS certificates)...

You only need to expose the S3 REST API (disable website mode) to your CloudFront distribution (using CloudFront Origin Access Controls), upload a correctly configured AWS S3 Bucket Browser into /s3-browser/, and use the following CloudFront Function to rewrite a few URLs. To support directory index listing I also modified to get the pathPrefix from "document.location.pathname" as a fallback.

-      bucketUrl: undefined,
-      keyExcludePatterns: [/^index\.html$/, /^Urban_3D_Challenge\//], // matches againt object key relative to rootPrefix
-      bucketMaskUrl: undefined,
+      bucketUrl: '/list-objects',
+      keyExcludePatterns: [/^s3-browser\//],
+      bucketMaskUrl: '/',

-        window.onpopstate = () => this.pathPrefix = decodeURIComponent(window.location.hash).replace(/^#/, '')
+        window.onpopstate = () => this.pathPrefix = decodeURIComponent(window.location.hash).replace(/^#/, '') || decodeURIComponent(window.location.pathname).replace(/^\//, '')

/**
 * Rewrite root and directory index to an S3 bucket browser web app using AWS CloudFront Functions.
 */

var s3ListingUri = "/";
var s3BrowserUri = "/s3-browser/index.html";

function handler(event) {
  var request = event.request;

  if (request.uri == '/list-objects') {
    // Request for XML listing points to the S3 root listing endpoint
    //   https://bucket.domain/list-objects
    request.uri = s3ListingUri;

  } else if (request.uri.endsWith('/')) {
    // Request for root or directory index points to the S3 browser
    //   https://bucket.domain/
    //   https://bucket.domain/my/path/
    request.uri = s3BrowserUri;
  }

  return request;
}

Does keyExcludePatterns exclude folders?

Awesome browser! Way nicer than what I seen out there.

I have been playing with the keyExcludePatterns and I can't seem to exclude folders. Is it supported?

Private Bucket support

I would like to thank for your amazing tool and I found it very useful.I request your inputs on how can we implement this for private buckets?

Issues with listing subdirectories

I'm hosting a static site on S3. The root of the bucket has an index.html file, but there are a few subdirectories where a listing is needed. I've tried a smattering of things but none seem to work. I get the error

Bucket URL http://dk.toastednet.org.s3-website-us-east-1.amazonaws.com/CM400A/ is not a valid bucket API URL, response does not contain <ListBucketResult><Delimiter> tag.

Can you take a look at it for me and tell me what's wrong? I have several other directories I would like to list so I can apply what I learn here to those :)

 `bucketUrl: undefined,
  // If bucketUrl is undefined, this script tries to determine bucket Rest API URL from this file location itself.
  //   This will only work for locations like these
  //   * https://s3.BUCKET-REGION.amazonaws.com/BUCKET-NAME/index.html
  //   * http://BUCKET-NAME.s3-website-BUCKET-REGION.amazonaws.com/index.html
  //   * https://storage.googleapis.com/BUCKET-NAME/index.html
  //   * https://BUCKET-NAME.s3-web.BUCKET-REGION.cloud-object-storage.appdomain.cloud/
  // If bucketUrl is set manually, ensure this is the bucket Rest API URL, e.g.
  //   * https://s3.BUCKET-REGION.amazonaws.com/BUCKET-NAME
  //   * https://storage.googleapis.com/BUCKET-NAME
  //   The URL should return an XML document with <ListBucketResult> as root element.
  rootPrefix: 'CM400A/', // e.g. 'subfolder/'
  keyExcludePatterns: [/^index\.html$/],
  pageSize: 50,

  bucketMaskUrl: undefined,
  // If bucketMaskUrl is set file urls will be changed from ${bucketUrl}/${file} to ${bucketMaskUrl}/${file}
  //   bucketMaskUrl: 'https://example.org'
  //     => https://example.org/foo/bar.txt
  //   bucketMaskUrl: document.location.origin
  //     => ${document.location.origin}/foo/bar.txt

  defaultOrder: 'name-asc' // (name|size|dateModified)-(asc|desc)
}`

Thank you!

Restrict permissions

Hi,
Amazing project. Thanks!

I am trying to figure out how to add a little layer of security not allowing public access but just limit the view to my hosted html.

Not sure how to achieve this.
I am thinking if we can limit the permissions so it will only work form my IP (where the hosted file is) or something like that.

Do you have any suggestions on how to do this?

Content downloaded instead of serving

Just as the first issue on this repo, I would like to serve certain files without a file extension but with the right mimetype, instead of downloading it when clicked on.

#1

The lines from the original issue do not exist anymore, how would one go about it now?

Very good job, I also use it on aliyun oss bucket

List Bucket response does not contain <ListBucketResult> tag!

Hi,

I have set up permissions according to the instructions.

When accessing the page I get the following error:
"List Bucket response does not contain ListBucketResult tag!"

Any hints?

Bucket has to be public in order to work with s3-bucket-brower ?

Bucket has to be public in order to work with s3-bucket-brower ?

I have added CORS Configuration and bucket policy. But anyone can download files within the bucket programmatically.

Use Case: Implemented s3-static-website + cloud-front with labda@Edge + aws congnito.
cloud-front url is authenticated with cognito to show the static-website.

Even though we have enable authentication at browser level for static website, files can be accessed programmatically from anywhere.

Could you please suggest a way how we can restrict the same ?

Content Security Policy: The page’s settings blocked the loading of a resource at inline (“script-src”).

I have configured s3 Static website + CloudFront with lambda@Edge + aws cognito.
When I open Cloudfront URL after authenticating with cognito it shows above errors that it is not able to load the page due to security issues.

Link to refer: https://medium.com/@saurishkar/setting-up-aws-http-authentication-on-cloudfront-s3-using-cognito-and-lambda-edge-166ee38d471e

added below line to HTML but still gives an error.

<meta http-equiv="Content-Security-Policy" content="default-src *; style-src 'self' 'unsafe-inline'; script-src 'self' 'unsafe-inline' 'unsafe-eval' * ">

Any help appreciated.

Google cloud storage support?

This project is super cool!

Is there a way you can add support for google cloud storage? It also returns a xml same as s3 but idk how to configure this to work with Google storage

Direct link to "folders"

The ability to link to "folders" would be nice.

Currently when I link to https://public.overte.org/index.html#build/overte/ it just shows the root of the bucket instead of /build/overte/

Files containing a "+" in their name are not linked properly

Hello @qoomon
First, thanks for the great script !

I've found a small issue when one of the listed files contains a "+" in the name.
When that happens, the download link will contain the "+" without escaping, which produces a 404 error if one clicks on the file.
If I replace the "+" with a "%2B", then the download link works.

So I guess it just lacks some logic to escape special characters in filenames to make it work in all cases.

Thanks !

Previously working sites have stopped working with CORS errors

A couple of sites that I've set up previously and were working last week are no longer showing the listing. Even a demo site mentioned in a previous issue isn't working: https://data.payless.health/

Failed to load resource: net::ERR_BLOCKED_BY_CLIENT
data.payless.health/:1 Access to script at 'https://unpkg.com/[email protected]/dist/buefy.min.js' from origin 'https://data.payless.health' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource.
buefy.min.js:1          Failed to load resource: net::ERR_FAILED
(index):364 Bucket REST API: https://s3-external-1.amazonaws.com/payless.health/
(index):375 Uncaught ReferenceError: Buefy is not defined
    at (index):375:13
DevTools failed to load source map: Could not load content for https://unpkg.com/[email protected]/min/moment.min.js.map: HTTP error: status code 520, net::ERR_HTTP_RESPONSE_CODE_FAILURE

issue with ssl

hi,

this works great with http://mydomain...

but returns failed to fetch with https://mydomain...

thanks.

Bucket URL https://bucket/index.html is not a valid bucket API URL, response does not contain <ListBucketResult><Delimiter> tag.

Bucket URL https://bucket/index.html is not a valid bucket API URL, response does not contain tag.

I only modified bucketUrl and rootPrefix in index.html.
made bucket as public.
added Bucket policy and CORS.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "PublicRead",
            "Effect": "Allow",
            "Principal": "*",
            "Action": [
                "s3:ListBucket",
                "s3:GetObject"
            ],
            "Resource": [
                "arn:aws:s3:::bucket",
                "arn:aws:s3:::bucket/*"
            ]
        }
    ]
}

[
    {
        "AllowedHeaders": [
            "*"
        ],
        "AllowedMethods": [
            "GET"
        ],
        "AllowedOrigins": [
            "*"
        ],
        "ExposeHeaders": [
            "x-amz-server-side-encryption",
            "x-amz-request-id",
            "x-amz-id-2"
        ],
        "MaxAgeSeconds": 3000
    }
]

Any help appreciated.