GithubHelp home page GithubHelp logo

ddev-minio's Introduction

ddev-minio - use MinIO object storage in DDEV

GitHub release (with filter) Daily tests semantic-release: angular project is maintained

This repository provides MinIO add-on to DDEV.

It's based on MinIO official image and DDEV custom compose files

Configuration

MinIO bucket

Login to MinIO console https://<project>.ddev.site:9090 login with credentials ddevminio:ddevminio and create a bucket.

Composer package

Your project will most likely require the AWS PHP SDK. You can install it like this

$ ddev composer require aws/aws-sdk-php

Project docker instances can access MinIO api via http://minio:10101

Public access

DDEV Router is configured to proxy the requests to https://<project>.ddev.site:10101 to MinIO S3 Api.

Commands

Addon exposes the following commands

Command Usage Description
minio ddev minio Launches the MinIO Console
mc ddev mc Launches the MinIo admin client

Based on the original ddev-contrib recipe
Developed and maintained by Oblak Studio

ddev-minio's People

Contributors

renovate[bot] avatar seebeen avatar

Stargazers

 avatar  avatar  avatar

Watchers

 avatar  avatar

ddev-minio's Issues

Using https for the endpoint results in error

Hi, I am using the add-on with my shopware setup. If I use https for the endpoint, then I get the error messaged that Could not contact remote server. . If I use only http then everything works fine in terms of connectivity and uploading the files, but when rendering those files from the server, I get failed response in status.

the following is my shopware.yaml configuration flie:

shopware:
  filesystem:
    public:
      type: "amazon-s3"
      url: 'https://abc.ddev.site:9090/media'
      config:
        bucket: "media"
        endpoint: "http://minio:10101"
        use_path_style_endpoint: true
        region: 'local'
        credentials:
          key: ddevminio
          secret: ddevminio
        options:
          visibility: "public"
    theme:
      type: "amazon-s3"
      url: 'https://abc.ddev.site:9090/theme'
      config:
        bucket: "theme"
        endpoint: "http://minio:10101"
        use_path_style_endpoint: true
        region: 'local'
        credentials:
          key: ddevminio
          secret: ddevminio
        options:
          visibility: "public"
  cdn:
    strategy: id

The files are there on the server:

image

And when loading the file on the front also, I get 200 OK status, but it gets failed somehow: (failed)net::ERR_BLOCKED_BY_ORB

image

What could be the problem?

Thanks.

ddev minio post start /usr/bin/mc commands

Where can I add a post start command to create a minio bucket. This runs in the webserver container and fails.

hooks:
  post-start:
    - exec mc mb minio/files

Before I upgraded to the latest minio, I think I had t working in .ddev/docker-compose.minio-z.yaml services. I'm not sure where I got this code. I don't know this uses /bin/sh. I don't think I need the "anonymous" line, but it was something I tried while trying to figure out #18. It is named -z to so that it runs last; again, I'm not sure that is necessary, but I originally named it -custom and it seems to work when I renamed it -z.

  createbuckets:
    image: minio/mc
    depends_on:
      - minio
    entrypoint: >
      /bin/sh -c "
      /usr/bin/mc mb minio/files;
      /usr/bin/mc anonymous set public minio/files
      exit 0;
      "

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Pending Approval

These branches will be created by Renovate only once you click their checkbox below.

  • chore(deps): update actions/checkout action to v4
  • chore(deps): update crazy-max/ghaction-import-gpg action to v6
  • chore(deps): update dependency semantic-release to v22
  • ๐Ÿ” Create all pending approval PRs at once ๐Ÿ”

Awaiting Schedule

These updates are awaiting their schedule. Click on a checkbox to get an update now.

  • chore(deps): lock file maintenance

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

Ignored or Blocked

These are blocked by an existing closed PR and will not be recreated unless you click a checkbox below.

Detected dependencies

docker-compose
docker-compose.minio-norouter.yaml
docker-compose.minio-proxy.yaml
docker-compose.minio.yaml
dockerfile
.minioImageBuild/Dockerfile
github-actions
.github/workflows/cron_tests.yml
  • oblakstudio/public-workflows v1
  • actions/checkout v4
  • gautamkrishnar/keepalive-workflow v1
.github/workflows/pull_request.yml
  • oblakstudio/public-workflows v1
.github/workflows/release.yml
  • actions/checkout v3
  • crazy-max/ghaction-import-gpg v5
  • actions/setup-node v3
npm
package.json
  • @semantic-release/exec ^6.0.3
  • @semantic-release/git ^10.0.1
  • @semantic-release/github ^9.0.4
  • semantic-release ^21.0.7
  • yarn 3.6.1

  • Check this box to trigger a request for Renovate to run again on this repository

"ddev minio" redirects to "Unresponsive/broken ddev back-end site"

I cannot connect to the minio site. "ddev minio" brings up the URL in my browser but says "Unresponsive/broken ddev back-end site". Has something changed recently (in ddev or in ddev-minio)?

I am using a Drupal 7 site but I've tested this on an OOTB Drupal 10.1 site and get the same error. And just to isolate it further, I created an empty directory

ddev config && ddev get oblakstudio/ddev-minio && ddev minio

redirects to a browser that says:

502: Unresponsive/broken ddev back-end site.

This is the ddev-router container: The back-end webserver at the URL you specified is not responding. You may want to use "ddev restart" to restart the site.

Using `ports` means that only one project can use this without modification

Congrats on the new add-on! Thanks for maintaining this.

I notice that you use ports in the docker-compose.*.yaml, that means this will only be able to be used on one project without modification. There are better ways. You already have things exposed, so you shouldn't need ports for any HTTP port. You would use HTTP_EXPOSE and HTTPS_EXPOSE, but you already have those in play. What is port 9000 for anyway?

(Also note that port 9000 often conflicts with other usages, like php-fpm, so it's usually a prescription for trouble to use 9000 for anything)

GetObject always fails from API

Aws\S3\StreamWrapper::openReadStream() always throws an Exception

Error executing "GetObject" on "https://MYSITE.com:10101/files/s3fs-public/somefile.jpg"; AWS HTTP error: An error was encountered while creating the response. 

I uploaded files using the Stream Wrapper. I can see the files from the console and web interface. I can download the file with the web interface.

Do you have any suggestions?

mc-config.json doesn't work

As far as I can tell, the mc-config.json file is not taking effect as expected.

ddev get oblakstudio/ddev-minio
ddev restart
ddev mc alias ls

Outputs: (Notice the warning about tmp configuration, and that there is no minio key)

mc: Configuration written to `/tmp/.mc/config.json`. Please update your access credentials.
mc: Successfully created `/tmp/.mc/share`.
mc: Initialized share uploads `/tmp/.mc/share/uploads.json` file.
mc: Initialized share downloads `/tmp/.mc/share/downloads.json` file.
gcs
  URL       : https://storage.googleapis.com
  AccessKey : YOUR-ACCESS-KEY-HERE
  SecretKey : YOUR-SECRET-KEY-HERE
  API       : S3v2
  Path      : dns

local
  URL       : http://localhost:9000
  AccessKey :
  SecretKey :
  API       :
  Path      : auto

play
  URL       : https://play.min.io
  AccessKey : Q3AM3UQ867SPQQA43P2F
  SecretKey : zuf+tfteSlswRu7BJ86wekitnifILbZam1KYY3TG
  API       : S3v4
  Path      : auto

s3
  URL       : https://s3.amazonaws.com
  AccessKey : YOUR-ACCESS-KEY-HERE
  SecretKey : YOUR-SECRET-KEY-HERE
  API       : S3v4
  Path      : dns

If you create it manually:

ddev mc alias set minio  http://localhost:10101 ddevminio ddevminio
ddev mc alias ls

It will be listed as expected, and you can go on to use it (eg mc mb minio/my-bucket).
As expected, the alias and bucket will go away with a ddev restart, as they are in tmp.

It also may make sense to make the alias local, as this seems to be the default?

Project fails to start on ddev v1.23 with minio installed

On the latest ddev 1.23 I am getting this error when starting the project with minio installed:

Failed to restart sw6: docker-compose build failed: composeCmd failed to run 'COMPOSE_PROJECT_NAME=ddev-sw6 docker-compose -f /home/huzaifa/code/sw6/.ddev/.ddev-docker-compose-full.yaml --progress=plain build', action='[--progress=plain build]', err='exit status 15', stdout='', stderr='2 error(s) decoding:

* error decoding 'services[minio].deploy.resources.limits.cpus': unexpected value type int for cpus
* error decoding 'services[minio].deploy.resources.reservations.cpus': unexpected value type int for cpus', output='', stderr='2 error(s) decoding:

* error decoding 'services[minio].deploy.resources.limits.cpus': unexpected value type int for cpus
* error decoding 'services[minio].deploy.resources.reservations.cpus': unexpected value type int for cpus' 

File does not load even though the status is 200

Hi, the https: issue is resolved thank you for that, but I am still getting issue when loading the file. Seems like a CORS issue as I get this status in the network tab: (failed)net::ERR_BLOCKED_BY_ORB

Here is my example request:

huzaifa@huzaifa-hp:~/code/sw6$ ddev exec curl -I -H "Origin: https://sw6.ddev.site" -X GET https://sw6.ddev.site:9090/theme/theme/cf699bcc55c87558a9fd64769093a26a/js/all.js
HTTP/2 200 
age: 0
cache-control: no-cache, private
content-type: text/html; charset=UTF-8
date: Tue, 23 Apr 2024 05:45:58 GMT
referrer-policy: strict-origin-when-cross-origin
server: Apache/2.4.57 (Debian)
set-cookie: session-=60cdf036df37198179086ce6a434886b; path=/; secure; httponly; samesite=lax
strict-transport-security: max-age=31536000; includeSubDomains
vary: Accept-Encoding
x-content-type-options: nosniff
x-debug-token: f3d9d0
x-debug-token-link: https://sw6.ddev.site/_profiler/f3d9d0
x-frame-options: deny
x-robots-tag: noindex
x-symfony-cache: GET /: miss, store

HTTP/2 200 
accept-ranges: bytes
content-security-policy: default-src 'self' 'unsafe-eval' 'unsafe-inline';
content-type: text/javascript
date: Tue, 23 Apr 2024 05:45:58 GMT
last-modified: Tue, 23 Apr 2024 05:45:58 GMT
referrer-policy: strict-origin-when-cross-origin
server: MinIO Console
x-content-type-options: nosniff
x-frame-options: DENY
x-xss-protection: 1; mode=block
content-length: 1310

Upon debugging a bit, it seems related to missing headers. I think the addon-on is missing the headers for fetching cross origin files.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.