oblakstudio / ddev-minio

MinIO addon for DDEV

ddev-minio - use MinIO object storage in DDEV

This repository provides MinIO add-on to DDEV.

It's based on MinIO official image and DDEV custom compose files

Configuration

MinIO bucket

Login to MinIO console https://<project>.ddev.site:9090 login with credentials ddevminio:ddevminio and create a bucket.

Composer package

Your project will most likely require the AWS PHP SDK. You can install it like this

$ ddev composer require aws/aws-sdk-php

Project docker instances can access MinIO api via http://minio:10101

Public access

DDEV Router is configured to proxy the requests to https://<project>.ddev.site:10101 to MinIO S3 Api.

Commands

Addon exposes the following commands

Command	Usage	Description
minio	ddev minio	Launches the MinIO Console
mc	ddev mc	Launches the MinIo admin client

---

Based on the original ddev-contrib recipe
Developed and maintained by Oblak Studio

ddev-minio's Issues

Using `ports` means that only one project can use this without modification

Congrats on the new add-on! Thanks for maintaining this.

I notice that you use ports in the docker-compose.*.yaml, that means this will only be able to be used on one project without modification. There are better ways. You already have things exposed, so you shouldn't need ports for any HTTP port. You would use HTTP_EXPOSE and HTTPS_EXPOSE, but you already have those in play. What is port 9000 for anyway?

(Also note that port 9000 often conflicts with other usages, like php-fpm, so it's usually a prescription for trouble to use 9000 for anything)

ddev minio post start /usr/bin/mc commands

Where can I add a post start command to create a minio bucket. This runs in the webserver container and fails.

hooks:
  post-start:
    - exec mc mb minio/files

Before I upgraded to the latest minio, I think I had t working in .ddev/docker-compose.minio-z.yaml services. I'm not sure where I got this code. I don't know this uses /bin/sh. I don't think I need the "anonymous" line, but it was something I tried while trying to figure out #18. It is named -z to so that it runs last; again, I'm not sure that is necessary, but I originally named it -custom and it seems to work when I renamed it -z.

  createbuckets:
    image: minio/mc
    depends_on:
      - minio
    entrypoint: >
      /bin/sh -c "
      /usr/bin/mc mb minio/files;
      /usr/bin/mc anonymous set public minio/files
      exit 0;
      "

"ddev minio" redirects to "Unresponsive/broken ddev back-end site"

I cannot connect to the minio site. "ddev minio" brings up the URL in my browser but says "Unresponsive/broken ddev back-end site". Has something changed recently (in ddev or in ddev-minio)?

I am using a Drupal 7 site but I've tested this on an OOTB Drupal 10.1 site and get the same error. And just to isolate it further, I created an empty directory

ddev config && ddev get oblakstudio/ddev-minio && ddev minio

redirects to a browser that says:

502: Unresponsive/broken ddev back-end site.

This is the ddev-router container: The back-end webserver at the URL you specified is not responding. You may want to use "ddev restart" to restart the site.

Using https for the endpoint results in error

Hi, I am using the add-on with my shopware setup. If I use https for the endpoint, then I get the error messaged that Could not contact remote server. . If I use only http then everything works fine in terms of connectivity and uploading the files, but when rendering those files from the server, I get failed response in status.

the following is my shopware.yaml configuration flie:

shopware:
  filesystem:
    public:
      type: "amazon-s3"
      url: 'https://abc.ddev.site:9090/media'
      config:
        bucket: "media"
        endpoint: "http://minio:10101"
        use_path_style_endpoint: true
        region: 'local'
        credentials:
          key: ddevminio
          secret: ddevminio
        options:
          visibility: "public"
    theme:
      type: "amazon-s3"
      url: 'https://abc.ddev.site:9090/theme'
      config:
        bucket: "theme"
        endpoint: "http://minio:10101"
        use_path_style_endpoint: true
        region: 'local'
        credentials:
          key: ddevminio
          secret: ddevminio
        options:
          visibility: "public"
  cdn:
    strategy: id

The files are there on the server:

And when loading the file on the front also, I get 200 OK status, but it gets failed somehow: (failed)net::ERR_BLOCKED_BY_ORB

What could be the problem?

Thanks.

Make addon official

Hello @rfay.

Can we make this addon official? All kinks and bugs ironed out.

GetObject always fails from API

Aws\S3\StreamWrapper::openReadStream() always throws an Exception

Error executing "GetObject" on "https://MYSITE.com:10101/files/s3fs-public/somefile.jpg"; AWS HTTP error: An error was encountered while creating the response. 

I uploaded files using the Stream Wrapper. I can see the files from the console and web interface. I can download the file with the web interface.

Do you have any suggestions?

With the new version I have a conflict on port 9000

Failed to restart my mylocal: Unable to listen on required ports, port 9000 is already in use,
Troubleshooting suggestions at https://ddev.readthedocs.io/en/stable/users/basics/troubleshooting/#unable-listen

Please get tests going, and nightly tests

For these add-ons to be successful, some decent automated tests (and run nightly) are critical. Please enable them and make sure the tests are good.

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Pending Approval

These branches will be created by Renovate only once you click their checkbox below.

• chore(deps): update actions/checkout action to v4
• chore(deps): update crazy-max/ghaction-import-gpg action to v6
• chore(deps): update dependency semantic-release to v22
• 🔐 Create all pending approval PRs at once 🔐

Awaiting Schedule

These updates are awaiting their schedule. Click on a checkbox to get an update now.

• chore(deps): lock file maintenance

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

• chore(deps): update dependency @semantic-release/github to v9.2.1

Ignored or Blocked

These are blocked by an existing closed PR and will not be recreated unless you click a checkbox below.

• chore(deps): update yarn to v3.6.3

Detected dependencies

docker-compose

docker-compose.minio-norouter.yaml

docker-compose.minio-proxy.yaml

docker-compose.minio.yaml

dockerfile

.minioImageBuild/Dockerfile

github-actions

.github/workflows/cron_tests.yml

• oblakstudio/public-workflows v1
• actions/checkout v4
• gautamkrishnar/keepalive-workflow v1

.github/workflows/pull_request.yml

• oblakstudio/public-workflows v1

.github/workflows/release.yml

• actions/checkout v3
• crazy-max/ghaction-import-gpg v5
• actions/setup-node v3

npm

package.json

• @semantic-release/exec ^6.0.3
• @semantic-release/git ^10.0.1
• @semantic-release/github ^9.0.4
• semantic-release ^21.0.7
• yarn 3.6.1

---

• Check this box to trigger a request for Renovate to run again on this repository

Project fails to start on ddev v1.23 with minio installed

On the latest ddev 1.23 I am getting this error when starting the project with minio installed:

Failed to restart sw6: docker-compose build failed: composeCmd failed to run 'COMPOSE_PROJECT_NAME=ddev-sw6 docker-compose -f /home/huzaifa/code/sw6/.ddev/.ddev-docker-compose-full.yaml --progress=plain build', action='[--progress=plain build]', err='exit status 15', stdout='', stderr='2 error(s) decoding:

* error decoding 'services[minio].deploy.resources.limits.cpus': unexpected value type int for cpus
* error decoding 'services[minio].deploy.resources.reservations.cpus': unexpected value type int for cpus', output='', stderr='2 error(s) decoding:

* error decoding 'services[minio].deploy.resources.limits.cpus': unexpected value type int for cpus
* error decoding 'services[minio].deploy.resources.reservations.cpus': unexpected value type int for cpus' 

File does not load even though the status is 200

Hi, the https: issue is resolved thank you for that, but I am still getting issue when loading the file. Seems like a CORS issue as I get this status in the network tab: (failed)net::ERR_BLOCKED_BY_ORB

Here is my example request:

huzaifa@huzaifa-hp:~/code/sw6$ ddev exec curl -I -H "Origin: https://sw6.ddev.site" -X GET https://sw6.ddev.site:9090/theme/theme/cf699bcc55c87558a9fd64769093a26a/js/all.js
HTTP/2 200 
age: 0
cache-control: no-cache, private
content-type: text/html; charset=UTF-8
date: Tue, 23 Apr 2024 05:45:58 GMT
referrer-policy: strict-origin-when-cross-origin
server: Apache/2.4.57 (Debian)
set-cookie: session-=60cdf036df37198179086ce6a434886b; path=/; secure; httponly; samesite=lax
strict-transport-security: max-age=31536000; includeSubDomains
vary: Accept-Encoding
x-content-type-options: nosniff
x-debug-token: f3d9d0
x-debug-token-link: https://sw6.ddev.site/_profiler/f3d9d0
x-frame-options: deny
x-robots-tag: noindex
x-symfony-cache: GET /: miss, store

HTTP/2 200 
accept-ranges: bytes
content-security-policy: default-src 'self' 'unsafe-eval' 'unsafe-inline';
content-type: text/javascript
date: Tue, 23 Apr 2024 05:45:58 GMT
last-modified: Tue, 23 Apr 2024 05:45:58 GMT
referrer-policy: strict-origin-when-cross-origin
server: MinIO Console
x-content-type-options: nosniff
x-frame-options: DENY
x-xss-protection: 1; mode=block
content-length: 1310

Upon debugging a bit, it seems related to missing headers. I think the addon-on is missing the headers for fetching cross origin files.

mc-config.json doesn't work

As far as I can tell, the mc-config.json file is not taking effect as expected.

ddev get oblakstudio/ddev-minio
ddev restart
ddev mc alias ls

Outputs: (Notice the warning about tmp configuration, and that there is no minio key)

mc: Configuration written to `/tmp/.mc/config.json`. Please update your access credentials.
mc: Successfully created `/tmp/.mc/share`.
mc: Initialized share uploads `/tmp/.mc/share/uploads.json` file.
mc: Initialized share downloads `/tmp/.mc/share/downloads.json` file.
gcs
  URL       : https://storage.googleapis.com
  AccessKey : YOUR-ACCESS-KEY-HERE
  SecretKey : YOUR-SECRET-KEY-HERE
  API       : S3v2
  Path      : dns

local
  URL       : http://localhost:9000
  AccessKey :
  SecretKey :
  API       :
  Path      : auto

play
  URL       : https://play.min.io
  AccessKey : Q3AM3UQ867SPQQA43P2F
  SecretKey : zuf+tfteSlswRu7BJ86wekitnifILbZam1KYY3TG
  API       : S3v4
  Path      : auto

s3
  URL       : https://s3.amazonaws.com
  AccessKey : YOUR-ACCESS-KEY-HERE
  SecretKey : YOUR-SECRET-KEY-HERE
  API       : S3v4
  Path      : dns

If you create it manually:

ddev mc alias set minio  http://localhost:10101 ddevminio ddevminio
ddev mc alias ls

It will be listed as expected, and you can go on to use it (eg mc mb minio/my-bucket).
As expected, the alias and bucket will go away with a ddev restart, as they are in tmp.

It also may make sense to make the alias local, as this seems to be the default?

Use standard test technique with github action

Would you mind using the standard test technique, which uses https://github.com/ddev/github-action-add-on-test ?

That way the expectations of what runs and how it shows in the github API would be the same all the time.

It's so very easy, as in https://github.com/ddev/ddev-addon-template/blob/main/.github/workflows/tests.yml