Comments (6)
See https://github.com/c95560/gaffer-docker/tree/bugfix/gh-75_redundant_docker_compose_image_builds
from gaffer-docker.
On a related note, you could reduce the size of all the docker-compose with "extends" to share common configuration given many differ only on "command": https://docs.docker.com/compose/extends/
from gaffer-docker.
Yep, you are right, docker-compose doesn't seem to recognise that the same dockerfile, context and args are being used for multiple services and unnecessarily builds them multiple times.
The reason why I decided to leave the duplicated build config in was to allow individual services to be rebuilt. For example, this doesn't work in your PR:
$ docker-compose --project-directory gaffer-docker/docker/gaffer -f gaffer-docker/docker/gaffer/docker-compose.yaml build accumulo-tserver
accumulo-tserver uses an image, skipping
Admittedly there are probably few use cases for this. Only one immediately springs to mind - local migration testing of new versions of custom iterators. I haven't actually tested this workflow works, but I image something along the lines of:
# Build all images with v1 of custom iterators
$ docker-compose build
# Spin up stack
$ docker-compose up -d
# Ingest data and query
# Stop tablet server running v1 of custom iterators
$ docker-compose stop accumulo-tserver
# Build image with v2 of custom iterators
$ docker-compose build accumulo-tserver
# Spin the tablet server up with v2
$ docker-compose start accumulo-tserver
# Run tests
If we remove the duplicated build config then you would need to know to re-build accumulo-master
instead of accumulo-tserver
. A minor inconvenience I guess...
Didn't know about "extends" - good spot! Would that help in this situation? Would using "extends" help docker-compose realise the services are using the same container image and only build it once, whilst still allowing us to trigger re-builds on any service?
from gaffer-docker.
Extends should let you re-trigger builds on demand, however it might be that you end up with redundant builds again... I'm not sure how intelligent docker-compose is in this regard. Might be worth a quick experiment.
Admittedly there are probably few use cases for this
You used to be able to specify multiple tags with the extends syntax, but I think the functionality has been removed since compose > 3.
from gaffer-docker.
Looks like extends was dropped for version > 3.0. (so much for the experiment)
# Stop tablet server running v1 of custom iterators
$ docker-compose stop accumulo-tserver
# Build image with v2 of custom iterators
$ docker-compose build accumulo-tserver
# Spin the tablet server up with v2
$ docker-compose start accumulo-tserver
If this is the desired behaviour then I guess you could use docker-compose.override.yml to replace the image of the accumulo-tserver service with v2 of custom iterators? This might be better practice in terms of repeatability.
e.g.
docker-compose.yml
version: "3.7"
services:
simple-a:
image: blah-a
build:
context: .
dockerfile: blah-a.Dockerfile
simple-b:
depends_on:
- simple-a
image: blah-a
testing v2
docker-compose.override.yml
version: "3.7"
services:
simple-b:
image: blah-a:v2
build:
context: .
dockerfile: blah-a:v2.Dockerfile
Then run aforementioned commands that stop / start simple-b (or accumulo-tserver) service only.
from gaffer-docker.
Merged into develop
from gaffer-docker.
Related Issues (20)
- Actions doesn't push Gremlin image
- Update Gaffer version to 2.2.0
- Issue with helm deployment on v2.2.0
- Allow for manually running the release action
- Gaffer-Gremlin example schema broken
- Update Gaffer Version to 2.2.1
- Allow for building Dockerfiles from develop branch of Gaffer HOT 1
- Upgrade GitHub Actions versions
- Gaffer gremlin should have tinkerpop v3 serialiser in by default
- Improve the gaffer-gremlin example
- Bug: Cannot install Gaffer Helm charts on Kubernetes 1.25+ HOT 3
- Add example of using the proxy store
- Upgrade source docker images
- Add Federated Store example
- Improve the federated store example
- Update Gaffer Version to 2.2.2
- Fix release notes generation
- Improve the release process
- Fix or remove the github artifacts upload
- Remove gaffer-gremlin and Migrate examples
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from gaffer-docker.