geocompx / docker Goto Github PK
View Code? Open in Web Editor NEWDockerfiles for Geocomputation
Home Page: https://github.com/geocompx/docker/pkgs/container/docker
License: GNU General Public License v3.0
Dockerfiles for Geocomputation
Home Page: https://github.com/geocompx/docker/pkgs/container/docker
License: GNU General Public License v3.0
Heads-up @michaeldorman.
Context: geocompx/geocompy#52 and #26
Or similar, see error messages here:
#10 757.2 Failed to build cartopy pandas
#10 757.2 ERROR: Could not build wheels for cartopy, which is required to install pyproject.toml-based projects
------
Dockerfile:13
--------------------
11 | RUN pip3 install rasterio rioxarray netcdf4 h5netcdf
12 | # Install Python packages for visualisation
13 | >>> RUN pip3 install jupyter matplotlib hvplot pyviz geoviews
14 | # Install quarto
15 | RUN curl -fsSL https://cli.github.com/packages/githubcli-archive-keyring.gpg | sudo dd of=/etc/apt/trusted.gpg.d/githubcli-archive-keyring.gpg
--------------------
error: failed to solve: process "/bin/sh -c pip3 install jupyter matplotlib hvplot pyviz geoviews" did not complete successfully: exit code: 1
Error: buildx failed with: error: failed to solve: process "/bin/sh -c pip3 install jupyter matplotlib hvplot pyviz geoviews" did not complete successfully: exit code: 1
https://github.com/geocompr/docker/runs/5438444509?check_suite_focus=true#step:8:6181
Any ideas @anitagraser or anyone?
Can we add GRASS, SAGA and qgisprocess to our geocompr:qgis image? This way we could harness the full potential of QGIS, and it would be also rather helpful for writing the qgisprocess vignette (see r-spatial/qgisprocess#16). For now, I have simply adjusted and rebuild your Dockerfile locally. Thinking it over again, yes, it is a good idea to add qgisprocess but for writing the vignette it makes more sense to mount my local qgisprocess repo. Hence, we would have to adjust the Dockerfile as follows (I can put in a PR if you want me to):
FROM geocompr/geocompr:qgis
RUN apt-get update \
&& apt-get install --no-install-recommends --no-install-suggests --allow-unauthenticated -y \
qgis-plugin-grass \
saga
RUN env QT_QPA_PLATFORM='offscreen' qgis_process
RUN echo "[PythonPlugins]\nprocessing=true" >> root/.local/share/QGIS/QGIS3/profiles/default/QGIS/QGIS3.ini
RUN R -e "remotes::install_github('paleolimbot/qgisprocess')"
The penultimate line is necessary to enable 3rd party providers when running qgis_process in a headless state (see qgis/QGIS#37989). And we need to execute the line before (RUN env QT_QPA_PLATFORM='offscreen' qgis_process) in order to create the QGIS3.ini file (maybe a bad hack, but it works).
FYI @paleolimbot
To include extras.
For some reason. Logs: https://github.com/geocompr/docker/runs/5431901131?check_suite_focus=true#step:8:695
Heads-up @anitagraser plan for now is to remove movingpandas and add later. I'll keep this issue open to track it.
I may have made a silly mistake, quite likely as currently my Python dev skills amount to guesswork!
Dailly is a bit intense.
I hit issues with sf
installation but this can be worked around following guidance from the Rocker maintainers here: rocker-org/rocker-versioned2#631
With the current set-up we don't do any tagging of different versions. However it would improve reproducibility if we could add tags making it easier for people to get images produced at a particular data/release. The action on which our Docker actions are based enable tagging but I wasn't sure how best to implement this https://github.com/docker/build-push-action/blob/master/docs/advanced/tags-labels.md
Any comments/suggestions on this very welcome.
As shown below:
Source: https://github.com/geocompx/docker/actions/workflows/suggests.yml
Related: geocompx/geocompr#1049
As discussed with @Nowosad currently the Docker images are rebuilt every time one of us pushes to the repo. This is not ideal. Any ideas for getting a more automated system going? It's tricky because DockerHub only triggers builds when it detects commits in this repo I think, so I was thinking of a cron job on my computer, but that is not an ideal solution. Suggestions welcome.
Solution: use ubuntugis stable ppa
With upstream containers building first.
Seeing this message:
#8 53.84 Downloading Cartopy-0.18.0.tar.gz (14.4 MB)
#8 53.99 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 14.4/14.4 MB 92.6 MB/s eta 0:00:00
#8 54.39 Preparing metadata (setup.py): started
#8 54.63 Preparing metadata (setup.py): finished with status 'error'
#8 54.64 error: subprocess-exited-with-error
#8 54.64
#8 54.64 × python setup.py egg_info did not run successfully.
#8 54.64 │ exit code: 1
#8 54.64 ╰─> [13 lines of output]
#8 54.64 Traceback (most recent call last):
#8 54.64 File "/tmp/pip-install-xpr0hdls/cartopy_f8366d3bf5f44cd6b51d81b49e18c0f5/setup.py", line 43, in <module>
#8 54.64 import numpy as np
#8 54.64 ModuleNotFoundError: No module named 'numpy'
#8 54.64
#8 54.64 During handling of the above exception, another exception occurred:
#8 54.64
#8 54.64 Traceback (most recent call last):
#8 54.64 File "<string>", line 2, in <module>
#8 54.64 File "<pip-setuptools-caller>", line 34, in <module>
#8 54.64 File "/tmp/pip-install-xpr0hdls/cartopy_f8366d3bf5f44cd6b51d81b49e18c0f5/setup.py", line 45, in <module>
#8 54.64 raise ImportError('NumPy 1.10+ is required to install cartopy.')
#8 54.64 ImportError: NumPy 1.10+ is required to install cartopy.
#8 54.64 [end of output]
Source: https://github.com/geocompr/docker/runs/7054120092?check_suite_focus=true#step:8:736 Any ideas welcome, call for help in the dark @martinfleis!
@Robinlovelace do I understand correctly that the geocompr images are based on rocker-org/geospatial? If so, https://github.com/rocker-org/geospatial uses very old R version and has not been updated in ~2 years. Should we do something with it?
Hi @Robinlovelace -- are there any plans to rename images from geocompr to geocompx?
The README is R focussed but in fact we now have the Python images.
As documented here: https://pythonspeed.com/articles/activate-conda-dockerfile/
This is an issue associated with images that build on geocompr/geocompr:minimal
:
#6 0.349 > remotes::install_github('geocompr/geocompkg', upgrade = TRUE, dependencies = TRUE)
#6 0.542 Skipping install of 'geocompkg' from a github remote, the SHA1 (6922a37b) has not changed since last install.
#6 0.542 Use `force = TRUE` to force installation
#6 0.542 >
#6 0.542 >
#6 DONE 3.0s
Source: https://github.com/geocompr/docker/runs/3776530772#step:8:270
Not through pip
...
Context: https://github.com/rocker-org/rocker-versioned2/blob/master/scripts/install_python.sh
As discussed with @benz0li, we could use the b-data
stack throughout, with more consistency and good R/Python support. First we would need to think about similarities/differences and pros/cons, leaving this as an open question, I'm a bit out of my depth, so any advice / PRs welcome : )
With the following message:
Dockerfile:48
--------------------
46 | RUN sh -c 'echo "deb [arch=amd64 signed-by=/etc/apt/keyrings/qgis-archive-keyring.gpg] https://qgis.org/ubuntugis `lsb_release -c -s` main" > /etc/apt/sources.list.d/qgis.list'
47 | RUN apt-get update
48 | >>> RUN apt-get install -y qgis qgis-plugin-grass saga
49 | RUN wget -qO sagang_plugin.zip https://plugins.qgis.org/plugins/processing_saga_nextgen/version/0.0.7/download/
50 | RUN unzip -q sagang_plugin.zip -d /home/rstudio/.local/share/QGIS/QGIS3/profiles/default/python/plugins
--------------------
ERROR: failed to solve: process "/bin/sh -c apt-get install -y qgis qgis-plugin-grass saga" did not complete successfully: exit code: 100
Error: buildx failed with: ERROR: failed to solve: process "/bin/sh -c apt-get install -y qgis qgis-plugin-grass saga" did not complete successfully: exit code: 100
Source: https://github.com/geocompx/docker/actions/runs/6321057960/job/17164486628#step:8:644
Relevant lines:
Lines 48 to 50 in 00f4c90
Any ideas @jannes-m ?
E.g. as can be seen here for proj: https://launchpad.net/~ubuntugis/+archive/ubuntu/ubuntugis-unstable?field.series_filter=focal
Plan: remove these Docker images.
Hot on the heels of #45
With this error:
Encountered problems while solving:
- nothing provides requested quarto
The environment can't be solved, aborting the operation
error libmamba Could not solve for environment specs
critical libmamba UnsatisfiableError
The command '/usr/local/bin/_dockerfile_shell.sh micromamba install -y -n base -f /tmp/env.yaml && micromamba clean --all --yes' returned a non-zero code: 1
I need this for my own purposes but will be useful for others I think.
I think I know why: geocompy is now in geocompx/geocompy not geocompx/py
Evidence: https://github.com/geocompx/docker/actions/runs/4501197328/jobs/7921300929#step:8:553
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.