sverhoeven / docker-cartodb Goto Github PK

Hi,

I have tried to install in CentOS and Mac OS but failed at gpg signing.
Any suggestion to solve the problem?
Thanks.

Firman.

---> Running in 94f7fb06d2a4
gpg: directory /root/.gnupg' created gpg: new configuration file /root/.gnupg/gpg.conf' created
gpg: WARNING: options in /root/.gnupg/gpg.conf' are not yet active during this run gpg: keyring /root/.gnupg/secring.gpg' created
gpg: keyring `/root/.gnupg/pubring.gpg' created
gpg: requesting key D39DC0E3 from hkp server keys.gnupg.net
?: keys.gnupg.net: Host not found
gpgkeys: HTTP fetch error 7: couldn't connect: Connection timed out
gpg: no valid OpenPGP data found.
gpg: Total number processed: 0
gpg: keyserver communications error: keyserver unreachable
gpg: keyserver communications error: public key not found
gpg: keyserver receive failed: public key not found
The command '/bin/sh -c gpg --keyserver hkp://keys.gnupg.net --recv-keys D39DC0E3 && curl -sSL https://raw.githubusercontent.com/wayneeseguin/rvm/stable/binscripts/rvm-installer | bash -s stable --ruby && echo 'source /usr/local/rvm/scripts/rvm' >> /etc/bash.bashrc && /bin/bash -l -c rvm requirements && echo rvm_max_time_flag=15 >> ~/.rvmrc && /bin/bash -l -c 'rvm install 2.2.3' && /bin/bash -l -c 'rvm use 2.2.3 --default' && /bin/bash -l -c 'gem install bundle archive-tar-minitar' && /bin/bash -l -c 'gem install bundler compass --no-doc --no-ri' && ln -s /usr/local/rvm/rubies/ruby-2.2.3/bin/ruby /usr/bin && rm -rf /usr/local/rvm/src' returned a non-zero code: 2

I am using this image https://github.com/sverhoeven/docker-cartodb to start my own container, and then created a reverse proxy by nginx to enable ssl.

The ssl website pulls up but the problem is carto does not detect the change in protocol, and i am getting mixed content warning due to api requests by cbd.js on HTTP protocol.

for example: http://carto.gq/user/dev/api/v1/viz/?tag_name=&q=&page=1&type=&exclude_shared=false&per_page=20&locked=&tags=&shared=no&only_liked=false&order=updated_at&types=table&exclude_raster=true.

What file I should to change the protocol of api calls, here is my installation https://carto.gq .

i am not a tech expert but i thing this image is missing ability to grab inject postgres database vaiables and use them to store data persistently, is there a way to hard code them for use.

As I start docker and running CartoDB,

I try to upload csv and playing with the map.

with the Analysis "Calculate Clusters of Points"

it coming with error "Analysis A1 failed: ImportError: cannot import name symeig"

http://gruntjs.com/getting-started
The command '/bin/sh -c git clone --recursive git://github.com/CartoDB/cartodb.git && cd cartodb && git checkout master && npm install [email protected] && rm -r /tmp/npm-* /root/.npm && perl -pi -e 's/gdal==1.10.0/gdal==1.11.3/' python_requirements.txt && pip install --no-use-wheel -r python_requirements.txt && /bin/bash -l -c 'bundle install' && /bin/bash -l -c 'bundle exec grunt --environment development' && rm -rf .git /root/.cache/pip node_modules' returned a non-zero code: 99
Any idea what could be wrong?

On a fresh build, logged in as admin4example user, the users page of an organization is empty.

Using devtools I see several js files like organization_new.js not being found.

This map couldn't be rendered Unauthorized. Please refresh the page and contact us if the error persists

i am unable to upload any dataset error which i am getting is that it shows pending upload for sometime and after that this code 8063ae73-3bb8-4cf5-9b7f-99338ba6c871

anyway it worked it worked for the first time but now its always getting stuck what can be the reason.

How are you,
We are developing progressive web pages - pwa - https://developers.google.com/web/progressive-web-apps/ to show carto maps and the implementation needs to use the https protocol.

I Take the code of your implementation and modify it, so that it runs with https and use cartodb production database the creation of the image (Dockerfile) runs without problems and the site of cartodb work properly I can consult data of the users, the organization and everything related to the administration part of the page, however I can not create maps, always shows me a blank page with the close button to return to the carto page implemented locally

I could give a light that I can review to correct this issue, I have reviewed these logs inside the container.

find / -name ".log" -exec ls -lt {} ; | grep -i "Aug"
tail -f /var/log/postgresql/postgresql-10-main.log
tail -f /var/log/nginx/cartodb_error.log
tail -f /var/log/nginx/cartodb_access.log
tail -f /var/log/redis/redis-server.log
tail -f /tmp/analysis.log
tail -f /cartodb/log/production.log
tail -f /cartodb/resque.log
tail -f /CartoDB-SQL-API/logs/cartodb-sql-api.log
The Windshaft-cartodb log does not appear anywhere

There would be a way to change the main carto configuration files (app_config.yml, database.yml),
Windshaft-cartodb (production.js), CartoDB-SQL-API (production.js) and restart the service
to prove that the change has emerged effect. Or invoke a request using postman to prove that separate services are working.

I attach the implementation (docker-cartodb-prod.tar.gz) and a manual (RunSteps.txt) to execute it.
To see if someone can give me a hand.
Runing Container

Source code
docker-cartodb-prod.tar.gz
Run Steps
RunSteps.txt
Link for downloading:
https://drive.google.com/open?id=1ZS9rLCvE9GygEJPvJi57sXvs9bo3KIkW
I has similar issue reported in case "carto not working #29" but in this topic don't detail the answer. My white blank showed when try to create new map is :

Thanks

Not exactly an issue, just that imho client_max_body_size should be the same value as DEFAULT_MAX_IMPORT_FILE_SIZE, which is 150M

Would it be possible to upgrade the cartodb version within this docker config, and if so, how? The upgrade.md from the original cartodb repo seems not to be up to date. Thanks.

Hello,

When I run these commands as part of the persist data workaround, I get a permission denied message:

docker cp cartodb_pgdata:/var/lib/postgresql $PWD/cartodb_pgdata

Then I get this:
jake@ubu:~$ docker cp cartodb_pgdata:/var/lib/postgresql $PWD/cartodb_pgdata
**mkdir /home/jake/cartodb_pgdata/postgresql: permission denied**

I can see that it won't let me copy from the cartodb_pgdata folder. When I look at the folder in the files GUI I get this message: You do not have the permissions necessary to view the contents of “cartodb_pgdata”.

Not sure what the way around this is. I built a lovely house price affordability over time/space map that I've saved as a .carto but would very much like it to always be there when I open it and eventually share it.

Thanks again for your amazing work on this, it's all very exciting!

Same issue as found here

I was just googling the issue and came up with that reported issue on another docker image. My guess is that it's a location issue when installing the node packages.

Hi,

Why did you choose to place all the components (cartodb, postgresql, redis,...) in a single container ?
Wouldn't it be cleaner with multiple containers, at least one distinct for Postgresql ?

Thanks by the way for this image which makes cartodb really easy to use !

HI.

It is possible that in the development implementation you can load files with .carto extension

when i create a map, the editor/builder doesn't show up, it just returns a blank screen

on google chrome console it shows:
Failed to load resource: the server responded with a status of 502 (Bad Gateway)

running docker build .

I get the following error
Running "browserify:test_specs_for_browserify_modules" (browserify) task

Error: Cannot find module '../../../../../../vendor/assets/javascripts/carto-node/carto-node' from '/cartodb/lib/assets/test/spec/cartodb/account'
Warning: Error running grunt-browserify. Use --force to continue.

Aborted due to warnings.
The command '/bin/sh -c git clone --recursive git://github.com/CartoDB/cartodb.git && cd cartodb && git checkout master && cd lib/sql && PGUSER=postgres make install && service postgresql start && /bin/su postgres -c /tmp/cartodb_pgsql.sh && service postgresql stop && cd - && npm install && rm -r /tmp/npm-* /root/.npm && perl -pi -e 's/gdal==1.10.0/gdal==1.11.3/' python_requirements.txt && pip install --upgrade pip && pip install --no-binary :all: -r python_requirements.txt && /bin/bash -l -c 'bundle install' && cp config/grunt_development.json ./config/grunt_true.json && /bin/bash -l -c 'bundle exec grunt' && rm -rf .git /root/.cache/pip node_modules' returned a non-zero code: 6

Any idea what could be wrong?

Hello,
Thanks to your previous answer, i ve got a working instance on my ubunutu.
I am starting "playing" but i realy don't know much about rubby...
Do you know where to find the different bundle tasks available ?
I found some interesting things into your setup_organization.sh but i am looking for tasks to delete users and organizations...
Thanks for your help and your work !

Hello,
I made a new install on ubuntu. It ran successfully until the end.
But then i can see only the header when i connect the app.

Any help or idea ?

Thanks !

Hi, is there any possibilities to run the container with Carto in production mode?

f9e4c4ce9bfe        sverhoeven/cartodb   "/bin/bash /opt/st..."   24 minutes ago      Exited (134) 14 minutes ago

405e7d58800d        sverhoeven/cartodb   "/bin/bash /opt/st..."   12 minutes ago      Exited (134) 2 minutes ago

Is there a timeout?

Looks like the User page isn't loading properly, so I can't create users.

Is there a way to do this manually? Or perhaps a fix for the Users page?

Here's what I get: https://www.dropbox.com/s/y6ftdi45blo9ltl/Screenshot%202018-05-29%2016.51.39.png?dl=0

I am getting the following error "Analysis A1 failed: ValueError: numpy.dtype has the wrong size, try recompiling" when running the Predict Trends and Volatility Analysis
I am using the following carto file - https://s3.amazonaws.com/cartodb/static/learn/predict-markov.zip from this link https://carto.com/learn/guides/analysis/predict-trends-and-volatility/

Building the bionic branch gives errors when aggregating.

After selecting the storms 2014 dataset.

When aggregating by square I get:

windshaft error:
"Failed to process "polygon-fill" property: unable to compute ramp, The requested grid is too big to be rendered

When zooming in at some point I get:

windshaft error:
relation "aggregation.agg_admin0" does not exist

When aggregate==animated and column==created_at

Uncaught Error: aggregation "millennium" is not defined
    at Object.helper.add (histogram-helper.js:87)
    at Object.helper.fillTimestampBuckets (histogram-helper.js:41)
    at child.parse (histogram-dataview-model.js:169)

Hi, I would like to know how to do the secure site setup since you moved the nginx into the docker file.

Also for example if you use lets encrypt to provide your ssl you would have to volume your lets encrypt file as well.

Thank you.

Hey great work! Thanks for that.

Would you please explain me how to install https://github.com/cartodb/airship to the docker?

You can see the installation at the bottom of the page.

I managed to add the Sync tables feature (http://cartodb.readthedocs.io/en/latest/operations/run_sync_tables.html) by doing the following:

• UPDATE users SET sync_tables_enabled = true;
• bundle exec rake cartodb:sync_tables

Would be great if we could enable this during setup.

I'm able to deploy the container to Azure, and everything seems to be running ok. But I'm trying to make data persistent and I'm unable of running the comands of the documentation since Azure limitations. Is there other way to make data persistent? For example inside the container.

Thanks

Docker image all set up, tested and working.
Go to fill the internal geocoder
Run:

docker exec -ti bash -c /cartodb/script/fill_geocoder.sh

from clean docker container Varnish error - log - https://gist.github.com/christophekey/853ad3df5e9d1d83550aa13c724b726b

problem seems to result from Varnish and not respecting timeouts, so kill running varnish processes run again, completes and reports - ** Everything OK **

log - https://gist.github.com/christophekey/5805653cab2ba4b40c503d5c7a6171f9

restart the container - geocoder only returns null in the geom table when running a geocode of IP to lat, lon.

Tried with a new clean build, issue persists.

Hello,

I am trying to change the default port(80) to 3000 but I am having no luck. Can you please explain how can I achieve that?

Thanks

After making the pg_data persistent we get the following error in the logs:

Missing argument in printf at /usr/bin/pg_lsclusters line 126.
 * Starting PostgreSQL 10 database server
 * Error: The cluster is owned by user id 105 which does not exist
   ...fail!

Hi,

I hope you can help me with this.

I want to make a remote connection from my local windows-pc (with pgadmin4 and QGIS) to the postgreSQL inside the sverhoeven/cartodb docker container. Normally with a server installation of postgresql (not docker) I would connect to default port 5432 but with the docker run command in your guide, I run the docker with port 80:80 and that port is used. Would the correct way be to run docker multiple ports docker run -p 80:80 -p 5432:5432?

Thanks you.

/Emil

Hello,
First off thanks for making this repo. My question is about enabling the ability to add ArcIGIS server endpoints as data sources. It seems this is possible with carto but not entirely sure the steps to enable it with this repo. I am currently getting Unsupported/Unrecognized file type (1002) when attempting to add the ArcGIS services, so I assume it is disabled by default. Any help is much appreciated, thanks.

Hi,
It seems the build is failing. I've run docker build -t=sverhoeven/cartodb docker-cartodb/ and I get the following error at step 23/49:

RUN service postgresql start && /bin/su postgres -c       /tmp/template_postgis.sh && service postgresql stop
 ---> Running in 67a91db53068
 * Starting PostgreSQL 10 database server
   ...done.
bash: /tmp/template_postgis.sh: /bin/bash^M: bad interpreter: No such file or directory
The command '/bin/sh -c service postgresql start && /bin/su postgres -c       /tmp/template_postgis.sh && service postgresql stop' returned a non-zero code: 126

Fetching thin 1.7.0
Downloading thin-1.7.0 revealed dependencies not in the API or the lockfile
(rack (< 3, >= 1)).
Either installing with `--full-index` or running `bundle update thin` should fix
the problem.

In Gemfile:
  thin
The command '/bin/sh -c git clone --recursive git://github.com/CartoDB/cartodb.git
&& cd cartodb && git checkout $CARTODB_VERSION && cd lib/sql &&     
PGUSER=postgres make install && service postgresql start && /bin/su postgres -c   
/tmp/cartodb_pgsql.sh && service postgresql stop && cd - && npm install &&     
rm -r /tmp/npm-* /root/.npm && perl -pi -e 's/gdal==1\.10\.0/gdal==2.2.2/'
python_requirements.txt  && pip install --no-binary :all: -r python_requirements.txt
&& gem install bundler bundle compass archive-tar-minitar &&
/bin/bash -l -c 'bundle install' && cp config/grunt_development.json
./config/grunt_true.json && /bin/bash -l -c 'bundle exec grunt''
returned a non-zero code: 34

Since $CARTODB_VERSION is set to master, this might have been introduced with recent commits to the remote repo's master branch.

Installed npm version does not match with required [3.10.9] Installed: 2.14.22

i am trying to build this image but with no success.

Hi,

In order to create a new user, the cartodb recommends to use /cartodb/script/create_dev_user.

But in this script, SUBDOMAIN, PASSWORD, ADMIN_PASSWORD and EMAIL are hardcoded, so it fails with error createdb: database creation failed: ERROR: database "carto_db_development" already exists

Then, is there a better way to create a new user ?

Sometimes when I try to update a layer's properties on a map with multiple layers, the server starts failing and eventually producing a 502 error. The docker service then reports the docker container as unhealthy. When I look into the docker logs, I find a few errors, such as these ones:

[2017-07-10 14:21:02.257] [ERROR] console - [millstone] processing style 'style0'
[2017-07-10 14:21:02.259] [ERROR] console - [millstone] finished processing '/tmp/cdb-tiler-dev/millstone-dev/base'

These happen multiple times. The only other error I find is this one:

PG::Error: SSL error: sslv3 alert bad record mac

Which actually seems to be isolated and the millstone errors happen around the time the server stopped working, so it seems the error is with millstone. Any ideas on what could be happening? Overall the server works pretty good, and this only happens sometimes when editing maps with multiple layers/analyses. Would appreciate any help, thanks!

Setup development server with docker image, with self assign ssl
setting up development on Ubuntu 12.04.5 LTS on aws ec2

1.When creating a new map with any dataset, the error starts to appear with jquery:8630 bad request. {"error":["schema "cdb_dataservices_client" does not exist"]}

Sir how to enable private maps, the privacy options for published maps are freezed.

Is there a workaround to avoid needing to specify the hostname when starting the container in a GKE kubernetes cluster, AWS fargate, Azure ACI?

I have been trying to get the container to run on Azure's ACI and it will run but I am getting some pretty serious error messages. As far as I know, there is no way to specify a hostname explicitly when starting the container as an ACI.
Is there a workaround where I can edit a file after it has been provisioned or something else?

Error Messages:
Started GET "/" for 127.0.0.1 at 2018-06-20 01:53:59 +0000
Processing by Admin::PagesController#index as */*
…
Redirected to http://localhost/login
…
Started GET "/favicon.ico" for 127.0.0.1 at 2018-06-20 01:54:25 +0000
ActionController::RoutingError (No route matches [GET] "/favicon.ico"):
…
Started POST "/sessions/create" for 127.0.0.1 at 2018-06-20 01:54:39 +0000
Started GET "/user/dev/dashboard/" for 127.0.0.1 at 2018-06-20 01:54:39 +0000

Here is a link to my S.O. Q: https://stackoverflow.com/questions/50959411/is-there-a-way-to-specify-hostname-when-starting-container-on-azure-cloud-instan

Hello,

First of all, I wanted to thank you very much for putting this together, it's an incredibly handy project!

Second, I have a query on the use of the container: if I access a large dataset (e.g. the NYC Pluto) and play around with the data a little bit (zoom in, out, etc.), after a while, it tells me that I've reached the limit of tile views and does not render any data. Instead, it "invites" me to upgrade to a premium account. I thought the software was running off the local server hence I could interact with it for as much as I wanted without hitting any of these limits. Have you encountered this problem? Do you have any suggestion on how to solve it? Thank you very much in advance!

I am hosting a container from this docker image on AWS EC-2 machine i have enable https in my security group and changed the port container listens to 443 problem is mydomain.com:443/login gets redirected to mydomain.com/login. This application ignores the port used is there a workaround.

I've runned the sudo docker build -t=sverhoeven/cartodb docker-cartodb/ and it gives me this error after some time...

Collecting pip
Downloading https://files.pythonhosted.org/packages/62/a1/0d452b6901b0157a0134fd27ba89bf95a857fbda64ba52e1ca2cf61d8412/pip-10.0.0-py2.py3-none-any.whl (1.3MB)
Installing collected packages: pip
Found existing installation: pip 8.1.1
Not uninstalling pip at /usr/lib/python2.7/dist-packages, outside environment /usr
Successfully installed pip-10.0.0
Traceback (most recent call last):
File "/usr/bin/pip", line 9, in
from pip import main
ImportError: cannot import name main
The command '/bin/sh -c git clone --recursive git://github.com/CartoDB/cartodb.git && cd cartodb && git checkout master && cd lib/sql && PGUSER=postgres make install && service postgresql start && /bin/su postgres -c /tmp/cartodb_pgsql.sh && service postgresql stop && cd - && npm install && rm -r /tmp/npm-* /root/.npm && perl -pi -e 's/gdal==1.10.0/gdal==1.11.3/' python_requirements.txt && pip install --upgrade pip && pip install --no-binary :all: -r python_requirements.txt && /bin/bash -l -c 'bundle install' && cp config/grunt_development.json ./config/grunt_true.json && /bin/bash -l -c 'bundle exec grunt' && rm -rf .git /root/.cache/pip node_modules' returned a non-zero code: 1

While building on the master branch is see the following message Extension created without PARALLEL support.

1. How to enable the internal geocoder API for other accounts as it works for the user 'geocoder'

2. How to enable the internal geocoder in builder for bother user 'geocoder' and other users.
   I did try this - bundle exec rake cartodb:db:configure_geocoder_extension_for_non_org_users[geocoder,true]
   I get an error at the end Sequel::Postgres::Database (0.9ms) SELECT * FROM "users" WHERE ("username" = 'geocoder') LIMIT 1
   ERROR geocoder

As indicated the internal geocoder works via the API for the user geocoder

If I run the following: http://cartodb.local/user/geocoder/api/v2/sql?q=SELECT%20(geocode_postalcode_points(Array[%2710013%27,%27G9H%27,%2703782%27],%20Array[%27USA%27,%20%27Canada%27,%20%27US%27])).*

I do get a response:
{"rows":[{"q":"10013","c":"USA","iso3":"USA","geom":"0101000020E610000090679AEE4D8052C0C4C6BC8E385C4440","success":true},{"q":"G9H","c":"Canada","iso3":"CAN","geom":"0101000020E610000096B20C71AC1B52C0423EE8D9AC2A4740","success":true},{"q":"03782","c":"US","iso3":"USA","geom":"0101000020E6100000AE47E17A140652C0E10B93A982B14540","success":true}],"time":0.038,"fields":{"q":{"type":"string"},"c":{"type":"string"},"iso3":{"type":"string"},"geom":{"type":"geometry"},"success":{"type":"boolean"}},"total_rows":3}

Hi,
I use your script today. Cartodb interface work fine and I loged in but when upload a file I get a error. I tried to create an empty dataset but the page is blocked (I don't see columns and when I add a new columns the page goes in wait).
Do you have a solution?
Regards
Matteo

A build in the bionic branch.

At some point navigating the site a metric is posted to /api/v3/metrics url with a not found error.

At least this is the case on azure. Apparently we cannot easily edit the ownership of mounted directories, at least when using file share. I posted an issue here and I invite you to have a look. Perhaps you can offer more knowledgeable information to solve the issue... https://stackoverflow.com/questions/51531270/changing-ownership-of-files-on-azure-file-share-mounted-on-aci

Whilst I am not docker expect, I would like to extend the current dockerfile with the observatory extension. In my research I came across this documentation - https://github.com/xzfvictor/mapping.community-carto_install_doc
I would appreciate any pointers as to how I could proceed?