amancevice / caravel

Docker image for AirBnB's Caravel

         .mm############mm.
       m####################m
     m#####`"#m m###"""'######m
    ######*"  "   "   "mm#######
  m####"  ,             m"#######m                                               _
 m#### m*" ,'  ;     ,   "########m               ___ ___  _ __ ___  _ __  _   _| |_ ___ _ __ ___ ✨ 
 ####### m*   m  |#m  ;  m ########         -----/ __/ _ \| '_ ` _ \| '_ \| | | | __/ _ \ '__/ __|
|######### mm#  |####  #m##########|   ---------| (_| (_) | | | | | | |_) | |_| | ||  __/ |  \__ \
 ###########|  |######m############      --------\___\___/|_| |_| |_| .__/ \__,_|\__\___|_|  |___/
 "##########|  |##################"                                 |_|
  "#########  |##################"   
    ########   '################
     "####☭"      "###########"
       """"""""""""""""""""""

caravel's Issues

Error: An API access token is required to use Mapbox GL.

The "Mapbox" visualisation indicates that it is loading, but the console has the has the following error:

explore.entry.js:34 Uncaught Error: An API access token is required to use Mapbox GL. See https://www.mapbox.com/developers/api/#access-tokens

Use sqlite configuration with error "unable to open database file"

When I use the sqlite configuration as described in the documentation:

docker run --detach --name caravel
--env SECRET_KEY="mySUPERsecretKEY"
--env SQLALCHEMY_DATABASE_URI="sqlite:////home/caravel/caravel.db"
--publish 8088:8088
--volume ~/caravel/caravel.db:/home/caravel/caravel.db
amancevice/caravel

the container exits and docker logs shows the error:

2016-05-30 12:42:15,123:ERROR:flask_appbuilder.security.sqla.manager:DB Creation and initialization failed: (sqlite3.OperationalError) unable to open database file

I think the cause is the permission of the folder /home/caravel/caravel.db , because its owner is root and the runner is caravel.

Running caravel db upgrade fails on 0.10.0 -> 0.11.0 ("Duplicate column name 'python_date_format'")

Whilst performing the upgrade steps in staging, I get the following error when upgrading the database.

We're running off a MariaDB backend - currently investigating if this is an issue upstream or if we skipped some upgrade steps between 0.10.0 and 0.11.0.

Here's the error thrown by docker exec caravel-new caravel db upgrade

INFO  [alembic.runtime.migration] Running upgrade 27ae655e4247 -> 960c69cb1f5b, add dttm_format related fields in table_columns
Traceback (most recent call last):
  File "/usr/lib/python3.5/site-packages/sqlalchemy/engine/base.py", line 1139, in _execute_context
    context)
  File "/usr/lib/python3.5/site-packages/sqlalchemy/engine/default.py", line 450, in do_execute
    cursor.execute(statement, parameters)
  File "/usr/lib/python3.5/site-packages/MySQLdb/cursors.py", line 226, in execute
    self.errorhandler(self, exc, value)
  File "/usr/lib/python3.5/site-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
    raise errorvalue
  File "/usr/lib/python3.5/site-packages/MySQLdb/cursors.py", line 217, in execute
    res = self._query(query)
  File "/usr/lib/python3.5/site-packages/MySQLdb/cursors.py", line 378, in _query
    rowcount = self._do_query(q)
  File "/usr/lib/python3.5/site-packages/MySQLdb/cursors.py", line 341, in _do_query
    db.query(q)
  File "/usr/lib/python3.5/site-packages/MySQLdb/connections.py", line 280, in query
    _mysql.connection.query(self, query)
_mysql_exceptions.OperationalError: (1060, "Duplicate column name 'python_date_format'")

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/bin/caravel", line 155, in <module>
    manager.run()
  File "/usr/lib/python3.5/site-packages/flask_script/__init__.py", line 412, in run
    result = self.handle(sys.argv[0], sys.argv[1:])
  File "/usr/lib/python3.5/site-packages/flask_script/__init__.py", line 383, in handle
    res = handle(*args, **config)
  File "/usr/lib/python3.5/site-packages/flask_script/commands.py", line 216, in __call__
    return self.run(*args, **kwargs)
  File "/usr/lib/python3.5/site-packages/flask_migrate/__init__.py", line 200, in upgrade
    command.upgrade(config, revision, sql=sql, tag=tag)
  File "/usr/lib/python3.5/site-packages/alembic/command.py", line 174, in upgrade
    script.run_env()
  File "/usr/lib/python3.5/site-packages/alembic/script/base.py", line 407, in run_env
    util.load_python_file(self.dir, 'env.py')
  File "/usr/lib/python3.5/site-packages/alembic/util/pyfiles.py", line 93, in load_python_file
    module = load_module_py(module_id, path)
  File "/usr/lib/python3.5/site-packages/alembic/util/compat.py", line 68, in load_module_py
    module_id, path).load_module(module_id)
  File "<frozen importlib._bootstrap_external>", line 388, in _check_name_wrapper
  File "<frozen importlib._bootstrap_external>", line 809, in load_module
  File "<frozen importlib._bootstrap_external>", line 668, in load_module
  File "<frozen importlib._bootstrap>", line 268, in _load_module_shim
  File "<frozen importlib._bootstrap>", line 693, in _load
  File "<frozen importlib._bootstrap>", line 673, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 665, in exec_module
  File "<frozen importlib._bootstrap>", line 222, in _call_with_frames_removed
  File "/usr/lib/python3.5/site-packages/caravel/migrations/env.py", line 100, in <module>
    run_migrations_online()
  File "/usr/lib/python3.5/site-packages/caravel/migrations/env.py", line 93, in run_migrations_online
    context.run_migrations()
  File "<string>", line 8, in run_migrations
  File "/usr/lib/python3.5/site-packages/alembic/runtime/environment.py", line 797, in run_migrations
    self.get_context().run_migrations(**kw)
  File "/usr/lib/python3.5/site-packages/alembic/runtime/migration.py", line 312, in run_migrations
    step.migration_fn(**kw)
  File "/usr/lib/python3.5/site-packages/caravel/migrations/versions/960c69cb1f5b_.py", line 18, in upgrade
    op.add_column('table_columns', sa.Column('python_date_format', sa.String(length=255), nullable=True))
  File "<string>", line 8, in add_column
  File "<string>", line 3, in add_column
  File "/usr/lib/python3.5/site-packages/alembic/operations/ops.py", line 1541, in add_column
    return operations.invoke(op)
  File "/usr/lib/python3.5/site-packages/alembic/operations/base.py", line 318, in invoke
    return fn(self, operation)
  File "/usr/lib/python3.5/site-packages/alembic/operations/toimpl.py", line 123, in add_column
    schema=schema
  File "/usr/lib/python3.5/site-packages/alembic/ddl/impl.py", line 172, in add_column
    self._exec(base.AddColumn(table_name, column, schema=schema))
  File "/usr/lib/python3.5/site-packages/alembic/ddl/impl.py", line 118, in _exec
    return conn.execute(construct, *multiparams, **params)
  File "/usr/lib/python3.5/site-packages/sqlalchemy/engine/base.py", line 914, in execute
    return meth(self, multiparams, params)
  File "/usr/lib/python3.5/site-packages/sqlalchemy/sql/ddl.py", line 68, in _execute_on_connection
    return connection._execute_ddl(self, multiparams, params)
  File "/usr/lib/python3.5/site-packages/sqlalchemy/engine/base.py", line 968, in _execute_ddl
    compiled
  File "/usr/lib/python3.5/site-packages/sqlalchemy/engine/base.py", line 1146, in _execute_context
    context)
  File "/usr/lib/python3.5/site-packages/sqlalchemy/engine/base.py", line 1341, in _handle_dbapi_exception
    exc_info
  File "/usr/lib/python3.5/site-packages/sqlalchemy/util/compat.py", line 202, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=cause)
  File "/usr/lib/python3.5/site-packages/sqlalchemy/util/compat.py", line 185, in reraise
    raise value.with_traceback(tb)
  File "/usr/lib/python3.5/site-packages/sqlalchemy/engine/base.py", line 1139, in _execute_context
    context)
  File "/usr/lib/python3.5/site-packages/sqlalchemy/engine/default.py", line 450, in do_execute
    cursor.execute(statement, parameters)
  File "/usr/lib/python3.5/site-packages/MySQLdb/cursors.py", line 226, in execute
    self.errorhandler(self, exc, value)
  File "/usr/lib/python3.5/site-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
    raise errorvalue
  File "/usr/lib/python3.5/site-packages/MySQLdb/cursors.py", line 217, in execute
    res = self._query(query)
  File "/usr/lib/python3.5/site-packages/MySQLdb/cursors.py", line 378, in _query
    rowcount = self._do_query(q)
  File "/usr/lib/python3.5/site-packages/MySQLdb/cursors.py", line 341, in _do_query
    db.query(q)
  File "/usr/lib/python3.5/site-packages/MySQLdb/connections.py", line 280, in query
    _mysql.connection.query(self, query)
sqlalchemy.exc.OperationalError: (_mysql_exceptions.OperationalError) (1060, "Duplicate column name 'python_date_format'") [SQL: 'ALTER TABLE table_columns ADD COLUMN python_date_format VARCHAR(255)']

Setting up Caching

Caravel's own documentation is non-existant but it suggests passing a JSON into the config like so, based on the Flask Cache:

CACHE_CONFIG = {
    'CACHE_TYPE': 'RedisCache',
    'CACHE_REDIS_HOST': '10.0.0.8',
    'CACHE_REDIS_PORT': '6379',
    'CACHE_KEY_PREFIX': '/caravel'
}

Via apache/superset#390

Therefore, I'd assume this should work for the docker:

$ docker run --detach --name caravel -e SECRET_KEY="blah" \
> -e SQLALCHEMY_DATABASE_URI="mysql://blah/caravel_prod" \
> -e CARAVEL_CACHE_CONFIG=\{CACHE_DEFAULT_TIMEOUT:600,CACHE_KEY_PREFIX:'caravel',CACHE_REDIS_HOST:'blah',CACHE_REDIS_PORT:'6379',CACHE_TYPE:'redis'\} \
> -e CARAVEL_MAPBOX_API_KEY=blah \
> --publish 8088:8088 amancevice/caravel

But it throws this when I start the container and check the logs:

$ docker logs caravel
/usr/lib/python3.5/site-packages/flask/exthook.py:71: ExtDeprecationWarning: Importing flask.ext.script is deprecated, use flask_script instead.
  .format(x=modname), ExtDeprecationWarning
/usr/lib/python3.5/site-packages/flask/exthook.py:71: ExtDeprecationWarning: Importing flask.ext.sqlalchemy is deprecated, use flask_sqlalchemy instead.
  .format(x=modname), ExtDeprecationWarning
Traceback (most recent call last):
  File "/usr/bin/caravel", line 15, in <module>
    import caravel
  File "/usr/lib/python3.5/site-packages/caravel/__init__.py", line 25, in <module>
    app.config.from_object(CONFIG_MODULE)
  File "/usr/lib/python3.5/site-packages/flask/config.py", line 163, in from_object
    obj = import_string(obj)
  File "/usr/lib/python3.5/site-packages/werkzeug/utils.py", line 418, in import_string
    __import__(import_name)
  File "/usr/lib/python3.5/site-packages/caravel/config.py", line 186, in <module>
    CACHE_DEFAULT_TIMEOUT = CACHE_CONFIG.get('CACHE_DEFAULT_TIMEOUT')
AttributeError: 'str' object has no attribute 'get'

EDIT: I'd be happy to do a pull request with further instructions setting up Redis Caching.

An unknown error occurred. (Status: 0) Maybe the request timed out?

"An unknown error occurred. (Status: 0) Maybe the request timed out?"

I'm suddenly getting a lot of these when loading graphs. I know this might be a very generic Caravel related issue. Caravel however seems to work fine without Docker on this system.

Kind regards,

Add recommended way to upgrade Docker container

Awesome container... Might be helpful to new Docker users (myself included), to provide some instructions on upgrading the Docker container:

As far as I know, upgrading should be just:

1. Stop existing container: docker stop caravel
2. Remove existing container: docker rm caravel
3. Pull new container from repo: docker pull amacevice/caravel
4. Run new container linking it up to the existing Docker storage on the host used for the previous container (as per instructions on front page)
5. Execute Caravel upgrade function in the container: caravel db upgrade

Is this correct?