instructure / canvas-data-loader Goto Github PK
View Code? Open in Web Editor NEWAn Example Application to download data from Canvas Data, and import it into a Database.
License: MIT License
An Example Application to download data from Canvas Data, and import it into a Database.
License: MIT License
Not really an issue, just more of a question. What is the expected performance of the data loader and what size of dumps has it been tested against?
I'm testing using skip_historical_imports = true
and only_load_final = true
which amounts to a 1.1 GB dump for our institution. I've yet to successfully complete a full load, locally or on AWS. My local host is a very dated Macbook Pro, so I decided to try on AWS. Even with the loader running on d2.xlarge and RDS instances on db.r4.2xlarge, the loads are too slow to be useful.
Before I investigate too much further, I figured I'd touch base to find out more about how CDL has been tested.
As of yesterday I get cdl_runner::db_client: Error(Db(DbError { severity: "ERROR", parsed_severity: Some(Error), code: SqlState("42703"), message: "column \"external_tool_id\" of relation \"assignment_dim\" does not exist", detail: None, hint: None, position: Some(Normal(182)), where_: None, schema: None, table: None, column: None, datatype: None, constraint: None, file: Some("parse_target.c"), line: Some(954), routine: Some("checkInsertTargets") }))
when running the loader.
When running cargo build --release
several errors are thrown for db_client.rs:
the trait r2d2::ManageConnection is not implemented for r2d2_postgres::PostgresConnectionManager
The errors seem to be related to changes in r2d2_postgres 0.14 to support r2d2 0.8.
The script has been running okay, but recently, we started getting this error, here is the full log:
INFO 2018-08-23T11:55:28Z: cdl_runner: Setting up API Client...
INFO 2018-08-23T11:55:28Z: cdl_runner: Connecting to RocksDB Store....
thread 'main' panicked at 'Failed to open RocksDB: Error { message: "IO error: No space left on deviceWhile appending to file: /tmp/cdl-rocksdb/000302.sst: No space left on device" }', /checkout/src/libcore/result.rs:916:5
note: Run with `RUST_BACKTRACE=1` for a backtrace.
We increased the disk space but still getting the same error.
I've managed to work through a bunch of compile errors by installing several other requirements not mentioned in your documentation, but I have come to a point where I can't find a solution. We are running Ubuntu 14.04. Any help would be very much appreciated.
This is the very long error message:
error: failed to run custom build command for librocksdb-sys v5.14.2
process didn't exit successfully: /home/ubuntu/canvas-data-loader/target/release/build/librocksdb-sys-7a8e65a237a57e67/build-script-build
(exit code: 101)
--- stdout
cargo:rerun-if-changed=build.rs
cargo:rerun-if-changed=rocksdb/
cargo:rerun-if-changed=snappy/
TARGET = Some("x86_64-unknown-linux-gnu")
OPT_LEVEL = Some("3")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
TARGET = Some("x86_64-unknown-linux-gnu")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
CXX_x86_64-unknown-linux-gnu = None
CXX_x86_64_unknown_linux_gnu = None
HOST_CXX = None
CXX = None
HOST = Some("x86_64-unknown-linux-gnu")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
CXXFLAGS_x86_64-unknown-linux-gnu = None
CXXFLAGS_x86_64_unknown_linux_gnu = None
HOST_CXXFLAGS = None
CXXFLAGS = None
DEBUG = Some("false")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
CXXFLAGS_x86_64-unknown-linux-gnu = None
CXXFLAGS_x86_64_unknown_linux_gnu = None
HOST_CXXFLAGS = None
CXXFLAGS = None
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
CXXFLAGS_x86_64-unknown-linux-gnu = None
CXXFLAGS_x86_64_unknown_linux_gnu = None
HOST_CXXFLAGS = None
CXXFLAGS = None
running: "c++" "-O3" "-ffunction-sections" "-fdata-sections" "-fPIC" "-m64" "-I" "rocksdb/include/" "-I" "rocksdb/" "-I" "rocksdb/third-party/gtest-1.7.0/fused-src/" "-I" "snappy/" "-I" "." "-Wall" "-Wextra" "-std=c++11" "-Wno-unused-parameter" "-DNDEBUG=1" "-DSNAPPY=1" "-DOS_LINUX=1" "-DROCKSDB_PLATFORM_POSIX=1" "-DROCKSDB_LIB_IO_POSIX=1" "-o" "/home/ubuntu/canvas-data-loader/target/release/build/librocksdb-sys-1366b337e61dd3f0/out/rocksdb/cache/clock_cache.o" "-c" "rocksdb/cache/clock_cache.cc"
exit code: 0
TARGET = Some("x86_64-unknown-linux-gnu")
OPT_LEVEL = Some("3")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
TARGET = Some("x86_64-unknown-linux-gnu")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
CXX_x86_64-unknown-linux-gnu = None
CXX_x86_64_unknown_linux_gnu = None
HOST_CXX = None
CXX = None
HOST = Some("x86_64-unknown-linux-gnu")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
CXXFLAGS_x86_64-unknown-linux-gnu = None
CXXFLAGS_x86_64_unknown_linux_gnu = None
HOST_CXXFLAGS = None
CXXFLAGS = None
DEBUG = Some("false")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
CXXFLAGS_x86_64-unknown-linux-gnu = None
CXXFLAGS_x86_64_unknown_linux_gnu = None
HOST_CXXFLAGS = None
CXXFLAGS = None
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
CXXFLAGS_x86_64-unknown-linux-gnu = None
CXXFLAGS_x86_64_unknown_linux_gnu = None
HOST_CXXFLAGS = None
CXXFLAGS = None
running: "c++" "-O3" "-ffunction-sections" "-fdata-sections" "-fPIC" "-m64" "-I" "rocksdb/include/" "-I" "rocksdb/" "-I" "rocksdb/third-party/gtest-1.7.0/fused-src/" "-I" "snappy/" "-I" "." "-Wall" "-Wextra" "-std=c++11" "-Wno-unused-parameter" "-DNDEBUG=1" "-DSNAPPY=1" "-DOS_LINUX=1" "-DROCKSDB_PLATFORM_POSIX=1" "-DROCKSDB_LIB_IO_POSIX=1" "-o" "/home/ubuntu/canvas-data-loader/target/release/build/librocksdb-sys-1366b337e61dd3f0/out/rocksdb/cache/lru_cache.o" "-c" "rocksdb/cache/lru_cache.cc"
exit code: 0
TARGET = Some("x86_64-unknown-linux-gnu")
OPT_LEVEL = Some("3")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
TARGET = Some("x86_64-unknown-linux-gnu")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
CXX_x86_64-unknown-linux-gnu = None
CXX_x86_64_unknown_linux_gnu = None
HOST_CXX = None
CXX = None
HOST = Some("x86_64-unknown-linux-gnu")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
CXXFLAGS_x86_64-unknown-linux-gnu = None
CXXFLAGS_x86_64_unknown_linux_gnu = None
HOST_CXXFLAGS = None
CXXFLAGS = None
DEBUG = Some("false")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
CXXFLAGS_x86_64-unknown-linux-gnu = None
CXXFLAGS_x86_64_unknown_linux_gnu = None
HOST_CXXFLAGS = None
CXXFLAGS = None
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
CXXFLAGS_x86_64-unknown-linux-gnu = None
CXXFLAGS_x86_64_unknown_linux_gnu = None
HOST_CXXFLAGS = None
CXXFLAGS = None
running: "c++" "-O3" "-ffunction-sections" "-fdata-sections" "-fPIC" "-m64" "-I" "rocksdb/include/" "-I" "rocksdb/" "-I" "rocksdb/third-party/gtest-1.7.0/fused-src/" "-I" "snappy/" "-I" "." "-Wall" "-Wextra" "-std=c++11" "-Wno-unused-parameter" "-DNDEBUG=1" "-DSNAPPY=1" "-DOS_LINUX=1" "-DROCKSDB_PLATFORM_POSIX=1" "-DROCKSDB_LIB_IO_POSIX=1" "-o" "/home/ubuntu/canvas-data-loader/target/release/build/librocksdb-sys-1366b337e61dd3f0/out/rocksdb/cache/sharded_cache.o" "-c" "rocksdb/cache/sharded_cache.cc"
exit code: 0
TARGET = Some("x86_64-unknown-linux-gnu")
OPT_LEVEL = Some("3")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
TARGET = Some("x86_64-unknown-linux-gnu")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
CXX_x86_64-unknown-linux-gnu = None
CXX_x86_64_unknown_linux_gnu = None
HOST_CXX = None
CXX = None
HOST = Some("x86_64-unknown-linux-gnu")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
CXXFLAGS_x86_64-unknown-linux-gnu = None
CXXFLAGS_x86_64_unknown_linux_gnu = None
HOST_CXXFLAGS = None
CXXFLAGS = None
DEBUG = Some("false")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
CXXFLAGS_x86_64-unknown-linux-gnu = None
CXXFLAGS_x86_64_unknown_linux_gnu = None
HOST_CXXFLAGS = None
CXXFLAGS = None
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
CXXFLAGS_x86_64-unknown-linux-gnu = None
CXXFLAGS_x86_64_unknown_linux_gnu = None
HOST_CXXFLAGS = None
CXXFLAGS = None
running: "c++" "-O3" "-ffunction-sections" "-fdata-sections" "-fPIC" "-m64" "-I" "rocksdb/include/" "-I" "rocksdb/" "-I" "rocksdb/third-party/gtest-1.7.0/fused-src/" "-I" "snappy/" "-I" "." "-Wall" "-Wextra" "-std=c++11" "-Wno-unused-parameter" "-DNDEBUG=1" "-DSNAPPY=1" "-DOS_LINUX=1" "-DROCKSDB_PLATFORM_POSIX=1" "-DROCKSDB_LIB_IO_POSIX=1" "-o" "/home/ubuntu/canvas-data-loader/target/release/build/librocksdb-sys-1366b337e61dd3f0/out/rocksdb/db/builder.o" "-c" "rocksdb/db/builder.cc"
exit code: 0
TARGET = Some("x86_64-unknown-linux-gnu")
OPT_LEVEL = Some("3")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
TARGET = Some("x86_64-unknown-linux-gnu")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
CXX_x86_64-unknown-linux-gnu = None
CXX_x86_64_unknown_linux_gnu = None
HOST_CXX = None
CXX = None
HOST = Some("x86_64-unknown-linux-gnu")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
CXXFLAGS_x86_64-unknown-linux-gnu = None
CXXFLAGS_x86_64_unknown_linux_gnu = None
HOST_CXXFLAGS = None
CXXFLAGS = None
DEBUG = Some("false")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
CXXFLAGS_x86_64-unknown-linux-gnu = None
CXXFLAGS_x86_64_unknown_linux_gnu = None
HOST_CXXFLAGS = None
CXXFLAGS = None
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
CXXFLAGS_x86_64-unknown-linux-gnu = None
CXXFLAGS_x86_64_unknown_linux_gnu = None
HOST_CXXFLAGS = None
CXXFLAGS = None
running: "c++" "-O3" "-ffunction-sections" "-fdata-sections" "-fPIC" "-m64" "-I" "rocksdb/include/" "-I" "rocksdb/" "-I" "rocksdb/third-party/gtest-1.7.0/fused-src/" "-I" "snappy/" "-I" "." "-Wall" "-Wextra" "-std=c++11" "-Wno-unused-parameter" "-DNDEBUG=1" "-DSNAPPY=1" "-DOS_LINUX=1" "-DROCKSDB_PLATFORM_POSIX=1" "-DROCKSDB_LIB_IO_POSIX=1" "-o" "/home/ubuntu/canvas-data-loader/target/release/build/librocksdb-sys-1366b337e61dd3f0/out/rocksdb/db/c.o" "-c" "rocksdb/db/c.cc"
exit code: 0
TARGET = Some("x86_64-unknown-linux-gnu")
OPT_LEVEL = Some("3")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
TARGET = Some("x86_64-unknown-linux-gnu")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
CXX_x86_64-unknown-linux-gnu = None
CXX_x86_64_unknown_linux_gnu = None
HOST_CXX = None
CXX = None
HOST = Some("x86_64-unknown-linux-gnu")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
CXXFLAGS_x86_64-unknown-linux-gnu = None
CXXFLAGS_x86_64_unknown_linux_gnu = None
HOST_CXXFLAGS = None
CXXFLAGS = None
DEBUG = Some("false")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
CXXFLAGS_x86_64-unknown-linux-gnu = None
CXXFLAGS_x86_64_unknown_linux_gnu = None
HOST_CXXFLAGS = None
CXXFLAGS = None
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
CXXFLAGS_x86_64-unknown-linux-gnu = None
CXXFLAGS_x86_64_unknown_linux_gnu = None
HOST_CXXFLAGS = None
CXXFLAGS = None
running: "c++" "-O3" "-ffunction-sections" "-fdata-sections" "-fPIC" "-m64" "-I" "rocksdb/include/" "-I" "rocksdb/" "-I" "rocksdb/third-party/gtest-1.7.0/fused-src/" "-I" "snappy/" "-I" "." "-Wall" "-Wextra" "-std=c++11" "-Wno-unused-parameter" "-DNDEBUG=1" "-DSNAPPY=1" "-DOS_LINUX=1" "-DROCKSDB_PLATFORM_POSIX=1" "-DROCKSDB_LIB_IO_POSIX=1" "-o" "/home/ubuntu/canvas-data-loader/target/release/build/librocksdb-sys-1366b337e61dd3f0/out/rocksdb/db/column_family.o" "-c" "rocksdb/db/column_family.cc"
exit code: 0
TARGET = Some("x86_64-unknown-linux-gnu")
OPT_LEVEL = Some("3")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
TARGET = Some("x86_64-unknown-linux-gnu")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
CXX_x86_64-unknown-linux-gnu = None
CXX_x86_64_unknown_linux_gnu = None
HOST_CXX = None
CXX = None
HOST = Some("x86_64-unknown-linux-gnu")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
CXXFLAGS_x86_64-unknown-linux-gnu = None
CXXFLAGS_x86_64_unknown_linux_gnu = None
HOST_CXXFLAGS = None
CXXFLAGS = None
DEBUG = Some("false")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
CXXFLAGS_x86_64-unknown-linux-gnu = None
CXXFLAGS_x86_64_unknown_linux_gnu = None
HOST_CXXFLAGS = None
CXXFLAGS = None
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
CXXFLAGS_x86_64-unknown-linux-gnu = None
CXXFLAGS_x86_64_unknown_linux_gnu = None
HOST_CXXFLAGS = None
CXXFLAGS = None
running: "c++" "-O3" "-ffunction-sections" "-fdata-sections" "-fPIC" "-m64" "-I" "rocksdb/include/" "-I" "rocksdb/" "-I" "rocksdb/third-party/gtest-1.7.0/fused-src/" "-I" "snappy/" "-I" "." "-Wall" "-Wextra" "-std=c++11" "-Wno-unused-parameter" "-DNDEBUG=1" "-DSNAPPY=1" "-DOS_LINUX=1" "-DROCKSDB_PLATFORM_POSIX=1" "-DROCKSDB_LIB_IO_POSIX=1" "-o" "/home/ubuntu/canvas-data-loader/target/release/build/librocksdb-sys-1366b337e61dd3f0/out/rocksdb/db/compacted_db_impl.o" "-c" "rocksdb/db/compacted_db_impl.cc"
exit code: 0
TARGET = Some("x86_64-unknown-linux-gnu")
OPT_LEVEL = Some("3")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
TARGET = Some("x86_64-unknown-linux-gnu")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
CXX_x86_64-unknown-linux-gnu = None
CXX_x86_64_unknown_linux_gnu = None
HOST_CXX = None
CXX = None
HOST = Some("x86_64-unknown-linux-gnu")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
CXXFLAGS_x86_64-unknown-linux-gnu = None
CXXFLAGS_x86_64_unknown_linux_gnu = None
HOST_CXXFLAGS = None
CXXFLAGS = None
DEBUG = Some("false")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
CXXFLAGS_x86_64-unknown-linux-gnu = None
CXXFLAGS_x86_64_unknown_linux_gnu = None
HOST_CXXFLAGS = None
CXXFLAGS = None
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
CXXFLAGS_x86_64-unknown-linux-gnu = None
CXXFLAGS_x86_64_unknown_linux_gnu = None
HOST_CXXFLAGS = None
CXXFLAGS = None
running: "c++" "-O3" "-ffunction-sections" "-fdata-sections" "-fPIC" "-m64" "-I" "rocksdb/include/" "-I" "rocksdb/" "-I" "rocksdb/third-party/gtest-1.7.0/fused-src/" "-I" "snappy/" "-I" "." "-Wall" "-Wextra" "-std=c++11" "-Wno-unused-parameter" "-DNDEBUG=1" "-DSNAPPY=1" "-DOS_LINUX=1" "-DROCKSDB_PLATFORM_POSIX=1" "-DROCKSDB_LIB_IO_POSIX=1" "-o" "/home/ubuntu/canvas-data-loader/target/release/build/librocksdb-sys-1366b337e61dd3f0/out/rocksdb/db/compaction.o" "-c" "rocksdb/db/compaction.cc"
exit code: 0
TARGET = Some("x86_64-unknown-linux-gnu")
OPT_LEVEL = Some("3")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
TARGET = Some("x86_64-unknown-linux-gnu")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
CXX_x86_64-unknown-linux-gnu = None
CXX_x86_64_unknown_linux_gnu = None
HOST_CXX = None
CXX = None
HOST = Some("x86_64-unknown-linux-gnu")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
CXXFLAGS_x86_64-unknown-linux-gnu = None
CXXFLAGS_x86_64_unknown_linux_gnu = None
HOST_CXXFLAGS = None
CXXFLAGS = None
DEBUG = Some("false")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
CXXFLAGS_x86_64-unknown-linux-gnu = None
CXXFLAGS_x86_64_unknown_linux_gnu = None
HOST_CXXFLAGS = None
CXXFLAGS = None
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
CXXFLAGS_x86_64-unknown-linux-gnu = None
CXXFLAGS_x86_64_unknown_linux_gnu = None
HOST_CXXFLAGS = None
CXXFLAGS = None
running: "c++" "-O3" "-ffunction-sections" "-fdata-sections" "-fPIC" "-m64" "-I" "rocksdb/include/" "-I" "rocksdb/" "-I" "rocksdb/third-party/gtest-1.7.0/fused-src/" "-I" "snappy/" "-I" "." "-Wall" "-Wextra" "-std=c++11" "-Wno-unused-parameter" "-DNDEBUG=1" "-DSNAPPY=1" "-DOS_LINUX=1" "-DROCKSDB_PLATFORM_POSIX=1" "-DROCKSDB_LIB_IO_POSIX=1" "-o" "/home/ubuntu/canvas-data-loader/target/release/build/librocksdb-sys-1366b337e61dd3f0/out/rocksdb/db/compaction_iterator.o" "-c" "rocksdb/db/compaction_iterator.cc"
exit code: 0
TARGET = Some("x86_64-unknown-linux-gnu")
OPT_LEVEL = Some("3")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
TARGET = Some("x86_64-unknown-linux-gnu")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
CXX_x86_64-unknown-linux-gnu = None
CXX_x86_64_unknown_linux_gnu = None
HOST_CXX = None
CXX = None
HOST = Some("x86_64-unknown-linux-gnu")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
CXXFLAGS_x86_64-unknown-linux-gnu = None
CXXFLAGS_x86_64_unknown_linux_gnu = None
HOST_CXXFLAGS = None
CXXFLAGS = None
DEBUG = Some("false")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
CXXFLAGS_x86_64-unknown-linux-gnu = None
CXXFLAGS_x86_64_unknown_linux_gnu = None
HOST_CXXFLAGS = None
CXXFLAGS = None
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
CXXFLAGS_x86_64-unknown-linux-gnu = None
CXXFLAGS_x86_64_unknown_linux_gnu = None
HOST_CXXFLAGS = None
CXXFLAGS = None
running: "c++" "-O3" "-ffunction-sections" "-fdata-sections" "-fPIC" "-m64" "-I" "rocksdb/include/" "-I" "rocksdb/" "-I" "rocksdb/third-party/gtest-1.7.0/fused-src/" "-I" "snappy/" "-I" "." "-Wall" "-Wextra" "-std=c++11" "-Wno-unused-parameter" "-DNDEBUG=1" "-DSNAPPY=1" "-DOS_LINUX=1" "-DROCKSDB_PLATFORM_POSIX=1" "-DROCKSDB_LIB_IO_POSIX=1" "-o" "/home/ubuntu/canvas-data-loader/target/release/build/librocksdb-sys-1366b337e61dd3f0/out/rocksdb/db/compaction_job.o" "-c" "rocksdb/db/compaction_job.cc"
exit code: 0
TARGET = Some("x86_64-unknown-linux-gnu")
OPT_LEVEL = Some("3")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
TARGET = Some("x86_64-unknown-linux-gnu")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
CXX_x86_64-unknown-linux-gnu = None
CXX_x86_64_unknown_linux_gnu = None
HOST_CXX = None
CXX = None
HOST = Some("x86_64-unknown-linux-gnu")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
CXXFLAGS_x86_64-unknown-linux-gnu = None
CXXFLAGS_x86_64_unknown_linux_gnu = None
HOST_CXXFLAGS = None
CXXFLAGS = None
DEBUG = Some("false")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
CXXFLAGS_x86_64-unknown-linux-gnu = None
CXXFLAGS_x86_64_unknown_linux_gnu = None
HOST_CXXFLAGS = None
CXXFLAGS = None
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
CXXFLAGS_x86_64-unknown-linux-gnu = None
CXXFLAGS_x86_64_unknown_linux_gnu = None
HOST_CXXFLAGS = None
CXXFLAGS = None
running: "c++" "-O3" "-ffunction-sections" "-fdata-sections" "-fPIC" "-m64" "-I" "rocksdb/include/" "-I" "rocksdb/" "-I" "rocksdb/third-party/gtest-1.7.0/fused-src/" "-I" "snappy/" "-I" "." "-Wall" "-Wextra" "-std=c++11" "-Wno-unused-parameter" "-DNDEBUG=1" "-DSNAPPY=1" "-DOS_LINUX=1" "-DROCKSDB_PLATFORM_POSIX=1" "-DROCKSDB_LIB_IO_POSIX=1" "-o" "/home/ubuntu/canvas-data-loader/target/release/build/librocksdb-sys-1366b337e61dd3f0/out/rocksdb/db/compaction_picker.o" "-c" "rocksdb/db/compaction_picker.cc"
exit code: 0
TARGET = Some("x86_64-unknown-linux-gnu")
OPT_LEVEL = Some("3")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
TARGET = Some("x86_64-unknown-linux-gnu")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
CXX_x86_64-unknown-linux-gnu = None
CXX_x86_64_unknown_linux_gnu = None
HOST_CXX = None
CXX = None
HOST = Some("x86_64-unknown-linux-gnu")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
CXXFLAGS_x86_64-unknown-linux-gnu = None
CXXFLAGS_x86_64_unknown_linux_gnu = None
HOST_CXXFLAGS = None
CXXFLAGS = None
DEBUG = Some("false")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
CXXFLAGS_x86_64-unknown-linux-gnu = None
CXXFLAGS_x86_64_unknown_linux_gnu = None
HOST_CXXFLAGS = None
CXXFLAGS = None
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
CXXFLAGS_x86_64-unknown-linux-gnu = None
CXXFLAGS_x86_64_unknown_linux_gnu = None
HOST_CXXFLAGS = None
CXXFLAGS = None
running: "c++" "-O3" "-ffunction-sections" "-fdata-sections" "-fPIC" "-m64" "-I" "rocksdb/include/" "-I" "rocksdb/" "-I" "rocksdb/third-party/gtest-1.7.0/fused-src/" "-I" "snappy/" "-I" "." "-Wall" "-Wextra" "-std=c++11" "-Wno-unused-parameter" "-DNDEBUG=1" "-DSNAPPY=1" "-DOS_LINUX=1" "-DROCKSDB_PLATFORM_POSIX=1" "-DROCKSDB_LIB_IO_POSIX=1" "-o" "/home/ubuntu/canvas-data-loader/target/release/build/librocksdb-sys-1366b337e61dd3f0/out/rocksdb/db/compaction_picker_universal.o" "-c" "rocksdb/db/compaction_picker_universal.cc"
exit code: 0
TARGET = Some("x86_64-unknown-linux-gnu")
OPT_LEVEL = Some("3")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
TARGET = Some("x86_64-unknown-linux-gnu")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
CXX_x86_64-unknown-linux-gnu = None
CXX_x86_64_unknown_linux_gnu = None
HOST_CXX = None
CXX = None
HOST = Some("x86_64-unknown-linux-gnu")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
CXXFLAGS_x86_64-unknown-linux-gnu = None
CXXFLAGS_x86_64_unknown_linux_gnu = None
HOST_CXXFLAGS = None
CXXFLAGS = None
DEBUG = Some("false")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
CXXFLAGS_x86_64-unknown-linux-gnu = None
CXXFLAGS_x86_64_unknown_linux_gnu = None
HOST_CXXFLAGS = None
CXXFLAGS = None
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
CXXFLAGS_x86_64-unknown-linux-gnu = None
CXXFLAGS_x86_64_unknown_linux_gnu = None
HOST_CXXFLAGS = None
CXXFLAGS = None
running: "c++" "-O3" "-ffunction-sections" "-fdata-sections" "-fPIC" "-m64" "-I" "rocksdb/include/" "-I" "rocksdb/" "-I" "rocksdb/third-party/gtest-1.7.0/fused-src/" "-I" "snappy/" "-I" "." "-Wall" "-Wextra" "-std=c++11" "-Wno-unused-parameter" "-DNDEBUG=1" "-DSNAPPY=1" "-DOS_LINUX=1" "-DROCKSDB_PLATFORM_POSIX=1" "-DROCKSDB_LIB_IO_POSIX=1" "-o" "/home/ubuntu/canvas-data-loader/target/release/build/librocksdb-sys-1366b337e61dd3f0/out/rocksdb/db/convenience.o" "-c" "rocksdb/db/convenience.cc"
exit code: 0
TARGET = Some("x86_64-unknown-linux-gnu")
OPT_LEVEL = Some("3")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
TARGET = Some("x86_64-unknown-linux-gnu")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
CXX_x86_64-unknown-linux-gnu = None
CXX_x86_64_unknown_linux_gnu = None
HOST_CXX = None
CXX = None
HOST = Some("x86_64-unknown-linux-gnu")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
CXXFLAGS_x86_64-unknown-linux-gnu = None
CXXFLAGS_x86_64_unknown_linux_gnu = None
HOST_CXXFLAGS = None
CXXFLAGS = None
DEBUG = Some("false")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
CXXFLAGS_x86_64-unknown-linux-gnu = None
CXXFLAGS_x86_64_unknown_linux_gnu = None
HOST_CXXFLAGS = None
CXXFLAGS = None
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
CXXFLAGS_x86_64-unknown-linux-gnu = None
CXXFLAGS_x86_64_unknown_linux_gnu = None
HOST_CXXFLAGS = None
CXXFLAGS = None
running: "c++" "-O3" "-ffunction-sections" "-fdata-sections" "-fPIC" "-m64" "-I" "rocksdb/include/" "-I" "rocksdb/" "-I" "rocksdb/third-party/gtest-1.7.0/fused-src/" "-I" "snappy/" "-I" "." "-Wall" "-Wextra" "-std=c++11" "-Wno-unused-parameter" "-DNDEBUG=1" "-DSNAPPY=1" "-DOS_LINUX=1" "-DROCKSDB_PLATFORM_POSIX=1" "-DROCKSDB_LIB_IO_POSIX=1" "-o" "/home/ubuntu/canvas-data-loader/target/release/build/librocksdb-sys-1366b337e61dd3f0/out/rocksdb/db/db_filesnapshot.o" "-c" "rocksdb/db/db_filesnapshot.cc"
exit code: 0
TARGET = Some("x86_64-unknown-linux-gnu")
OPT_LEVEL = Some("3")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
TARGET = Some("x86_64-unknown-linux-gnu")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
CXX_x86_64-unknown-linux-gnu = None
CXX_x86_64_unknown_linux_gnu = None
HOST_CXX = None
CXX = None
HOST = Some("x86_64-unknown-linux-gnu")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
CXXFLAGS_x86_64-unknown-linux-gnu = None
CXXFLAGS_x86_64_unknown_linux_gnu = None
HOST_CXXFLAGS = None
CXXFLAGS = None
DEBUG = Some("false")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
CXXFLAGS_x86_64-unknown-linux-gnu = None
CXXFLAGS_x86_64_unknown_linux_gnu = None
HOST_CXXFLAGS = None
CXXFLAGS = None
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
CXXFLAGS_x86_64-unknown-linux-gnu = None
CXXFLAGS_x86_64_unknown_linux_gnu = None
HOST_CXXFLAGS = None
CXXFLAGS = None
running: "c++" "-O3" "-ffunction-sections" "-fdata-sections" "-fPIC" "-m64" "-I" "rocksdb/include/" "-I" "rocksdb/" "-I" "rocksdb/third-party/gtest-1.7.0/fused-src/" "-I" "snappy/" "-I" "." "-Wall" "-Wextra" "-std=c++11" "-Wno-unused-parameter" "-DNDEBUG=1" "-DSNAPPY=1" "-DOS_LINUX=1" "-DROCKSDB_PLATFORM_POSIX=1" "-DROCKSDB_LIB_IO_POSIX=1" "-o" "/home/ubuntu/canvas-data-loader/target/release/build/librocksdb-sys-1366b337e61dd3f0/out/rocksdb/db/db_impl.o" "-c" "rocksdb/db/db_impl.cc"
cargo:warning=c++: internal compiler error: Killed (program cc1plus)
cargo:warning=Please submit a full bug report,
cargo:warning=with preprocessed source if appropriate.
cargo:warning=See file:///usr/share/doc/gcc-4.8/README.Bugs for instructions.
exit code: 4
--- stderr
rocksdb/include/rocksdb/c.h:48:9: warning: #pragma once in main file [-Wpragma-once-outside-header], err: false
thread 'main' panicked at '
Internal error occurred: Command "c++" "-O3" "-ffunction-sections" "-fdata-sections" "-fPIC" "-m64" "-I" "rocksdb/include/" "-I" "rocksdb/" "-I" "rocksdb/third-party/gtest-1.7.0/fused-src/" "-I" "snappy/" "-I" "." "-Wall" "-Wextra" "-std=c++11" "-Wno-unused-parameter" "-DNDEBUG=1" "-DSNAPPY=1" "-DOS_LINUX=1" "-DROCKSDB_PLATFORM_POSIX=1" "-DROCKSDB_LIB_IO_POSIX=1" "-o" "/home/ubuntu/canvas-data-loader/target/release/build/librocksdb-sys-1366b337e61dd3f0/out/rocksdb/db/db_impl.o" "-c" "rocksdb/db/db_impl.cc" with args "c++" did not execute successfully (status code exit code: 4).
', /home/ubuntu/.cargo/registry/src/github.com-1ecc6299db9ec823/cc-1.0.23/src/lib.rs:2253:5
note: Run with RUST_BACKTRACE=1
for a backtrace.
--- Delete everything above this line ---
Installed all dependencies and database(s), ran import, waited almost 2 days for it to complete.
INFO 2019-10-04T22:38:17Z: cdl_runner: Connecting to the DB
ERROR 2019-10-06T13:15:22Z: cdl_runner::db_client: insert error
ERROR 2019-10-06T13:15:22Z: cdl_runner::db_client: MySqlError { ERROR 1264 (22003): Out of range value for column '(null)' at row 1 }
ERROR 2019-10-06T13:15:22Z: cdl_runner::importer: process -> for line in finalized_string -> !is_volatile -> ins_res -> is_err
ERROR 2019-10-06T13:15:22Z: cdl_runner::importer: Error(MysqlErr, State { next_error: None, backtrace: InternalBacktrace { backtrace: None } })
RUST_LOG=trace
at the beginning and provide us the debug logs.)I have installed the Canvas-data-loader on a Windows 10 platform.
Version of rust
rustc 1.32.0 (9fda7c223 2019-01-16)
When I get to step 11 from the command prompt.
I have changed directories to canvas-data-loader-master
Everything compiled properly
Compiling librocksdb-sys v5.14.3
Compiling rocksdb v0.10.1
Compiling cdl-runner v0.4.0 (I:\CanvasData\canvas-data-loader-master)
Finished release [optimized] target(s) in 7m 00s
Is there something else that needs to be installed on a Windows machine?
Hello @securityinsanity, @UniversalSuperB made a remark on finding out that some id columns gets padded with 555555. I believe this is why I get the following error when I tried to run this command:
[root@localhost home]# RUST_LOG=info ./target/release/cdl-runner
Error:
INFO 2018-11-13T16:42:52Z: cdl_runner: Setting up API Client...
INFO 2018-11-13T16:42:52Z: cdl_runner: Connecting to RocksDB Store....
INFO 2018-11-13T16:42:52Z: cdl_runner: Skipping dump: f2861f59-f11c-4c97-8d09-c349e7f4f3f6
INFO 2018-11-13T16:42:52Z: cdl_runner: Skipping dump: 259dfc10-69b7-4f9f-bd89-013082e6c1ef due to only final selected
INFO 2018-11-13T16:42:52Z: cdl_runner: Skipping dump: a8d4271f-2013-4ff0-8813-b8aa79e85e0b due to only final selected
INFO 2018-11-13T16:42:53Z: cdl_runner: Connecting to the DB
ERROR 2018-11-13T16:50:35Z: cdl_runner::db_client: insert error
ERROR 2018-11-13T16:50:35Z: cdl_runner::db_client: MySqlError { ERROR 1264 (22003): Out of range value for column 'position' at row 1 }
ERROR 2018-11-13T16:50:35Z: cdl_runner::importer: process -> for line in finalized_string -> !is_volatile -> ins_res -> is_err
ERROR 2018-11-13T16:50:35Z: cdl_runner::importer: Error(MysqlErr, State { next_error: None, backtrace: None })
INFO 2018-11-13T16:50:35Z: cdl_runner: Done!
My question is: Is there a fix for this?
Originally posted by @latifatbraimah in #6 (comment)
Encountering some odd behaviour with Canvas Data Loader. CDL runs but files aren't downloaded and no data is loaded into the database. DumpIDs are recorded in rocksdb and it opens a connection to postgres, but with no files downloaded there's nothing to load.
I've never used Rust, but I've managed to trace it to api_client.rs:
let uri = try!(file_to_download.url.parse());
work_to_do.push(client.get(uri).and_then(move |res| {
let download_path = cloned_download_path;
let mut file = File::create(Path::new(&download_path)).expect("Failed to create download file");
res.body().for_each(move |chunk| {
let _ = file.write(&*chunk);
Ok(())
})
})
);
}
}
}
try!(core.run(join_all(work_to_do)));
If I modify core.run to use expect instead of try: core.run(join_all(work_to_do)).expect("Failed");
an Io Error is thrown.
thread 'main' panicked at 'Failed: Io(Error { repr: Custom(Custom { kind: Other, error: StringError("failed to lookup address information: nodename nor servname provided, or not known") }) })', src/libcore/result.rs:906:4
stack backtrace:
0: std::sys::imp::backtrace::tracing::imp::unwind_backtrace
1: std::sys_common::backtrace::_print
2: std::panicking::default_hook::{{closure}}
3: std::panicking::default_hook
4: std::panicking::rust_panic_with_hook
5: std::panicking::begin_panic
6: std::panicking::begin_panic_fmt
7: rust_begin_unwind
8: core::panicking::panic_fmt
9: core::result::unwrap_failed
10: cdl_runner::api_client::CanvasDataApiClient::download_files_for_dump
11: cdl_runner::main::{{closure}}
12: cdl_runner::main
13: __rust_maybe_catch_panic
14: std::rt::lang_start
It seems that I can get CDL working if I download each file individually:
let mut work = client.get(uri).and_then(move |res| {
let download_path = cloned_download_path;
let mut file = File::create(Path::new(&download_path)).expect("Failed to create download file");
res.body().for_each(move |chunk| {
let _ = file.write(&*chunk);
Ok(())
})
});
try!(core.run(work));
I'm happy to chase this down further, but being new to Rust, I could use some guidance. The lookup failure just seems particularly weird to me.
When I run cargo build --release, I am receiving the following error:
= note: liblibrocksdb_sys-4fe83a8a2d9f911a.rlib(env_win.o) : error LNK2019: unresolved external symbol __imp_PathIsRelativeA referenced in function "public: virtual class rocksdb::Status __cdecl rocksdb::port::WinEnvIO::GetAbsolutePath(class std::basic_string<char,struct std::char_traits,class std::allocator > const &,class std::basic_string<char,struct std::char_traits,class std::allocator > *)" (?GetAbsolutePath@WinEnvIO@port@rocksdb@@UEAA?AVStatus@3@AEBV?$basic_string@DU?$char_traits@D@std@@v?$allocator@D@2@@std@@PEAV56@@z)
C:\canvas-data-loader\target\release\deps\cdl_runner-7e809c944750d409.exe : fatal error LNK1120: 1 unresolved externals
error: aborting due to previous error
error: Could not compile cdl-runner
.
When I run 'cargo build --release' I get the following error:
error[E0599]: no method named
expectfound for type
()` in the current scope
--> src/main.rs:57:22
|
57 | env_logger::init().expect("Failed to setup logger!");
| ^^^^^^
error: aborting due to previous error
error: Could not compile cdl-runner
.`
I am not proficient in rust and am not sure where to start debugging.
Thanks in advance for any help.
I am following the steps in canvas DOC-11943
at Step 9, I can see that %USERPROFILE%.cargo\bin is added to my Windows USER path instead of SYSTEM path.
next, I don't know what "to configure current shell run" mean. I blindly followed and executed source "%USERPROFILE%.cargo\env" and it gave this error
'source' is not recognized as an internal or external command,
operable program or batch file.
probably there should be some window's way of doing it that I don't know.
at Step 10, I initially faced Virtual studio error and then I installed Build tools with C++ and now building release version it gave me following error
[2019-06-12T19:35:34Z INFO cargo::core::compiler::job_queue] start: librocksdb-sys v5.18.3 => Target(script)/Profile() => Host
Compiling librocksdb-sys v5.18.3
[2019-06-12T19:35:34Z INFO cargo::core::compiler::job_queue] start: cookie_store v0.7.0 => Target(lib)/Profile(release) => Host
[2019-06-12T19:35:34Z INFO cargo::core::compiler::job_queue] end: cookie_store v0.7.0 => Target(lib)/Profile(release) => Host
[2019-06-12T19:35:34Z INFO cargo::core::compiler::job_queue] start: reqwest v0.9.18 => Target(lib)/Profile(release) => Host
[2019-06-12T19:35:34Z INFO cargo::core::compiler::job_queue] end: reqwest v0.9.18 => Target(lib)/Profile(release) => Host
[2019-06-12T19:35:34Z INFO cargo::core::compiler::job_queue] end: librocksdb-sys v5.18.3 => Target(script)/Profile() => Host
error: failed to run custom build command for `librocksdb-sys v5.18.3`
process didn't exit successfully: `C:\Softwares\CanvasDataLoader\canvas-data-loader\target\release\build\librocksdb-sys-10a20a9d59bb2f10\build-script-build` (exit code: 101)
--- stdout
cargo:rerun-if-changed=build.rs
cargo:rerun-if-changed=rocksdb/
cargo:rerun-if-changed=snappy/
cargo:rerun-if-changed=lz4/
cargo:rerun-if-changed=zstd/
cargo:rerun-if-changed=zlib/
cargo:rerun-if-changed=bzip2/
--- stderr
thread 'main' panicked at 'Unable to find libclang: "couldn\'t find any valid shared libraries matching: [\'clang.dll\', \'libclang.dll\'], set the `LIBCLANG_PATH` environment variable to a path where one of these files can be found (invalid: [])"', src\libcore\result.rs:997:5
note: Run with `RUST_BACKTRACE=1` environment variable to display a backtrace.
C:\Softwares\CanvasDataLoader\canvas-data-loader>
I am not sure if I didn't do something right in step 9 which gave me this error in 10 or something else. but I am stuck and cannot move forward.
do I need to wait for successful completion of this step to see cdl-runner.bat/cdl-runner.exe file under my C:\Softwares\CanvasDataLoader\canvas-data-loader\target\release directory?
because I am not seeing anything currently
rustc 1.35.0 (3c235d560 2019-05-20)
Windows 10 Desktop
*Logs:
Attached the cargo --build --verbose
BuildOutput.txt
The biggest bottlenecks of the loader are really:
All of these facts do slow it down considerably, and we've in the past avoided fixing these bottlenecks because we don't want people to rely on this full time working with Canvas Data. It's meant to be an example application showing users how to interact with the API, and how they could do things like "avoid historical refreshes".
We could one day down the road, fix these features, and turn the CDL into a fully supported solution, but that would have to be a product decision.
Originally posted by @securityinsanity in #6 (comment)
@securityinsanity, @UniversalsuperB made a remark on finding out that some id columns gets padded with 555555. I believe this is why I get the following error when I tried to run this command:
[root@localhost home]# RUST_LOG=info ./target/release/cdl-runner
Error:
INFO 2018-11-13T16:42:52Z: cdl_runner: Setting up API Client...
INFO 2018-11-13T16:42:52Z: cdl_runner: Connecting to RocksDB Store....
INFO 2018-11-13T16:42:52Z: cdl_runner: Skipping dump: f2861f59-f11c-4c97-8d09-c349e7f4f3f6
INFO 2018-11-13T16:42:52Z: cdl_runner: Skipping dump: 259dfc10-69b7-4f9f-bd89-013082e6c1ef due to only final selected
INFO 2018-11-13T16:42:52Z: cdl_runner: Skipping dump: a8d4271f-2013-4ff0-8813-b8aa79e85e0b due to only final selected
INFO 2018-11-13T16:42:53Z: cdl_runner: Connecting to the DB
ERROR 2018-11-13T16:50:35Z: cdl_runner::db_client: insert error
ERROR 2018-11-13T16:50:35Z: cdl_runner::db_client: MySqlError { ERROR 1264 (22003): Out of range value for column 'position' at row 1 }
ERROR 2018-11-13T16:50:35Z: cdl_runner::importer: process -> for line in finalized_string -> !is_volatile -> ins_res -> is_err
ERROR 2018-11-13T16:50:35Z: cdl_runner::importer: Error(MysqlErr, State { next_error: None, backtrace: None })
INFO 2018-11-13T16:50:35Z: cdl_runner: Done!
My question is: Is there a fix for this?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.