snowflakedb / snowflake-connector-python Goto Github PK
View Code? Open in Web Editor NEWSnowflake Connector for Python
Home Page: https://pypi.python.org/pypi/snowflake-connector-python/
License: Apache License 2.0
Snowflake Connector for Python
Home Page: https://pypi.python.org/pypi/snowflake-connector-python/
License: Apache License 2.0
When attempting to connect to the snowflake engine:
import snowflake.connector
cnx = snowflake.connector.connect(user=***, password=***, account=***)
I get the following stack trace. Take note of line 3 who is hitting error?
which shows up here:
snowflake-connector-python/network.py
Line 760 in 13cf42c
Snowflake Connector for Python Version: 1.3.17, Python Version: 2.7.12, Platform: Linux-4.4.0-45-generic-x86_64-with-Ubuntu-16.04-xenial
Starting new HTTPS connection (1): ***.snowflakecomputing.com
who is hitting error?
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/snowflake/connector/network.py", line 704, in request_exec
auth=SnowflakeAuth(token),
File "/usr/local/lib/python2.7/dist-packages/botocore/vendored/requests/sessions.py", line 465, in request
resp = self.send(prep, **send_kwargs)
File "/usr/local/lib/python2.7/dist-packages/botocore/vendored/requests/sessions.py", line 573, in send
r = adapter.send(request, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/botocore/vendored/requests/adapters.py", line 370, in send
timeout=timeout
File "/usr/local/lib/python2.7/dist-packages/botocore/vendored/requests/packages/urllib3/connectionpool.py", line 544, in urlopen
body=body, headers=headers)
File "/usr/local/lib/python2.7/dist-packages/botocore/vendored/requests/packages/urllib3/connectionpool.py", line 341, in _make_request
self._validate_conn(conn)
File "/usr/local/lib/python2.7/dist-packages/botocore/vendored/requests/packages/urllib3/connectionpool.py", line 761, in _validate_conn
conn.connect()
File "/usr/local/lib/python2.7/dist-packages/botocore/vendored/requests/packages/urllib3/connection.py", line 238, in connect
ssl_version=resolved_ssl_version)
File "/usr/local/lib/python2.7/dist-packages/snowflake/connector/ssl_wrap_socket.py", line 469, in ssl_wrap_socket_with_ocsp
ssl_version=ssl_version)
File "/usr/local/lib/python2.7/dist-packages/snowflake/connector/ssl_wrap_socket.py", line 452, in ssl_wrap_socket
rd, _, _ = select.select([sock], [], [], sock.gettimeout())
ValueError: filedescriptor out of range in select()
I was able to remedy the issue by restarting the container.
Snowflake Connector for Python Version: 1.3.14
Python Version: 2.7.13
Platform: Darwin-16.4.0-x86_64-i386-64bit
According to the docs, the following code snippet should work:
self.cnx.cursor().execute("USE ROLE %s", self.sf_role)
However, when the execute()
method parses the parameter, it wraps it in single quote so that the resulting SQL query is USE ROLE 'my_role'
, which is invalid syntax. Specifically: syntax error line 1 at position 9 unexpected ''my_role''.
Even though Python-style formatting, e.g., self.cnx.cursor().execute("USE ROLE %s" % self.sf_role)
, is discouraged because of SQL injection risks, there doesn't seem to be any other way around it.
I created a test table with this query...
create or replace table perf_1m(seq, text, ts TIMESTAMP, f FLOAT, d DECIMAL) as select seq8(), randstr(100, random()), TO_TIMESTAMP(1400000000 * abs(random() / random(1))/ 10), random(), TO_DECIMAL(random() * 1121.1121212214) from table(generator(rowcount=>1 * 1000 * 1000));
Attempting to SELECT from that table generates the following exception.
Command 'main' error, traceback...
Traceback (most recent call last)
2. File "/usr/lib/python3.5/site-packages/snowflake_connector_python-1.3.12-py3.5.egg/snowflake/connector/cursor.py", line 837, in _row_to_python
row[idx] = conv_method(col, conv_ctx)
1. File "/usr/lib/python3.5/site-packages/snowflake_connector_python-1.3.12-py3.5.egg/snowflake/connector/converter.py", line 295, in _TIMESTAMP_NTZ_to_python
t = ZERO_EPOCH + timedelta(seconds=(microseconds / float(1000000)))
OverflowError: date value out of range
During handling of the above exception, another exception occurred:
Traceback (most recent call last)
8. File "/home/mayfield/project/shellish/shellish/session.py", line 116, in execute
raise e
7. File "/home/mayfield/project/shellish/shellish/session.py", line 113, in execute
result = command.run_wrap(args)
6. File "/home/mayfield/project/shellish/shellish/command/command.py", line 169, in run_wrap
raise e
5. File "/home/mayfield/project/shellish/shellish/command/command.py", line 165, in run_wrap
result = self.run(args)
4. File "/home/mayfield/project/shellish/shellish/command/autocommand.py", line 52, in run
return self.func(*positionals, **keywords)
3. File "bench.py", line 54, in main
while q.fetchone() is not None:
2. File "/usr/lib/python3.5/site-packages/snowflake_connector_python-1.3.12-py3.5.egg/snowflake/connector/cursor.py", line 671, in fetchone
return self._row_to_python(row) if row is not None else None
1. File "/usr/lib/python3.5/site-packages/snowflake_connector_python-1.3.12-py3.5.egg/snowflake/connector/cursor.py", line 844, in _row_to_python
type=FIELD_ID_TO_NAME(col_desc[1]),
TypeError: 'collections.defaultdict' object is not callable
I'm seeing intermittent hangs with the new hillclimbing cursor_algo in AWS. It occurs with big and small instances in various regions. For some reason it does not happen outside AWS so I think AWS may enforce stricter throttling for internal machines accessing S3..
root@ip-172-31-47-83:~/snowflake-connector-python# time python3.6 /bench/bench.py --limit 10000000 --noprof --latency 0 --jitter 10 --simple
[2017-04-15 19:25:09,524] [250] [MainThread] [snowflake.connector.connection] [INFO] Snowflake Connector for Python Version: 1.3.15, Python Version: 3.6.1, Platform: Linux-4.9.17-8.31.amzn1.x86_64-x86_64-with-debian-8.7
[2017-04-15 19:25:09,524] [250] [MainThread] [snowflake.connector.connection] [DEBUG] connect
[2017-04-15 19:25:09,525] [250] [MainThread] [snowflake.connector.connection] [DEBUG] __config
[2017-04-15 19:25:09,525] [250] [MainThread] [snowflake.connector.connection] [INFO] THIS CONNECTION IS IN INSECURE MODE. IT MEANS THE CERTIFICATE WILL BE VALIDATED BUT THE CERTIFICATE REVOCATION STATUS WILL NOT BE CHECKED.
[2017-04-15 19:25:09,526] [250] [MainThread] [snowflake.connector.connection] [DEBUG] REST API object was created: bounty2.snowflakecomputing.com:443, proxy=None:None, proxy_user=None
[2017-04-15 19:25:09,526] [250] [MainThread] [snowflake.connector.network] [DEBUG] authenticate
[2017-04-15 19:25:09,527] [250] [MainThread] [snowflake.connector.network] [DEBUG] saml: False
[2017-04-15 19:25:09,527] [250] [MainThread] [snowflake.connector.network] [DEBUG] account=bounty2, user=jmayfield, database=None, schema=None, warehouse=None, role=None, request_id=d979e481-a3a1-45e3-8681-42e6fe15de12
[2017-04-15 19:25:09,527] [250] [MainThread] [snowflake.connector.network] [DEBUG] body['data']: {'CLIENT_APP_ID': 'PythonConnector', 'CLIENT_APP_VERSION': '1.3.15', 'SVN_REVISION': None, 'ACCOUNT_NAME': 'bounty2', 'CLIENT_ENVIRONMENT': {'APPLICATION': 'PythonConnector', 'OS_VERSION': 'Linux-4.9.17-8.31.amzn1.x86_64-x86_64-with-debian-8.7', 'PYTHON_VERSION': '3.6.1', 'PYTHON_RUNTIME': 'CPython', 'PYTHON_COMPILER': 'GCC 4.9.2'}, 'LOGIN_NAME': 'jmayfield'}
[2017-04-15 19:25:09,528] [250] [MainThread] [botocore.vendored.requests.packages.urllib3.util.retry] [DEBUG] Converted retries value: 5 -> Retry(total=5, connect=None, read=None, redirect=None)
[2017-04-15 19:25:09,528] [250] [MainThread] [botocore.vendored.requests.packages.urllib3.util.retry] [DEBUG] Converted retries value: 5 -> Retry(total=5, connect=None, read=None, redirect=None)
[2017-04-15 19:25:09,529] [250] [MainThread] [snowflake.connector.network] [DEBUG] using requests session [0 uses]: active 1, idle: 0
[2017-04-15 19:25:09,529] [250] [MainThread] [snowflake.connector.network] [DEBUG] request thread timeout: 60, rest of request timeout: 60, retry cnt: 1
[2017-04-15 19:25:09,531] [250] [RequestExec-0] [botocore.vendored.requests.packages.urllib3.connectionpool] [INFO] Starting new HTTPS connection (1): bounty2.snowflakecomputing.com
[2017-04-15 19:25:09,811] [250] [RequestExec-0] [snowflake.connector.ssl_wrap_socket] [DEBUG] insecure_mode: True, OCSP response cache file name: None, PROXY_HOST: None, PROXY_PORT: None, PROXY_USER: None PROXY_PASSWORD: None
[2017-04-15 19:25:09,812] [250] [RequestExec-0] [snowflake.connector.ssl_wrap_socket] [INFO] THIS CONNECTION IS IN INSECURE MODE. IT MEANS THE CERTIFICATE WILL BE VALIDATED BUT THE CERTIFICATE REVOCATION STATUS WILL NOT BE CHECKED.
[2017-04-15 19:25:10,261] [250] [RequestExec-0] [botocore.vendored.requests.packages.urllib3.connectionpool] [DEBUG] "POST /session/v1/login-request?request_id=d979e481-a3a1-45e3-8681-42e6fe15de12 HTTP/1.1" 200 1091
[2017-04-15 19:25:10,262] [250] [RequestExec-0] [snowflake.connector.network] [DEBUG] SUCCESS
[2017-04-15 19:25:10,262] [250] [MainThread] [snowflake.connector.network] [DEBUG] request thread joined
[2017-04-15 19:25:10,263] [250] [MainThread] [snowflake.connector.network] [DEBUG] request thread returned object
[2017-04-15 19:25:10,263] [250] [MainThread] [snowflake.connector.network] [DEBUG] freed requests sessions: active 0, idle: 0
[2017-04-15 19:25:10,263] [250] [MainThread] [snowflake.connector.network] [DEBUG] ret[code] = None, after post request
[2017-04-15 19:25:10,264] [250] [MainThread] [snowflake.connector.network] [DEBUG] completed authentication
[2017-04-15 19:25:10,264] [250] [MainThread] [snowflake.connector.network] [DEBUG] token = ETMsDgAAAVtzE4LGABRBRVMvQ0JDL1BLQ1M1UGFkZGluZwCAABAAEGmm977AMcQr2KF9ytZHHwQAAABwK/LynhrYK4Hplwkx+v1aFeo6iYiALQ4BqDLnulRqyZMu5QD1uZCNenV6+GTkEn+zCvhNkg9O+9gvUGGkgXzgev5d+PfuiZD7IVc9oP+yGxZc5KLJ1U4ZY7Yb39fH0/KGTOuYDkWD77P3aH+WoZ36mAAU7mcc2bQOCZscVjTXxPjAyISPnm4=
[2017-04-15 19:25:10,265] [250] [MainThread] [snowflake.connector.network] [DEBUG] master_token = ETMsDgAAAVtzE4KjABRBRVMvQ0JDL1BLQ1M1UGFkZGluZwCAABAAEBkpJwTQxat3Bq9JTurTa10AAACQFnNYMg1kmLOU+h9VZuKwd7ZgKLuU0mZq1cbLu3dYYmkrsBhunJVcNARlZxLPFn4fc6dcsXYTM3SXOGeIJzDaUT/DQv4No89MZTrbKxG1JBFqfLAPyHLhKIV6NwJRldgcE1ptJxCzRwceyW4oHdDDOVe27H4nOVolyQDPbWXtpgk9WeO2dLBd7gUOXxeLoEVNABSQRoGqC7OetmM5yeNAYizb4bwDVw==
[2017-04-15 19:25:10,265] [250] [MainThread] [snowflake.connector.connection] [DEBUG] __post connection
[2017-04-15 19:25:10,265] [250] [MainThread] [snowflake.connector.converter] [DEBUG] use_sfbinaryformat: False, use_numpy: False
[2017-04-15 19:25:10,266] [250] [MainThread] [snowflake.connector.connection] [DEBUG] cursor
[2017-04-15 19:25:10,266] [250] [MainThread] [snowflake.connector.cursor] [DEBUG] executing SQL/command
[2017-04-15 19:25:10,266] [250] [MainThread] [snowflake.connector.cursor] [DEBUG] binding: USE WAREHOUSE jm1 with input=None, processed={}
[2017-04-15 19:25:10,267] [250] [MainThread] [snowflake.connector.cursor] [DEBUG] query: [USE WAREHOUSE jm1]
[2017-04-15 19:25:10,267] [250] [MainThread] [snowflake.connector.connection] [DEBUG] sequence counter: 1
[2017-04-15 19:25:10,268] [250] [MainThread] [snowflake.connector.cursor] [DEBUG] running query [USE WAREHOUSE jm1]
[2017-04-15 19:25:10,268] [250] [MainThread] [snowflake.connector.cursor] [DEBUG] is_file_transfer: False
[2017-04-15 19:25:10,268] [250] [MainThread] [snowflake.connector.connection] [DEBUG] _cmd_query
[2017-04-15 19:25:10,269] [250] [MainThread] [snowflake.connector.connection] [DEBUG] sql=[USE WAREHOUSE jm1], sequence_id=[1], is_file_transfer=[None]
[2017-04-15 19:25:10,269] [250] [MainThread] [snowflake.connector.network] [DEBUG] using requests session [0 uses]: active 1, idle: 0
[2017-04-15 19:25:10,270] [250] [MainThread] [snowflake.connector.network] [DEBUG] request thread timeout: 60, rest of request timeout: 120, retry cnt: 1
[2017-04-15 19:25:10,452] [250] [RequestExec-1] [botocore.vendored.requests.packages.urllib3.connectionpool] [DEBUG] "POST /queries/v1/query-request?requestId=3d073340-fa1c-4493-8410-f88c61fa4f0e HTTP/1.1" 200 706
[2017-04-15 19:25:10,453] [250] [RequestExec-1] [snowflake.connector.network] [DEBUG] SUCCESS
[2017-04-15 19:25:10,454] [250] [MainThread] [snowflake.connector.network] [DEBUG] request thread joined
[2017-04-15 19:25:10,454] [250] [MainThread] [snowflake.connector.network] [DEBUG] request thread returned object
[2017-04-15 19:25:10,454] [250] [MainThread] [snowflake.connector.network] [DEBUG] freed requests sessions: active 0, idle: 0
[2017-04-15 19:25:10,455] [250] [MainThread] [snowflake.connector.network] [DEBUG] ret[code] = None, after post request
[2017-04-15 19:25:10,455] [250] [MainThread] [snowflake.connector.cursor] [DEBUG] cancelled timebomb in finally
[2017-04-15 19:25:10,455] [250] [MainThread] [snowflake.connector.cursor] [DEBUG] sfqid=61a66fca-dbd8-47ad-a6fe-301f68976af8
[2017-04-15 19:25:10,456] [250] [MainThread] [snowflake.connector.cursor] [DEBUG] SUCCESS
[2017-04-15 19:25:10,456] [250] [MainThread] [snowflake.connector.cursor] [DEBUG] PUT OR GET: None
[2017-04-15 19:25:10,456] [250] [MainThread] [snowflake.connector.cursor] [DEBUG] executing SQL/command
[2017-04-15 19:25:10,457] [250] [MainThread] [snowflake.connector.cursor] [DEBUG] binding: USE DATABASE jm1 with input=None, processed={}
[2017-04-15 19:25:10,457] [250] [MainThread] [snowflake.connector.cursor] [DEBUG] query: [USE DATABASE jm1]
[2017-04-15 19:25:10,457] [250] [MainThread] [snowflake.connector.connection] [DEBUG] sequence counter: 2
[2017-04-15 19:25:10,458] [250] [MainThread] [snowflake.connector.cursor] [DEBUG] running query [USE DATABASE jm1]
[2017-04-15 19:25:10,458] [250] [MainThread] [snowflake.connector.cursor] [DEBUG] is_file_transfer: False
[2017-04-15 19:25:10,458] [250] [MainThread] [snowflake.connector.connection] [DEBUG] _cmd_query
[2017-04-15 19:25:10,458] [250] [MainThread] [snowflake.connector.connection] [DEBUG] sql=[USE DATABASE jm1], sequence_id=[2], is_file_transfer=[None]
[2017-04-15 19:25:10,459] [250] [MainThread] [snowflake.connector.network] [DEBUG] using requests session [1 uses]: active 1, idle: 0
[2017-04-15 19:25:10,459] [250] [MainThread] [snowflake.connector.network] [DEBUG] request thread timeout: 60, rest of request timeout: 120, retry cnt: 1
[2017-04-15 19:25:10,605] [250] [RequestExec-2] [botocore.vendored.requests.packages.urllib3.connectionpool] [DEBUG] "POST /queries/v1/query-request?requestId=29684c0d-0c6d-4072-9329-c6696979559f HTTP/1.1" 200 716
[2017-04-15 19:25:10,606] [250] [RequestExec-2] [snowflake.connector.network] [DEBUG] SUCCESS
[2017-04-15 19:25:10,607] [250] [MainThread] [snowflake.connector.network] [DEBUG] request thread joined
[2017-04-15 19:25:10,607] [250] [MainThread] [snowflake.connector.network] [DEBUG] request thread returned object
[2017-04-15 19:25:10,608] [250] [MainThread] [snowflake.connector.network] [DEBUG] freed requests sessions: active 0, idle: 0
[2017-04-15 19:25:10,608] [250] [MainThread] [snowflake.connector.network] [DEBUG] ret[code] = None, after post request
[2017-04-15 19:25:10,608] [250] [MainThread] [snowflake.connector.cursor] [DEBUG] cancelled timebomb in finally
[2017-04-15 19:25:10,609] [250] [MainThread] [snowflake.connector.cursor] [DEBUG] sfqid=771eed52-9ea8-42bc-82d0-a48313929336
[2017-04-15 19:25:10,609] [250] [MainThread] [snowflake.connector.cursor] [DEBUG] SUCCESS
[2017-04-15 19:25:10,609] [250] [MainThread] [snowflake.connector.cursor] [DEBUG] PUT OR GET: None
[2017-04-15 19:25:10,610] [250] [MainThread] [root] [CRITICAL] STARTING: SELECT QUERY
[2017-04-15 19:25:10,610] [250] [MainThread] [snowflake.connector.cursor] [DEBUG] executing SQL/command
[2017-04-15 19:25:10,611] [250] [MainThread] [snowflake.connector.cursor] [DEBUG] binding: SELECT * FROM perf_simple_100m LIMIT 10000000 with input=None, processed={}
[2017-04-15 19:25:10,611] [250] [MainThread] [snowflake.connector.cursor] [DEBUG] query: [SELECT * FROM perf_simple_100m LIMIT 10000000]
[2017-04-15 19:25:10,611] [250] [MainThread] [snowflake.connector.connection] [DEBUG] sequence counter: 3
[2017-04-15 19:25:10,612] [250] [MainThread] [snowflake.connector.cursor] [DEBUG] running query [SELECT * FROM perf_simple_100m LIMIT 10000000]
[2017-04-15 19:25:10,612] [250] [MainThread] [snowflake.connector.cursor] [DEBUG] is_file_transfer: False
[2017-04-15 19:25:10,612] [250] [MainThread] [snowflake.connector.connection] [DEBUG] _cmd_query
[2017-04-15 19:25:10,613] [250] [MainThread] [snowflake.connector.connection] [DEBUG] sql=[SELECT * FROM perf_simple_100m LIMIT 10000000], sequence_id=[3], is_file_transfer=[None]
[2017-04-15 19:25:10,613] [250] [MainThread] [snowflake.connector.network] [DEBUG] using requests session [2 uses]: active 1, idle: 0
[2017-04-15 19:25:10,614] [250] [MainThread] [snowflake.connector.network] [DEBUG] request thread timeout: 60, rest of request timeout: 120, retry cnt: 1
[2017-04-15 19:25:11,355] [250] [RequestExec-3] [botocore.vendored.requests.packages.urllib3.connectionpool] [DEBUG] "POST /queries/v1/query-request?requestId=1c6038d2-f8a4-41b5-8c05-e058f6b791fa HTTP/1.1" 200 None
[2017-04-15 19:25:11,357] [250] [RequestExec-3] [snowflake.connector.network] [DEBUG] SUCCESS
[2017-04-15 19:25:11,359] [250] [MainThread] [snowflake.connector.network] [DEBUG] request thread joined
[2017-04-15 19:25:11,359] [250] [MainThread] [snowflake.connector.network] [DEBUG] request thread returned object
[2017-04-15 19:25:11,359] [250] [MainThread] [snowflake.connector.network] [DEBUG] freed requests sessions: active 0, idle: 0
[2017-04-15 19:25:11,360] [250] [MainThread] [snowflake.connector.network] [DEBUG] ret[code] = None, after post request
[2017-04-15 19:25:11,360] [250] [MainThread] [snowflake.connector.cursor] [DEBUG] cancelled timebomb in finally
[2017-04-15 19:25:11,360] [250] [MainThread] [snowflake.connector.cursor] [DEBUG] sfqid=63c8b05a-7b18-4f50-b868-cbb9f0defdca
[2017-04-15 19:25:11,361] [250] [MainThread] [snowflake.connector.cursor] [DEBUG] SUCCESS
[2017-04-15 19:25:11,361] [250] [MainThread] [snowflake.connector.cursor] [DEBUG] PUT OR GET: None
[2017-04-15 19:25:11,361] [250] [MainThread] [snowflake.connector.cursor] [DEBUG] added chunk header: key=x-amz-server-side-encryption-customer-key, value=lZSvKoOkdXa7JLAqt7fxbCxQMjJxWpV1nEqORajHM6k=
[2017-04-15 19:25:11,361] [250] [MainThread] [snowflake.connector.cursor] [DEBUG] added chunk header: key=x-amz-server-side-encryption-customer-key-md5, value=9CT9WM8c4lok375GoQKsjQ==
[2017-04-15 19:25:11,362] [250] [MainThread] [snowflake.connector.cursor] [DEBUG] qrmk=lZSvKoOkdXa7JLAqt7fxbCxQMjJxWpV1nEqORajHM6k=
[2017-04-15 19:25:11,362] [250] [MainThread] [root] [CRITICAL] FINISHED: SELECT QUERY
SELECT QUERY...
TOOK 0.7524612519998755
USER 0.00999999999999801
SYS 0.0
IOWAIT 0.0
IDLE 1.4899999999997817
[2017-04-15 19:25:11,363] [250] [MainThread] [root] [CRITICAL] STARTING: FETCHONE() ITER
[2017-04-15 19:25:11,363] [250] [ChunkDownloader_1] [snowflake.connector.network] [DEBUG] using requests session [3 uses]: active 1, idle: 0
[2017-04-15 19:25:11,365] [250] [RequestExec-4] [botocore.vendored.requests.packages.urllib3.connectionpool] [INFO] Starting new HTTPS connection (1): sfc-ds2-customer-stage.s3.amazonaws.com
[2017-04-15 19:25:11,365] [250] [ChunkDownloader_1] [snowflake.connector.network] [DEBUG] request thread timeout: 60, rest of request timeout: 60, retry cnt: 1
[2017-04-15 19:25:11,614] [250] [ChunkDownloader_2] [botocore.vendored.requests.packages.urllib3.util.retry] [DEBUG] Converted retries value: 5 -> Retry(total=5, connect=None, read=None, redirect=None)
[2017-04-15 19:25:11,615] [250] [ChunkDownloader_2] [botocore.vendored.requests.packages.urllib3.util.retry] [DEBUG] Converted retries value: 5 -> Retry(total=5, connect=None, read=None, redirect=None)
[2017-04-15 19:25:11,615] [250] [ChunkDownloader_2] [snowflake.connector.network] [DEBUG] using requests session [0 uses]: active 2, idle: 0
[2017-04-15 19:25:11,616] [250] [RequestExec-5] [botocore.vendored.requests.packages.urllib3.connectionpool] [INFO] Starting new HTTPS connection (1): sfc-ds2-customer-stage.s3.amazonaws.com
[2017-04-15 19:25:11,616] [250] [ChunkDownloader_2] [snowflake.connector.network] [DEBUG] request thread timeout: 60, rest of request timeout: 60, retry cnt: 1
[2017-04-15 19:25:11,835] [250] [RequestExec-5] [snowflake.connector.ssl_wrap_socket] [DEBUG] insecure_mode: True, OCSP response cache file name: None, PROXY_HOST: None, PROXY_PORT: None, PROXY_USER: None PROXY_PASSWORD: None
[2017-04-15 19:25:11,835] [250] [RequestExec-5] [snowflake.connector.ssl_wrap_socket] [INFO] THIS CONNECTION IS IN INSECURE MODE. IT MEANS THE CERTIFICATE WILL BE VALIDATED BUT THE CERTIFICATE REVOCATION STATUS WILL NOT BE CHECKED.
[2017-04-15 19:25:11,898] [250] [RequestExec-4] [snowflake.connector.ssl_wrap_socket] [DEBUG] insecure_mode: True, OCSP response cache file name: None, PROXY_HOST: None, PROXY_PORT: None, PROXY_USER: None PROXY_PASSWORD: None
[2017-04-15 19:25:11,898] [250] [RequestExec-4] [snowflake.connector.ssl_wrap_socket] [INFO] THIS CONNECTION IS IN INSECURE MODE. IT MEANS THE CERTIFICATE WILL BE VALIDATED BUT THE CERTIFICATE REVOCATION STATUS WILL NOT BE CHECKED.
[2017-04-15 19:25:11,962] [250] [RequestExec-5] [botocore.vendored.requests.packages.urllib3.connectionpool] [DEBUG] "GET /udoi-s-bounty2-/results/b1f97eed-9f52-4f6b-a14e-b5de377ae1bf/main/data_0_0_1?x-amz-server-side-encryption-customer-algorithm=AES256&response-content-encoding=gzip&AWSAccessKeyId=AKIAIVMHKINF3K536SLA&Expires=1492305910&Signature=5SQrkPNNAqb%2FZ8rJ%2BSJCsBA1UJo%3D HTTP/1.1" 200 109922
[2017-04-15 19:25:11,962] [250] [RequestExec-5] [snowflake.connector.network] [DEBUG] SUCCESS
[2017-04-15 19:25:11,997] [250] [RequestExec-4] [botocore.vendored.requests.packages.urllib3.connectionpool] [DEBUG] "GET /udoi-s-bounty2-/results/b1f97eed-9f52-4f6b-a14e-b5de377ae1bf/main/data_0_0_0?x-amz-server-side-encryption-customer-algorithm=AES256&response-content-encoding=gzip&AWSAccessKeyId=AKIAIVMHKINF3K536SLA&Expires=1492305910&Signature=He9frS81LzfiXrDUodsOPpyp7wo%3D HTTP/1.1" 200 85670
[2017-04-15 19:25:11,998] [250] [RequestExec-4] [snowflake.connector.network] [DEBUG] SUCCESS
[2017-04-15 19:25:12,350] [250] [ChunkDownloader_2] [snowflake.connector.network] [DEBUG] request thread joined
[2017-04-15 19:25:12,350] [250] [ChunkDownloader_2] [snowflake.connector.network] [DEBUG] request thread returned object
[2017-04-15 19:25:12,351] [250] [ChunkDownloader_2] [snowflake.connector.network] [DEBUG] freed requests sessions: active 1, idle: 0
[2017-04-15 19:25:12,351] [250] [ChunkDownloader_2] [BENCHMARK] [INFO] Chunk download time: 737 ms (0 ms latency)
[2017-04-15 19:25:12,352] [250] [ChunkDownloader_3] [snowflake.connector.network] [DEBUG] using requests session [0 uses]: active 2, idle: 0
[2017-04-15 19:25:12,353] [250] [ChunkDownloader_3] [snowflake.connector.network] [DEBUG] request thread timeout: 60, rest of request timeout: 60, retry cnt: 1
[2017-04-15 19:25:12,454] [250] [RequestExec-6] [botocore.vendored.requests.packages.urllib3.connectionpool] [DEBUG] "GET /udoi-s-bounty2-/results/b1f97eed-9f52-4f6b-a14e-b5de377ae1bf/main/data_0_0_2?x-amz-server-side-encryption-customer-algorithm=AES256&response-content-encoding=gzip&AWSAccessKeyId=AKIAIVMHKINF3K536SLA&Expires=1492305910&Signature=Z8F%2Bj4MW62cAN4YLh1DYyQsTMwk%3D HTTP/1.1" 200 208389
[2017-04-15 19:25:12,455] [250] [RequestExec-6] [snowflake.connector.network] [DEBUG] SUCCESS
[2017-04-15 19:25:12,866] [250] [ChunkDownloader_4] [botocore.vendored.requests.packages.urllib3.util.retry] [DEBUG] Converted retries value: 5 -> Retry(total=5, connect=None, read=None, redirect=None)
[2017-04-15 19:25:12,867] [250] [ChunkDownloader_4] [botocore.vendored.requests.packages.urllib3.util.retry] [DEBUG] Converted retries value: 5 -> Retry(total=5, connect=None, read=None, redirect=None)
[2017-04-15 19:25:12,867] [250] [ChunkDownloader_4] [snowflake.connector.network] [DEBUG] using requests session [0 uses]: active 3, idle: 0
[2017-04-15 19:25:12,868] [250] [RequestExec-7] [botocore.vendored.requests.packages.urllib3.connectionpool] [INFO] Starting new HTTPS connection (1): sfc-ds2-customer-stage.s3.amazonaws.com
[2017-04-15 19:25:12,868] [250] [ChunkDownloader_4] [snowflake.connector.network] [DEBUG] request thread timeout: 60, rest of request timeout: 60, retry cnt: 1
[2017-04-15 19:25:12,871] [250] [ChunkDownloader_3] [snowflake.connector.network] [DEBUG] request thread joined
[2017-04-15 19:25:12,871] [250] [ChunkDownloader_3] [snowflake.connector.network] [DEBUG] request thread returned object
[2017-04-15 19:25:12,871] [250] [ChunkDownloader_3] [snowflake.connector.network] [DEBUG] freed requests sessions: active 2, idle: 0
[2017-04-15 19:25:12,872] [250] [ChunkDownloader_3] [BENCHMARK] [INFO] Chunk download time: 520 ms (0 ms latency)
[2017-04-15 19:25:13,083] [250] [RequestExec-7] [snowflake.connector.ssl_wrap_socket] [DEBUG] insecure_mode: True, OCSP response cache file name: None, PROXY_HOST: None, PROXY_PORT: None, PROXY_USER: None PROXY_PASSWORD: None
[2017-04-15 19:25:13,083] [250] [RequestExec-7] [snowflake.connector.ssl_wrap_socket] [INFO] THIS CONNECTION IS IN INSECURE MODE. IT MEANS THE CERTIFICATE WILL BE VALIDATED BUT THE CERTIFICATE REVOCATION STATUS WILL NOT BE CHECKED.
[2017-04-15 19:25:13,182] [250] [RequestExec-7] [botocore.vendored.requests.packages.urllib3.connectionpool] [DEBUG] "GET /udoi-s-bounty2-/results/b1f97eed-9f52-4f6b-a14e-b5de377ae1bf/main/data_0_0_3?x-amz-server-side-encryption-customer-algorithm=AES256&response-content-encoding=gzip&AWSAccessKeyId=AKIAIVMHKINF3K536SLA&Expires=1492305910&Signature=pkN9ImnQmetuiEK3VWqNpsTAwIM%3D HTTP/1.1" 200 417481
[2017-04-15 19:25:13,183] [250] [RequestExec-7] [snowflake.connector.network] [DEBUG] SUCCESS
[2017-04-15 19:25:13,543] [250] [ChunkDownloader_1] [snowflake.connector.network] [DEBUG] request thread joined
[2017-04-15 19:25:13,544] [250] [ChunkDownloader_1] [snowflake.connector.network] [DEBUG] request thread returned object
[2017-04-15 19:25:13,544] [250] [ChunkDownloader_1] [snowflake.connector.network] [DEBUG] freed requests sessions: active 1, idle: 1
[2017-04-15 19:25:13,544] [250] [ChunkDownloader_1] [BENCHMARK] [INFO] Chunk download time: 2181 ms (0 ms latency)
[2017-04-15 19:25:13,545] [250] [ChunkDownloader_5] [snowflake.connector.network] [DEBUG] using requests session [1 uses]: active 2, idle: 1
[2017-04-15 19:25:13,547] [250] [ChunkDownloader_5] [snowflake.connector.network] [DEBUG] request thread timeout: 60, rest of request timeout: 60, retry cnt: 1
[2017-04-15 19:25:13,547] [250] [ChunkDownloader_6] [snowflake.connector.network] [DEBUG] using requests session [4 uses]: active 3, idle: 0
[2017-04-15 19:25:13,548] [250] [ChunkDownloader_7] [botocore.vendored.requests.packages.urllib3.util.retry] [DEBUG] Converted retries value: 5 -> Retry(total=5, connect=None, read=None, redirect=None)
[2017-04-15 19:25:13,550] [250] [ChunkDownloader_7] [botocore.vendored.requests.packages.urllib3.util.retry] [DEBUG] Converted retries value: 5 -> Retry(total=5, connect=None, read=None, redirect=None)
[2017-04-15 19:25:13,550] [250] [ChunkDownloader_6] [snowflake.connector.network] [DEBUG] request thread timeout: 60, rest of request timeout: 60, retry cnt: 1
[2017-04-15 19:25:13,551] [250] [ChunkDownloader_7] [snowflake.connector.network] [DEBUG] using requests session [0 uses]: active 4, idle: 0
[2017-04-15 19:25:13,552] [250] [RequestExec-10] [botocore.vendored.requests.packages.urllib3.connectionpool] [INFO] Starting new HTTPS connection (1): sfc-ds2-customer-stage.s3.amazonaws.com
[2017-04-15 19:25:13,554] [250] [ChunkDownloader_7] [snowflake.connector.network] [DEBUG] request thread timeout: 60, rest of request timeout: 60, retry cnt: 1
[2017-04-15 19:25:13,644] [250] [RequestExec-8] [botocore.vendored.requests.packages.urllib3.connectionpool] [DEBUG] "GET /udoi-s-bounty2-/results/b1f97eed-9f52-4f6b-a14e-b5de377ae1bf/main/data_0_0_4?x-amz-server-side-encryption-customer-algorithm=AES256&response-content-encoding=gzip&AWSAccessKeyId=AKIAIVMHKINF3K536SLA&Expires=1492305910&Signature=LhT%2BktfwoJQBSUVE6Q5h%2Fa%2FmPx4%3D HTTP/1.1" 200 834206
[2017-04-15 19:25:13,645] [250] [RequestExec-8] [snowflake.connector.network] [DEBUG] SUCCESS
[2017-04-15 19:25:13,663] [250] [RequestExec-9] [botocore.vendored.requests.packages.urllib3.connectionpool] [DEBUG] "GET /udoi-s-bounty2-/results/b1f97eed-9f52-4f6b-a14e-b5de377ae1bf/main/data_0_0_5?x-amz-server-side-encryption-customer-algorithm=AES256&response-content-encoding=gzip&AWSAccessKeyId=AKIAIVMHKINF3K536SLA&Expires=1492305910&Signature=SZC%2F3Hsx1FpNQkkoBXAwKXBqLU8%3D HTTP/1.1" 200 1655681
[2017-04-15 19:25:13,664] [250] [RequestExec-9] [snowflake.connector.network] [DEBUG] SUCCESS
[2017-04-15 19:25:13,794] [250] [RequestExec-10] [snowflake.connector.ssl_wrap_socket] [DEBUG] insecure_mode: True, OCSP response cache file name: None, PROXY_HOST: None, PROXY_PORT: None, PROXY_USER: None PROXY_PASSWORD: None
[2017-04-15 19:25:13,794] [250] [RequestExec-10] [snowflake.connector.ssl_wrap_socket] [INFO] THIS CONNECTION IS IN INSECURE MODE. IT MEANS THE CERTIFICATE WILL BE VALIDATED BUT THE CERTIFICATE REVOCATION STATUS WILL NOT BE CHECKED.
[2017-04-15 19:25:13,801] [250] [ChunkDownloader_8] [botocore.vendored.requests.packages.urllib3.util.retry] [DEBUG] Converted retries value: 5 -> Retry(total=5, connect=None, read=None, redirect=None)
[2017-04-15 19:25:13,801] [250] [ChunkDownloader_8] [botocore.vendored.requests.packages.urllib3.util.retry] [DEBUG] Converted retries value: 5 -> Retry(total=5, connect=None, read=None, redirect=None)
[2017-04-15 19:25:13,802] [250] [ChunkDownloader_8] [snowflake.connector.network] [DEBUG] using requests session [0 uses]: active 5, idle: 0
[2017-04-15 19:25:13,803] [250] [RequestExec-11] [botocore.vendored.requests.packages.urllib3.connectionpool] [INFO] Starting new HTTPS connection (1): sfc-ds2-customer-stage.s3.amazonaws.com
[2017-04-15 19:25:13,804] [250] [ChunkDownloader_8] [snowflake.connector.network] [DEBUG] request thread timeout: 60, rest of request timeout: 60, retry cnt: 1
[2017-04-15 19:25:13,951] [250] [RequestExec-10] [botocore.vendored.requests.packages.urllib3.connectionpool] [DEBUG] "GET /udoi-s-bounty2-/results/b1f97eed-9f52-4f6b-a14e-b5de377ae1bf/main/data_0_0_6?x-amz-server-side-encryption-customer-algorithm=AES256&response-content-encoding=gzip&AWSAccessKeyId=AKIAIVMHKINF3K536SLA&Expires=1492305910&Signature=4RC54cNcKd6fjNNkYbwfK2ypZq8%3D HTTP/1.1" 200 3301792
[2017-04-15 19:25:13,952] [250] [RequestExec-10] [snowflake.connector.network] [DEBUG] SUCCESS
[2017-04-15 19:25:14,024] [250] [RequestExec-11] [snowflake.connector.ssl_wrap_socket] [DEBUG] insecure_mode: True, OCSP response cache file name: None, PROXY_HOST: None, PROXY_PORT: None, PROXY_USER: None PROXY_PASSWORD: None
[2017-04-15 19:25:14,024] [250] [RequestExec-11] [snowflake.connector.ssl_wrap_socket] [INFO] THIS CONNECTION IS IN INSECURE MODE. IT MEANS THE CERTIFICATE WILL BE VALIDATED BUT THE CERTIFICATE REVOCATION STATUS WILL NOT BE CHECKED.
[2017-04-15 19:25:14,041] [250] [ChunkDownloader_4] [snowflake.connector.network] [DEBUG] request thread joined
[2017-04-15 19:25:14,041] [250] [ChunkDownloader_4] [snowflake.connector.network] [DEBUG] request thread returned object
[2017-04-15 19:25:14,042] [250] [ChunkDownloader_4] [snowflake.connector.network] [DEBUG] freed requests sessions: active 4, idle: 0
[2017-04-15 19:25:14,042] [250] [ChunkDownloader_4] [BENCHMARK] [INFO] Chunk download time: 1175 ms (0 ms latency)
[2017-04-15 19:25:14,042] [250] [ChunkDownloader_9] [snowflake.connector.network] [DEBUG] using requests session [0 uses]: active 5, idle: 0
[2017-04-15 19:25:14,043] [250] [ChunkDownloader_10] [botocore.vendored.requests.packages.urllib3.util.retry] [DEBUG] Converted retries value: 5 -> Retry(total=5, connect=None, read=None, redirect=None)
[2017-04-15 19:25:14,045] [250] [ChunkDownloader_10] [botocore.vendored.requests.packages.urllib3.util.retry] [DEBUG] Converted retries value: 5 -> Retry(total=5, connect=None, read=None, redirect=None)
[2017-04-15 19:25:14,045] [250] [ChunkDownloader_9] [snowflake.connector.network] [DEBUG] request thread timeout: 60, rest of request timeout: 60, retry cnt: 1
[2017-04-15 19:25:14,046] [250] [ChunkDownloader_10] [snowflake.connector.network] [DEBUG] using requests session [0 uses]: active 6, idle: 0
[2017-04-15 19:25:14,048] [250] [RequestExec-13] [botocore.vendored.requests.packages.urllib3.connectionpool] [INFO] Starting new HTTPS connection (1): sfc-ds2-customer-stage.s3.amazonaws.com
[2017-04-15 19:25:14,048] [250] [ChunkDownloader_10] [snowflake.connector.network] [DEBUG] request thread timeout: 60, rest of request timeout: 60, retry cnt: 1
[2017-04-15 19:25:14,147] [250] [RequestExec-11] [botocore.vendored.requests.packages.urllib3.connectionpool] [DEBUG] "GET /udoi-s-bounty2-/results/b1f97eed-9f52-4f6b-a14e-b5de377ae1bf/main/data_0_0_7?x-amz-server-side-encryption-customer-algorithm=AES256&response-content-encoding=gzip&AWSAccessKeyId=AKIAIVMHKINF3K536SLA&Expires=1492305910&Signature=jAWR91aDw3W%2BbrMFydelJ7HYk6w%3D HTTP/1.1" 200 6577279
[2017-04-15 19:25:14,148] [250] [RequestExec-11] [snowflake.connector.network] [DEBUG] SUCCESS
[2017-04-15 19:25:14,166] [250] [RequestExec-12] [botocore.vendored.requests.packages.urllib3.connectionpool] [DEBUG] "GET /udoi-s-bounty2-/results/b1f97eed-9f52-4f6b-a14e-b5de377ae1bf/main/data_0_0_8?x-amz-server-side-encryption-customer-algorithm=AES256&response-content-encoding=gzip&AWSAccessKeyId=AKIAIVMHKINF3K536SLA&Expires=1492305910&Signature=XfV%2BG6DoZLJobffnHKApLdjUU6Q%3D HTTP/1.1" 200 13135153
[2017-04-15 19:25:14,167] [250] [RequestExec-12] [snowflake.connector.network] [DEBUG] SUCCESS
[2017-04-15 19:25:14,270] [250] [RequestExec-13] [snowflake.connector.ssl_wrap_socket] [DEBUG] insecure_mode: True, OCSP response cache file name: None, PROXY_HOST: None, PROXY_PORT: None, PROXY_USER: None PROXY_PASSWORD: None
[2017-04-15 19:25:14,270] [250] [RequestExec-13] [snowflake.connector.ssl_wrap_socket] [INFO] THIS CONNECTION IS IN INSECURE MODE. IT MEANS THE CERTIFICATE WILL BE VALIDATED BUT THE CERTIFICATE REVOCATION STATUS WILL NOT BE CHECKED.
[2017-04-15 19:25:14,297] [250] [ChunkDownloader_11] [botocore.vendored.requests.packages.urllib3.util.retry] [DEBUG] Converted retries value: 5 -> Retry(total=5, connect=None, read=None, redirect=None)
[2017-04-15 19:25:14,297] [250] [ChunkDownloader_11] [botocore.vendored.requests.packages.urllib3.util.retry] [DEBUG] Converted retries value: 5 -> Retry(total=5, connect=None, read=None, redirect=None)
[2017-04-15 19:25:14,298] [250] [ChunkDownloader_11] [snowflake.connector.network] [DEBUG] using requests session [0 uses]: active 7, idle: 0
[2017-04-15 19:25:14,299] [250] [RequestExec-14] [botocore.vendored.requests.packages.urllib3.connectionpool] [INFO] Starting new HTTPS connection (1): sfc-ds2-customer-stage.s3.amazonaws.com
[2017-04-15 19:25:14,299] [250] [ChunkDownloader_11] [snowflake.connector.network] [DEBUG] request thread timeout: 60, rest of request timeout: 60, retry cnt: 1
[2017-04-15 19:25:14,518] [250] [RequestExec-14] [snowflake.connector.ssl_wrap_socket] [DEBUG] insecure_mode: True, OCSP response cache file name: None, PROXY_HOST: None, PROXY_PORT: None, PROXY_USER: None PROXY_PASSWORD: None
[2017-04-15 19:25:14,519] [250] [RequestExec-14] [snowflake.connector.ssl_wrap_socket] [INFO] THIS CONNECTION IS IN INSECURE MODE. IT MEANS THE CERTIFICATE WILL BE VALIDATED BUT THE CERTIFICATE REVOCATION STATUS WILL NOT BE CHECKED.
[2017-04-15 19:25:14,548] [250] [ChunkDownloader_12] [botocore.vendored.requests.packages.urllib3.util.retry] [DEBUG] Converted retries value: 5 -> Retry(total=5, connect=None, read=None, redirect=None)
[2017-04-15 19:25:14,549] [250] [ChunkDownloader_12] [botocore.vendored.requests.packages.urllib3.util.retry] [DEBUG] Converted retries value: 5 -> Retry(total=5, connect=None, read=None, redirect=None)
[2017-04-15 19:25:14,549] [250] [ChunkDownloader_12] [snowflake.connector.network] [DEBUG] using requests session [0 uses]: active 8, idle: 0
[2017-04-15 19:25:14,551] [250] [RequestExec-15] [botocore.vendored.requests.packages.urllib3.connectionpool] [INFO] Starting new HTTPS connection (1): sfc-ds2-customer-stage.s3.amazonaws.com
[2017-04-15 19:25:14,551] [250] [ChunkDownloader_12] [snowflake.connector.network] [DEBUG] request thread timeout: 60, rest of request timeout: 60, retry cnt: 1
[2017-04-15 19:25:14,572] [250] [RequestExec-13] [botocore.vendored.requests.packages.urllib3.connectionpool] [DEBUG] "GET /udoi-s-bounty2-/results/b1f97eed-9f52-4f6b-a14e-b5de377ae1bf/main/data_0_0_9?x-amz-server-side-encryption-customer-algorithm=AES256&response-content-encoding=gzip&AWSAccessKeyId=AKIAIVMHKINF3K536SLA&Expires=1492305910&Signature=i3YH6CMdQx2jxPt9UHc34sg%2FhIQ%3D HTTP/1.1" 200 16802677
[2017-04-15 19:25:14,573] [250] [RequestExec-13] [snowflake.connector.network] [DEBUG] SUCCESS
[2017-04-15 19:25:14,752] [250] [RequestExec-14] [botocore.vendored.requests.packages.urllib3.connectionpool] [DEBUG] "GET /udoi-s-bounty2-/results/b1f97eed-9f52-4f6b-a14e-b5de377ae1bf/main/data_0_0_10?x-amz-server-side-encryption-customer-algorithm=AES256&response-content-encoding=gzip&AWSAccessKeyId=AKIAIVMHKINF3K536SLA&Expires=1492305910&Signature=hndyPK4v8L36Ikff0WGLpr7rbns%3D HTTP/1.1" 200 16809236
[2017-04-15 19:25:14,753] [250] [RequestExec-14] [snowflake.connector.network] [DEBUG] SUCCESS
[2017-04-15 19:25:14,772] [250] [RequestExec-15] [snowflake.connector.ssl_wrap_socket] [DEBUG] insecure_mode: True, OCSP response cache file name: None, PROXY_HOST: None, PROXY_PORT: None, PROXY_USER: None PROXY_PASSWORD: None
[2017-04-15 19:25:14,773] [250] [RequestExec-15] [snowflake.connector.ssl_wrap_socket] [INFO] THIS CONNECTION IS IN INSECURE MODE. IT MEANS THE CERTIFICATE WILL BE VALIDATED BUT THE CERTIFICATE REVOCATION STATUS WILL NOT BE CHECKED.
[2017-04-15 19:25:14,799] [250] [ChunkDownloader_13] [botocore.vendored.requests.packages.urllib3.util.retry] [DEBUG] Converted retries value: 5 -> Retry(total=5, connect=None, read=None, redirect=None)
[2017-04-15 19:25:14,800] [250] [ChunkDownloader_13] [botocore.vendored.requests.packages.urllib3.util.retry] [DEBUG] Converted retries value: 5 -> Retry(total=5, connect=None, read=None, redirect=None)
[2017-04-15 19:25:14,800] [250] [ChunkDownloader_13] [snowflake.connector.network] [DEBUG] using requests session [0 uses]: active 9, idle: 0
[2017-04-15 19:25:14,801] [250] [RequestExec-16] [botocore.vendored.requests.packages.urllib3.connectionpool] [INFO] Starting new HTTPS connection (1): sfc-ds2-customer-stage.s3.amazonaws.com
[2017-04-15 19:25:14,802] [250] [ChunkDownloader_13] [snowflake.connector.network] [DEBUG] request thread timeout: 60, rest of request timeout: 60, retry cnt: 1
[2017-04-15 19:25:14,881] [250] [RequestExec-15] [botocore.vendored.requests.packages.urllib3.connectionpool] [DEBUG] "GET /udoi-s-bounty2-/results/b1f97eed-9f52-4f6b-a14e-b5de377ae1bf/main/data_0_0_11?x-amz-server-side-encryption-customer-algorithm=AES256&response-content-encoding=gzip&AWSAccessKeyId=AKIAIVMHKINF3K536SLA&Expires=1492305910&Signature=3IOXrK%2Fg%2FUTBrbrDbDXpFY1Yen8%3D HTTP/1.1" 200 16809616
[2017-04-15 19:25:14,882] [250] [RequestExec-15] [snowflake.connector.network] [DEBUG] SUCCESS
[2017-04-15 19:25:15,020] [250] [RequestExec-16] [snowflake.connector.ssl_wrap_socket] [DEBUG] insecure_mode: True, OCSP response cache file name: None, PROXY_HOST: None, PROXY_PORT: None, PROXY_USER: None PROXY_PASSWORD: None
[2017-04-15 19:25:15,021] [250] [RequestExec-16] [snowflake.connector.ssl_wrap_socket] [INFO] THIS CONNECTION IS IN INSECURE MODE. IT MEANS THE CERTIFICATE WILL BE VALIDATED BUT THE CERTIFICATE REVOCATION STATUS WILL NOT BE CHECKED.
[2017-04-15 19:25:15,050] [250] [ChunkDownloader_14] [botocore.vendored.requests.packages.urllib3.util.retry] [DEBUG] Converted retries value: 5 -> Retry(total=5, connect=None, read=None, redirect=None)
[2017-04-15 19:25:15,051] [250] [ChunkDownloader_14] [botocore.vendored.requests.packages.urllib3.util.retry] [DEBUG] Converted retries value: 5 -> Retry(total=5, connect=None, read=None, redirect=None)
[2017-04-15 19:25:15,051] [250] [ChunkDownloader_14] [snowflake.connector.network] [DEBUG] using requests session [0 uses]: active 10, idle: 0
[2017-04-15 19:25:15,052] [250] [RequestExec-17] [botocore.vendored.requests.packages.urllib3.connectionpool] [INFO] Starting new HTTPS connection (1): sfc-ds2-customer-stage.s3.amazonaws.com
[2017-04-15 19:25:15,053] [250] [ChunkDownloader_14] [snowflake.connector.network] [DEBUG] request thread timeout: 60, rest of request timeout: 60, retry cnt: 1
[2017-04-15 19:25:15,108] [250] [ChunkDownloader_5] [snowflake.connector.network] [DEBUG] request thread joined
[2017-04-15 19:25:15,108] [250] [ChunkDownloader_5] [snowflake.connector.network] [DEBUG] request thread returned object
[2017-04-15 19:25:15,109] [250] [ChunkDownloader_5] [snowflake.connector.network] [DEBUG] freed requests sessions: active 9, idle: 0
[2017-04-15 19:25:15,109] [250] [ChunkDownloader_5] [BENCHMARK] [INFO] Chunk download time: 1564 ms (0 ms latency)
[2017-04-15 19:25:15,110] [250] [ChunkDownloader_15] [snowflake.connector.network] [DEBUG] using requests session [2 uses]: active 10, idle: 0
[2017-04-15 19:25:15,110] [250] [ChunkDownloader_16] [botocore.vendored.requests.packages.urllib3.util.retry] [DEBUG] Converted retries value: 5 -> Retry(total=5, connect=None, read=None, redirect=None)
[2017-04-15 19:25:15,112] [250] [ChunkDownloader_16] [botocore.vendored.requests.packages.urllib3.util.retry] [DEBUG] Converted retries value: 5 -> Retry(total=5, connect=None, read=None, redirect=None)
[2017-04-15 19:25:15,112] [250] [ChunkDownloader_15] [snowflake.connector.network] [DEBUG] request thread timeout: 60, rest of request timeout: 60, retry cnt: 1
[2017-04-15 19:25:15,115] [250] [ChunkDownloader_16] [snowflake.connector.network] [DEBUG] using requests session [0 uses]: active 11, idle: 0
[2017-04-15 19:25:15,116] [250] [RequestExec-19] [botocore.vendored.requests.packages.urllib3.connectionpool] [INFO] Starting new HTTPS connection (1): sfc-ds2-customer-stage.s3.amazonaws.com
[2017-04-15 19:25:15,117] [250] [ChunkDownloader_16] [snowflake.connector.network] [DEBUG] request thread timeout: 60, rest of request timeout: 60, retry cnt: 1
[2017-04-15 19:25:15,173] [250] [RequestExec-16] [botocore.vendored.requests.packages.urllib3.connectionpool] [DEBUG] "GET /udoi-s-bounty2-/results/b1f97eed-9f52-4f6b-a14e-b5de377ae1bf/main/data_0_0_12?x-amz-server-side-encryption-customer-algorithm=AES256&response-content-encoding=gzip&AWSAccessKeyId=AKIAIVMHKINF3K536SLA&Expires=1492305910&Signature=PUnlkFF2sOdle6Sj4Dridx4day4%3D HTTP/1.1" 200 16808150
[2017-04-15 19:25:15,174] [250] [RequestExec-16] [snowflake.connector.network] [DEBUG] SUCCESS
[2017-04-15 19:25:15,241] [250] [RequestExec-18] [botocore.vendored.requests.packages.urllib3.connectionpool] [DEBUG] "GET /udoi-s-bounty2-/results/b1f97eed-9f52-4f6b-a14e-b5de377ae1bf/main/data_0_0_14?x-amz-server-side-encryption-customer-algorithm=AES256&response-content-encoding=gzip&AWSAccessKeyId=AKIAIVMHKINF3K536SLA&Expires=1492305910&Signature=3J623hB%2BvsvvRa8lppLjC9v50LE%3D HTTP/1.1" 200 16812365
[2017-04-15 19:25:15,242] [250] [RequestExec-18] [snowflake.connector.network] [DEBUG] SUCCESS
[2017-04-15 19:25:15,266] [250] [RequestExec-17] [snowflake.connector.ssl_wrap_socket] [DEBUG] insecure_mode: True, OCSP response cache file name: None, PROXY_HOST: None, PROXY_PORT: None, PROXY_USER: None PROXY_PASSWORD: None
[2017-04-15 19:25:15,267] [250] [RequestExec-17] [snowflake.connector.ssl_wrap_socket] [INFO] THIS CONNECTION IS IN INSECURE MODE. IT MEANS THE CERTIFICATE WILL BE VALIDATED BUT THE CERTIFICATE REVOCATION STATUS WILL NOT BE CHECKED.
[2017-04-15 19:25:15,337] [250] [RequestExec-19] [snowflake.connector.ssl_wrap_socket] [DEBUG] insecure_mode: True, OCSP response cache file name: None, PROXY_HOST: None, PROXY_PORT: None, PROXY_USER: None PROXY_PASSWORD: None
[2017-04-15 19:25:15,337] [250] [RequestExec-19] [snowflake.connector.ssl_wrap_socket] [INFO] THIS CONNECTION IS IN INSECURE MODE. IT MEANS THE CERTIFICATE WILL BE VALIDATED BUT THE CERTIFICATE REVOCATION STATUS WILL NOT BE CHECKED.
[2017-04-15 19:25:15,365] [250] [ChunkDownloader_17] [botocore.vendored.requests.packages.urllib3.util.retry] [DEBUG] Converted retries value: 5 -> Retry(total=5, connect=None, read=None, redirect=None)
[2017-04-15 19:25:15,366] [250] [ChunkDownloader_17] [botocore.vendored.requests.packages.urllib3.util.retry] [DEBUG] Converted retries value: 5 -> Retry(total=5, connect=None, read=None, redirect=None)
[2017-04-15 19:25:15,366] [250] [ChunkDownloader_17] [snowflake.connector.network] [DEBUG] using requests session [0 uses]: active 12, idle: 0
[2017-04-15 19:25:15,369] [250] [RequestExec-20] [botocore.vendored.requests.packages.urllib3.connectionpool] [INFO] Starting new HTTPS connection (1): sfc-ds2-customer-stage.s3.amazonaws.com
[2017-04-15 19:25:15,370] [250] [ChunkDownloader_17] [snowflake.connector.network] [DEBUG] request thread timeout: 60, rest of request timeout: 60, retry cnt: 1
[2017-04-15 19:25:15,401] [250] [RequestExec-17] [botocore.vendored.requests.packages.urllib3.connectionpool] [DEBUG] "GET /udoi-s-bounty2-/results/b1f97eed-9f52-4f6b-a14e-b5de377ae1bf/main/data_0_0_13?x-amz-server-side-encryption-customer-algorithm=AES256&response-content-encoding=gzip&AWSAccessKeyId=AKIAIVMHKINF3K536SLA&Expires=1492305910&Signature=Ny5sLTm2RmOLfKq4RX6GAQHFkeA%3D HTTP/1.1" 200 16812923
[2017-04-15 19:25:15,402] [250] [RequestExec-17] [snowflake.connector.network] [DEBUG] SUCCESS
[2017-04-15 19:25:15,566] [250] [RequestExec-19] [botocore.vendored.requests.packages.urllib3.connectionpool] [DEBUG] "GET /udoi-s-bounty2-/results/b1f97eed-9f52-4f6b-a14e-b5de377ae1bf/main/data_0_0_15?x-amz-server-side-encryption-customer-algorithm=AES256&response-content-encoding=gzip&AWSAccessKeyId=AKIAIVMHKINF3K536SLA&Expires=1492305910&Signature=bjyRIpGdqdHCZXqD0qwPuo7ELcA%3D HTTP/1.1" 200 16809445
[2017-04-15 19:25:15,566] [250] [RequestExec-19] [snowflake.connector.network] [DEBUG] SUCCESS
[2017-04-15 19:25:15,589] [250] [RequestExec-20] [snowflake.connector.ssl_wrap_socket] [DEBUG] insecure_mode: True, OCSP response cache file name: None, PROXY_HOST: None, PROXY_PORT: None, PROXY_USER: None PROXY_PASSWORD: None
[2017-04-15 19:25:15,589] [250] [RequestExec-20] [snowflake.connector.ssl_wrap_socket] [INFO] THIS CONNECTION IS IN INSECURE MODE. IT MEANS THE CERTIFICATE WILL BE VALIDATED BUT THE CERTIFICATE REVOCATION STATUS WILL NOT BE CHECKED.
[2017-04-15 19:25:15,620] [250] [ChunkDownloader_18] [botocore.vendored.requests.packages.urllib3.util.retry] [DEBUG] Converted retries value: 5 -> Retry(total=5, connect=None, read=None, redirect=None)
[2017-04-15 19:25:15,621] [250] [ChunkDownloader_18] [botocore.vendored.requests.packages.urllib3.util.retry] [DEBUG] Converted retries value: 5 -> Retry(total=5, connect=None, read=None, redirect=None)
[2017-04-15 19:25:15,621] [250] [ChunkDownloader_18] [snowflake.connector.network] [DEBUG] using requests session [0 uses]: active 13, idle: 0
[2017-04-15 19:25:15,623] [250] [RequestExec-21] [botocore.vendored.requests.packages.urllib3.connectionpool] [INFO] Starting new HTTPS connection (1): sfc-ds2-customer-stage.s3.amazonaws.com
[2017-04-15 19:25:15,623] [250] [ChunkDownloader_18] [snowflake.connector.network] [DEBUG] request thread timeout: 60, rest of request timeout: 60, retry cnt: 1
[2017-04-15 19:25:15,850] [250] [RequestExec-21] [snowflake.connector.ssl_wrap_socket] [DEBUG] insecure_mode: True, OCSP response cache file name: None, PROXY_HOST: None, PROXY_PORT: None, PROXY_USER: None PROXY_PASSWORD: None
[2017-04-15 19:25:15,850] [250] [RequestExec-21] [snowflake.connector.ssl_wrap_socket] [INFO] THIS CONNECTION IS IN INSECURE MODE. IT MEANS THE CERTIFICATE WILL BE VALIDATED BUT THE CERTIFICATE REVOCATION STATUS WILL NOT BE CHECKED.
[2017-04-15 19:25:15,875] [250] [ChunkDownloader_19] [botocore.vendored.requests.packages.urllib3.util.retry] [DEBUG] Converted retries value: 5 -> Retry(total=5, connect=None, read=None, redirect=None)
[2017-04-15 19:25:15,877] [250] [ChunkDownloader_19] [botocore.vendored.requests.packages.urllib3.util.retry] [DEBUG] Converted retries value: 5 -> Retry(total=5, connect=None, read=None, redirect=None)
[2017-04-15 19:25:15,877] [250] [ChunkDownloader_19] [snowflake.connector.network] [DEBUG] using requests session [0 uses]: active 14, idle: 0
[2017-04-15 19:25:15,878] [250] [RequestExec-22] [botocore.vendored.requests.packages.urllib3.connectionpool] [INFO] Starting new HTTPS connection (1): sfc-ds2-customer-stage.s3.amazonaws.com
[2017-04-15 19:25:15,878] [250] [ChunkDownloader_19] [snowflake.connector.network] [DEBUG] request thread timeout: 60, rest of request timeout: 60, retry cnt: 1
[2017-04-15 19:25:15,879] [250] [RequestExec-20] [botocore.vendored.requests.packages.urllib3.connectionpool] [DEBUG] "GET /udoi-s-bounty2-/results/b1f97eed-9f52-4f6b-a14e-b5de377ae1bf/main/data_0_0_16?x-amz-server-side-encryption-customer-algorithm=AES256&response-content-encoding=gzip&AWSAccessKeyId=AKIAIVMHKINF3K536SLA&Expires=1492305910&Signature=fcCy6EJ6Uiy5pQHTkpCCSHs%2BscY%3D HTTP/1.1" 200 16803057
[2017-04-15 19:25:15,880] [250] [RequestExec-20] [snowflake.connector.network] [DEBUG] SUCCESS
[2017-04-15 19:25:15,969] [250] [RequestExec-21] [botocore.vendored.requests.packages.urllib3.connectionpool] [DEBUG] "GET /udoi-s-bounty2-/results/b1f97eed-9f52-4f6b-a14e-b5de377ae1bf/main/data_0_0_17?x-amz-server-side-encryption-customer-algorithm=AES256&response-content-encoding=gzip&AWSAccessKeyId=AKIAIVMHKINF3K536SLA&Expires=1492305910&Signature=Scg6i7Dl2nu8LHQm%2F%2FS6mCeFKBo%3D HTTP/1.1" 200 16814153
[2017-04-15 19:25:15,969] [250] [RequestExec-21] [snowflake.connector.network] [DEBUG] SUCCESS
[2017-04-15 19:25:16,100] [250] [RequestExec-22] [snowflake.connector.ssl_wrap_socket] [DEBUG] insecure_mode: True, OCSP response cache file name: None, PROXY_HOST: None, PROXY_PORT: None, PROXY_USER: None PROXY_PASSWORD: None
[2017-04-15 19:25:16,101] [250] [RequestExec-22] [snowflake.connector.ssl_wrap_socket] [INFO] THIS CONNECTION IS IN INSECURE MODE. IT MEANS THE CERTIFICATE WILL BE VALIDATED BUT THE CERTIFICATE REVOCATION STATUS WILL NOT BE CHECKED.
[2017-04-15 19:25:16,126] [250] [ChunkDownloader_20] [botocore.vendored.requests.packages.urllib3.util.retry] [DEBUG] Converted retries value: 5 -> Retry(total=5, connect=None, read=None, redirect=None)
[2017-04-15 19:25:16,126] [250] [ChunkDownloader_20] [botocore.vendored.requests.packages.urllib3.util.retry] [DEBUG] Converted retries value: 5 -> Retry(total=5, connect=None, read=None, redirect=None)
[2017-04-15 19:25:16,127] [250] [ChunkDownloader_20] [snowflake.connector.network] [DEBUG] using requests session [0 uses]: active 15, idle: 0
[2017-04-15 19:25:16,128] [250] [RequestExec-23] [botocore.vendored.requests.packages.urllib3.connectionpool] [INFO] Starting new HTTPS connection (1): sfc-ds2-customer-stage.s3.amazonaws.com
[2017-04-15 19:25:16,128] [250] [ChunkDownloader_20] [snowflake.connector.network] [DEBUG] request thread timeout: 60, rest of request timeout: 60, retry cnt: 1
[2017-04-15 19:25:16,266] [250] [RequestExec-22] [botocore.vendored.requests.packages.urllib3.connectionpool] [DEBUG] "GET /udoi-s-bounty2-/results/b1f97eed-9f52-4f6b-a14e-b5de377ae1bf/main/data_0_0_18?x-amz-server-side-encryption-customer-algorithm=AES256&response-content-encoding=gzip&AWSAccessKeyId=AKIAIVMHKINF3K536SLA&Expires=1492305910&Signature=VnxXLMkQQQpHSRA4prs4orEJuU0%3D HTTP/1.1" 200 16810398
[2017-04-15 19:25:16,266] [250] [RequestExec-22] [snowflake.connector.network] [DEBUG] SUCCESS
[2017-04-15 19:25:16,353] [250] [RequestExec-23] [snowflake.connector.ssl_wrap_socket] [DEBUG] insecure_mode: True, OCSP response cache file name: None, PROXY_HOST: None, PROXY_PORT: None, PROXY_USER: None PROXY_PASSWORD: None
[2017-04-15 19:25:16,354] [250] [RequestExec-23] [snowflake.connector.ssl_wrap_socket] [INFO] THIS CONNECTION IS IN INSECURE MODE. IT MEANS THE CERTIFICATE WILL BE VALIDATED BUT THE CERTIFICATE REVOCATION STATUS WILL NOT BE CHECKED.
[2017-04-15 19:25:16,377] [250] [ChunkDownloader_21] [botocore.vendored.requests.packages.urllib3.util.retry] [DEBUG] Converted retries value: 5 -> Retry(total=5, connect=None, read=None, redirect=None)
[2017-04-15 19:25:16,378] [250] [ChunkDownloader_21] [botocore.vendored.requests.packages.urllib3.util.retry] [DEBUG] Converted retries value: 5 -> Retry(total=5, connect=None, read=None, redirect=None)
[2017-04-15 19:25:16,378] [250] [ChunkDownloader_21] [snowflake.connector.network] [DEBUG] using requests session [0 uses]: active 16, idle: 0
[2017-04-15 19:25:16,380] [250] [RequestExec-24] [botocore.vendored.requests.packages.urllib3.connectionpool] [INFO] Starting new HTTPS connection (1): sfc-ds2-customer-stage.s3.amazonaws.com
[2017-04-15 19:25:16,380] [250] [ChunkDownloader_21] [snowflake.connector.network] [DEBUG] request thread timeout: 60, rest of request timeout: 60, retry cnt: 1
[2017-04-15 19:25:16,500] [250] [RequestExec-23] [botocore.vendored.requests.packages.urllib3.connectionpool] [DEBUG] "GET /udoi-s-bounty2-/results/b1f97eed-9f52-4f6b-a14e-b5de377ae1bf/main/data_0_0_19?x-amz-server-side-encryption-customer-algorithm=AES256&response-content-encoding=gzip&AWSAccessKeyId=AKIAIVMHKINF3K536SLA&Expires=1492305910&Signature=V4y5OFM%2FYBIqsCc3Hgn7di78bzU%3D HTTP/1.1" 200 16810537
[2017-04-15 19:25:16,501] [250] [RequestExec-23] [snowflake.connector.network] [DEBUG] SUCCESS
[2017-04-15 19:25:16,602] [250] [RequestExec-24] [snowflake.connector.ssl_wrap_socket] [DEBUG] insecure_mode: True, OCSP response cache file name: None, PROXY_HOST: None, PROXY_PORT: None, PROXY_USER: None PROXY_PASSWORD: None
[2017-04-15 19:25:16,602] [250] [RequestExec-24] [snowflake.connector.ssl_wrap_socket] [INFO] THIS CONNECTION IS IN INSECURE MODE. IT MEANS THE CERTIFICATE WILL BE VALIDATED BUT THE CERTIFICATE REVOCATION STATUS WILL NOT BE CHECKED.
[2017-04-15 19:25:16,628] [250] [ChunkDownloader_22] [botocore.vendored.requests.packages.urllib3.util.retry] [DEBUG] Converted retries value: 5 -> Retry(total=5, connect=None, read=None, redirect=None)
[2017-04-15 19:25:16,629] [250] [ChunkDownloader_22] [botocore.vendored.requests.packages.urllib3.util.retry] [DEBUG] Converted retries value: 5 -> Retry(total=5, connect=None, read=None, redirect=None)
[2017-04-15 19:25:16,629] [250] [ChunkDownloader_22] [snowflake.connector.network] [DEBUG] using requests session [0 uses]: active 17, idle: 0
[2017-04-15 19:25:16,630] [250] [RequestExec-25] [botocore.vendored.requests.packages.urllib3.connectionpool] [INFO] Starting new HTTPS connection (1): sfc-ds2-customer-stage.s3.amazonaws.com
[2017-04-15 19:25:16,630] [250] [ChunkDownloader_22] [snowflake.connector.network] [DEBUG] request thread timeout: 60, rest of request timeout: 60, retry cnt: 1
[2017-04-15 19:25:16,795] [250] [RequestExec-24] [botocore.vendored.requests.packages.urllib3.connectionpool] [DEBUG] "GET /udoi-s-bounty2-/results/b1f97eed-9f52-4f6b-a14e-b5de377ae1bf/main/data_0_0_20?x-amz-server-side-encryption-customer-algorithm=AES256&response-content-encoding=gzip&AWSAccessKeyId=AKIAIVMHKINF3K536SLA&Expires=1492305910&Signature=95TQqfnpPN4T%2BDVB3MoEwkVUINw%3D HTTP/1.1" 200 16810496
[2017-04-15 19:25:16,796] [250] [RequestExec-24] [snowflake.connector.network] [DEBUG] SUCCESS
[2017-04-15 19:25:16,851] [250] [RequestExec-25] [snowflake.connector.ssl_wrap_socket] [DEBUG] insecure_mode: True, OCSP response cache file name: None, PROXY_HOST: None, PROXY_PORT: None, PROXY_USER: None PROXY_PASSWORD: None
[2017-04-15 19:25:16,851] [250] [RequestExec-25] [snowflake.connector.ssl_wrap_socket] [INFO] THIS CONNECTION IS IN INSECURE MODE. IT MEANS THE CERTIFICATE WILL BE VALIDATED BUT THE CERTIFICATE REVOCATION STATUS WILL NOT BE CHECKED.
[2017-04-15 19:25:16,880] [250] [ChunkDownloader_23] [botocore.vendored.requests.packages.urllib3.util.retry] [DEBUG] Converted retries value: 5 -> Retry(total=5, connect=None, read=None, redirect=None)
[2017-04-15 19:25:16,881] [250] [ChunkDownloader_23] [botocore.vendored.requests.packages.urllib3.util.retry] [DEBUG] Converted retries value: 5 -> Retry(total=5, connect=None, read=None, redirect=None)
[2017-04-15 19:25:16,881] [250] [ChunkDownloader_23] [snowflake.connector.network] [DEBUG] using requests session [0 uses]: active 18, idle: 0
[2017-04-15 19:25:16,882] [250] [RequestExec-26] [botocore.vendored.requests.packages.urllib3.connectionpool] [INFO] Starting new HTTPS connection (1): sfc-ds2-customer-stage.s3.amazonaws.com
[2017-04-15 19:25:16,883] [250] [ChunkDownloader_23] [snowflake.connector.network] [DEBUG] request thread timeout: 60, rest of request timeout: 60, retry cnt: 1
[2017-04-15 19:25:16,995] [250] [RequestExec-25] [botocore.vendored.requests.packages.urllib3.connectionpool] [DEBUG] "GET /udoi-s-bounty2-/results/b1f97eed-9f52-4f6b-a14e-b5de377ae1bf/main/data_0_0_21?x-amz-server-side-encryption-customer-algorithm=AES256&response-content-encoding=gzip&AWSAccessKeyId=AKIAIVMHKINF3K536SLA&Expires=1492305910&Signature=9KnZ0VpDQAFTwr5vJoqC4%2B3SCKI%3D HTTP/1.1" 200 16805256
[2017-04-15 19:25:16,996] [250] [RequestExec-25] [snowflake.connector.network] [DEBUG] SUCCESS
[2017-04-15 19:25:17,104] [250] [RequestExec-26] [snowflake.connector.ssl_wrap_socket] [DEBUG] insecure_mode: True, OCSP response cache file name: None, PROXY_HOST: None, PROXY_PORT: None, PROXY_USER: None PROXY_PASSWORD: None
[2017-04-15 19:25:17,105] [250] [RequestExec-26] [snowflake.connector.ssl_wrap_socket] [INFO] THIS CONNECTION IS IN INSECURE MODE. IT MEANS THE CERTIFICATE WILL BE VALIDATED BUT THE CERTIFICATE REVOCATION STATUS WILL NOT BE CHECKED.
[2017-04-15 19:25:17,131] [250] [ChunkDownloader_24] [botocore.vendored.requests.packages.urllib3.util.retry] [DEBUG] Converted retries value: 5 -> Retry(total=5, connect=None, read=None, redirect=None)
[2017-04-15 19:25:17,132] [250] [ChunkDownloader_24] [botocore.vendored.requests.packages.urllib3.util.retry] [DEBUG] Converted retries value: 5 -> Retry(total=5, connect=None, read=None, redirect=None)
[2017-04-15 19:25:17,132] [250] [ChunkDownloader_24] [snowflake.connector.network] [DEBUG] using requests session [0 uses]: active 19, idle: 0
[2017-04-15 19:25:17,134] [250] [RequestExec-27] [botocore.vendored.requests.packages.urllib3.connectionpool] [INFO] Starting new HTTPS connection (1): sfc-ds2-customer-stage.s3.amazonaws.com
[2017-04-15 19:25:17,134] [250] [ChunkDownloader_24] [snowflake.connector.network] [DEBUG] request thread timeout: 60, rest of request timeout: 60, retry cnt: 1
[2017-04-15 19:25:17,236] [250] [RequestExec-26] [botocore.vendored.requests.packages.urllib3.connectionpool] [DEBUG] "GET /udoi-s-bounty2-/results/b1f97eed-9f52-4f6b-a14e-b5de377ae1bf/main/data_0_0_22?x-amz-server-side-encryption-customer-algorithm=AES256&response-content-encoding=gzip&AWSAccessKeyId=AKIAIVMHKINF3K536SLA&Expires=1492305910&Signature=5AOj4JyGnkFo%2BBgabunh90PGXkI%3D HTTP/1.1" 200 16815963
[2017-04-15 19:25:17,237] [250] [RequestExec-26] [snowflake.connector.network] [DEBUG] SUCCESS
[2017-04-15 19:25:17,356] [250] [RequestExec-27] [snowflake.connector.ssl_wrap_socket] [DEBUG] insecure_mode: True, OCSP response cache file name: None, PROXY_HOST: None, PROXY_PORT: None, PROXY_USER: None PROXY_PASSWORD: None
[2017-04-15 19:25:17,357] [250] [RequestExec-27] [snowflake.connector.ssl_wrap_socket] [INFO] THIS CONNECTION IS IN INSECURE MODE. IT MEANS THE CERTIFICATE WILL BE VALIDATED BUT THE CERTIFICATE REVOCATION STATUS WILL NOT BE CHECKED.
[2017-04-15 19:25:17,383] [250] [ChunkDownloader_25] [botocore.vendored.requests.packages.urllib3.util.retry] [DEBUG] Converted retries value: 5 -> Retry(total=5, connect=None, read=None, redirect=None)
[2017-04-15 19:25:17,384] [250] [ChunkDownloader_25] [botocore.vendored.requests.packages.urllib3.util.retry] [DEBUG] Converted retries value: 5 -> Retry(total=5, connect=None, read=None, redirect=None)
[2017-04-15 19:25:17,384] [250] [ChunkDownloader_25] [snowflake.connector.network] [DEBUG] using requests session [0 uses]: active 20, idle: 0
[2017-04-15 19:25:17,386] [250] [RequestExec-28] [botocore.vendored.requests.packages.urllib3.connectionpool] [INFO] Starting new HTTPS connection (1): sfc-ds2-customer-stage.s3.amazonaws.com
[2017-04-15 19:25:17,386] [250] [ChunkDownloader_25] [snowflake.connector.network] [DEBUG] request thread timeout: 60, rest of request timeout: 60, retry cnt: 1
[2017-04-15 19:25:17,609] [250] [RequestExec-28] [snowflake.connector.ssl_wrap_socket] [DEBUG] insecure_mode: True, OCSP response cache file name: None, PROXY_HOST: None, PROXY_PORT: None, PROXY_USER: None PROXY_PASSWORD: None
[2017-04-15 19:25:17,609] [250] [RequestExec-28] [snowflake.connector.ssl_wrap_socket] [INFO] THIS CONNECTION IS IN INSECURE MODE. IT MEANS THE CERTIFICATE WILL BE VALIDATED BUT THE CERTIFICATE REVOCATION STATUS WILL NOT BE CHECKED.
[2017-04-15 19:25:17,637] [250] [ChunkDownloader_26] [botocore.vendored.requests.packages.urllib3.util.retry] [DEBUG] Converted retries value: 5 -> Retry(total=5, connect=None, read=None, redirect=None)
[2017-04-15 19:25:17,637] [250] [ChunkDownloader_26] [botocore.vendored.requests.packages.urllib3.util.retry] [DEBUG] Converted retries value: 5 -> Retry(total=5, connect=None, read=None, redirect=None)
[2017-04-15 19:25:17,638] [250] [ChunkDownloader_26] [snowflake.connector.network] [DEBUG] using requests session [0 uses]: active 21, idle: 0
[2017-04-15 19:25:17,639] [250] [RequestExec-29] [botocore.vendored.requests.packages.urllib3.connectionpool] [INFO] Starting new HTTPS connection (1): sfc-ds2-customer-stage.s3.amazonaws.com
[2017-04-15 19:25:17,640] [250] [ChunkDownloader_26] [snowflake.connector.network] [DEBUG] request thread timeout: 60, rest of request timeout: 60, retry cnt: 1
[2017-04-15 19:25:17,728] [250] [RequestExec-27] [botocore.vendored.requests.packages.urllib3.connectionpool] [DEBUG] "GET /udoi-s-bounty2-/results/b1f97eed-9f52-4f6b-a14e-b5de377ae1bf/main/data_0_0_23?x-amz-server-side-encryption-customer-algorithm=AES256&response-content-encoding=gzip&AWSAccessKeyId=AKIAIVMHKINF3K536SLA&Expires=1492305910&Signature=pWtKL7kgriRwMpOGe5uAzy7XcgI%3D HTTP/1.1" 200 16802566
[2017-04-15 19:25:17,729] [250] [RequestExec-27] [snowflake.connector.network] [DEBUG] SUCCESS
[2017-04-15 19:25:17,764] [250] [RequestExec-28] [botocore.vendored.requests.packages.urllib3.connectionpool] [DEBUG] "GET /udoi-s-bounty2-/results/b1f97eed-9f52-4f6b-a14e-b5de377ae1bf/main/data_0_0_24?x-amz-server-side-encryption-customer-algorithm=AES256&response-content-encoding=gzip&AWSAccessKeyId=AKIAIVMHKINF3K536SLA&Expires=1492305910&Signature=meIDK6%2FjnqBcXGtELbTSNc8qcEc%3D HTTP/1.1" 200 16805944
[2017-04-15 19:25:17,765] [250] [RequestExec-28] [snowflake.connector.network] [DEBUG] SUCCESS
[2017-04-15 19:25:17,854] [250] [RequestExec-29] [snowflake.connector.ssl_wrap_socket] [DEBUG] insecure_mode: True, OCSP response cache file name: None, PROXY_HOST: None, PROXY_PORT: None, PROXY_USER: None PROXY_PASSWORD: None
[2017-04-15 19:25:17,855] [250] [RequestExec-29] [snowflake.connector.ssl_wrap_socket] [INFO] THIS CONNECTION IS IN INSECURE MODE. IT MEANS THE CERTIFICATE WILL BE VALIDATED BUT THE CERTIFICATE REVOCATION STATUS WILL NOT BE CHECKED.
[2017-04-15 19:25:17,888] [250] [ChunkDownloader_27] [botocore.vendored.requests.packages.urllib3.util.retry] [DEBUG] Converted retries value: 5 -> Retry(total=5, connect=None, read=None, redirect=None)
[2017-04-15 19:25:17,889] [250] [ChunkDownloader_27] [botocore.vendored.requests.packages.urllib3.util.retry] [DEBUG] Converted retries value: 5 -> Retry(total=5, connect=None, read=None, redirect=None)
[2017-04-15 19:25:17,889] [250] [ChunkDownloader_27] [snowflake.connector.network] [DEBUG] using requests session [0 uses]: active 22, idle: 0
[2017-04-15 19:25:17,891] [250] [RequestExec-30] [botocore.vendored.requests.packages.urllib3.connectionpool] [INFO] Starting new HTTPS connection (1): sfc-ds2-customer-stage.s3.amazonaws.com
[2017-04-15 19:25:17,891] [250] [ChunkDownloader_27] [snowflake.connector.network] [DEBUG] request thread timeout: 60, rest of request timeout: 60, retry cnt: 1
[2017-04-15 19:25:18,017] [250] [RequestExec-29] [botocore.vendored.requests.packages.urllib3.connectionpool] [DEBUG] "GET /udoi-s-bounty2-/results/b1f97eed-9f52-4f6b-a14e-b5de377ae1bf/main/data_0_0_25?x-amz-server-side-encryption-customer-algorithm=AES256&response-content-encoding=gzip&AWSAccessKeyId=AKIAIVMHKINF3K536SLA&Expires=1492305910&Signature=r95K7sdj%2F2jaxgt5%2FbxRXRWGgh8%3D HTTP/1.1" 200 16800992
[2017-04-15 19:25:18,018] [250] [RequestExec-29] [snowflake.connector.network] [DEBUG] SUCCESS
[2017-04-15 19:25:18,112] [250] [RequestExec-30] [snowflake.connector.ssl_wrap_socket] [DEBUG] insecure_mode: True, OCSP response cache file name: None, PROXY_HOST: None, PROXY_PORT: None, PROXY_USER: None PROXY_PASSWORD: None
[2017-04-15 19:25:18,113] [250] [RequestExec-30] [snowflake.connector.ssl_wrap_socket] [INFO] THIS CONNECTION IS IN INSECURE MODE. IT MEANS THE CERTIFICATE WILL BE VALIDATED BUT THE CERTIFICATE REVOCATION STATUS WILL NOT BE CHECKED.
[2017-04-15 19:25:18,140] [250] [ChunkDownloader_28] [botocore.vendored.requests.packages.urllib3.util.retry] [DEBUG] Converted retries value: 5 -> Retry(total=5, connect=None, read=None, redirect=None)
[2017-04-15 19:25:18,141] [250] [ChunkDownloader_28] [botocore.vendored.requests.packages.urllib3.util.retry] [DEBUG] Converted retries value: 5 -> Retry(total=5, connect=None, read=None, redirect=None)
[2017-04-15 19:25:18,141] [250] [ChunkDownloader_28] [snowflake.connector.network] [DEBUG] using requests session [0 uses]: active 23, idle: 0
[2017-04-15 19:25:18,142] [250] [RequestExec-31] [botocore.vendored.requests.packages.urllib3.connectionpool] [INFO] Starting new HTTPS connection (1): sfc-ds2-customer-stage.s3.amazonaws.com
[2017-04-15 19:25:18,143] [250] [ChunkDownloader_28] [snowflake.connector.network] [DEBUG] request thread timeout: 60, rest of request timeout: 60, retry cnt: 1
[2017-04-15 19:25:18,228] [250] [ChunkDownloader_7] [snowflake.connector.network] [DEBUG] request thread joined
[2017-04-15 19:25:18,228] [250] [ChunkDownloader_7] [snowflake.connector.network] [DEBUG] request thread returned object
[2017-04-15 19:25:18,229] [250] [ChunkDownloader_7] [snowflake.connector.network] [DEBUG] freed requests sessions: active 22, idle: 0
[2017-04-15 19:25:18,230] [250] [ChunkDownloader_7] [BENCHMARK] [INFO] Chunk download time: 4682 ms (0 ms latency)
[2017-04-15 19:25:18,231] [250] [ChunkDownloader_29] [snowflake.connector.network] [DEBUG] using requests session [0 uses]: active 23, idle: 0
[2017-04-15 19:25:18,239] [250] [ChunkDownloader_29] [snowflake.connector.network] [DEBUG] request thread timeout: 60, rest of request timeout: 60, retry cnt: 1
[2017-04-15 19:25:18,250] [250] [RequestExec-30] [botocore.vendored.requests.packages.urllib3.connectionpool] [DEBUG] "GET /udoi-s-bounty2-/results/b1f97eed-9f52-4f6b-a14e-b5de377ae1bf/main/data_0_0_26?x-amz-server-side-encryption-customer-algorithm=AES256&response-content-encoding=gzip&AWSAccessKeyId=AKIAIVMHKINF3K536SLA&Expires=1492305910&Signature=08tzMUXgVe80RVm4%2BMSy7CEsHmw%3D HTTP/1.1" 200 16810024
[2017-04-15 19:25:18,251] [250] [RequestExec-30] [snowflake.connector.network] [DEBUG] SUCCESS
[2017-04-15 19:25:18,385] [250] [RequestExec-31] [snowflake.connector.ssl_wrap_socket] [DEBUG] insecure_mode: True, OCSP response cache file name: None, PROXY_HOST: None, PROXY_PORT: None, PROXY_USER: None PROXY_PASSWORD: None
[2017-04-15 19:25:18,386] [250] [RequestExec-31] [snowflake.connector.ssl_wrap_socket] [INFO] THIS CONNECTION IS IN INSECURE MODE. IT MEANS THE CERTIFICATE WILL BE VALIDATED BUT THE CERTIFICATE REVOCATION STATUS WILL NOT BE CHECKED.
[2017-04-15 19:25:18,391] [250] [ChunkDownloader_30] [botocore.vendored.requests.packages.urllib3.util.retry] [DEBUG] Converted retries value: 5 -> Retry(total=5, connect=None, read=None, redirect=None)
[2017-04-15 19:25:18,391] [250] [ChunkDownloader_30] [botocore.vendored.requests.packages.urllib3.util.retry] [DEBUG] Converted retries value: 5 -> Retry(total=5, connect=None, read=None, redirect=None)
[2017-04-15 19:25:18,392] [250] [ChunkDownloader_30] [snowflake.connector.network] [DEBUG] using requests session [0 uses]: active 24, idle: 0
[2017-04-15 19:25:18,393] [250] [RequestExec-33] [botocore.vendored.requests.packages.urllib3.connectionpool] [INFO] Starting new HTTPS connection (1): sfc-ds2-customer-stage.s3.amazonaws.com
[2017-04-15 19:25:18,394] [250] [ChunkDownloader_30] [snowflake.connector.network] [DEBUG] request thread timeout: 60, rest of request timeout: 60, retry cnt: 1
[2017-04-15 19:25:18,444] [250] [RequestExec-32] [botocore.vendored.requests.packages.urllib3.connectionpool] [DEBUG] "GET /udoi-s-bounty2-/results/b1f97eed-9f52-4f6b-a14e-b5de377ae1bf/main/data_0_0_28?x-amz-server-side-encryption-customer-algorithm=AES256&response-content-encoding=gzip&AWSAccessKeyId=AKIAIVMHKINF3K536SLA&Expires=1492305910&Signature=L9Dto6hmqIvd53ehRm86XEqx9NM%3D HTTP/1.1" 200 16805073
[2017-04-15 19:25:18,445] [250] [RequestExec-32] [snowflake.connector.network] [DEBUG] SUCCESS
[2017-04-15 19:25:18,615] [250] [RequestExec-33] [snowflake.connector.ssl_wrap_socket] [DEBUG] insecure_mode: True, OCSP response cache file name: None, PROXY_HOST: None, PROXY_PORT: None, PROXY_USER: None PROXY_PASSWORD: None
[2017-04-15 19:25:18,615] [250] [RequestExec-33] [snowflake.connector.ssl_wrap_socket] [INFO] THIS CONNECTION IS IN INSECURE MODE. IT MEANS THE CERTIFICATE WILL BE VALIDATED BUT THE CERTIFICATE REVOCATION STATUS WILL NOT BE CHECKED.
[2017-04-15 19:25:18,616] [250] [RequestExec-31] [botocore.vendored.requests.packages.urllib3.connectionpool] [DEBUG] "GET /udoi-s-bounty2-/results/b1f97eed-9f52-4f6b-a14e-b5de377ae1bf/main/data_0_0_27?x-amz-server-side-encryption-customer-algorithm=AES256&response-content-encoding=gzip&AWSAccessKeyId=AKIAIVMHKINF3K536SLA&Expires=1492305910&Signature=IkDXqaKAdSJFhY222%2BBEXBo01s0%3D HTTP/1.1" 200 16812732
[2017-04-15 19:25:18,617] [250] [RequestExec-31] [snowflake.connector.network] [DEBUG] SUCCESS
[2017-04-15 19:25:18,643] [250] [ChunkDownloader_31] [botocore.vendored.requests.packages.urllib3.util.retry] [DEBUG] Converted retries value: 5 -> Retry(total=5, connect=None, read=None, redirect=None)
[2017-04-15 19:25:18,643] [250] [ChunkDownloader_31] [botocore.vendored.requests.packages.urllib3.util.retry] [DEBUG] Converted retries value: 5 -> Retry(total=5, connect=None, read=None, redirect=None)
[2017-04-15 19:25:18,644] [250] [ChunkDownloader_31] [snowflake.connector.network] [DEBUG] using requests session [0 uses]: active 25, idle: 0
[2017-04-15 19:25:18,645] [250] [RequestExec-34] [botocore.vendored.requests.packages.urllib3.connectionpool] [INFO] Starting new HTTPS connection (1): sfc-ds2-customer-stage.s3.amazonaws.com
[2017-04-15 19:25:18,646] [250] [ChunkDownloader_31] [snowflake.connector.network] [DEBUG] request thread timeout: 60, rest of request timeout: 60, retry cnt: 1
[2017-04-15 19:25:18,757] [250] [RequestExec-33] [botocore.vendored.requests.packages.urllib3.connectionpool] [DEBUG] "GET /udoi-s-bounty2-/results/b1f97eed-9f52-4f6b-a14e-b5de377ae1bf/main/data_0_0_29?x-amz-server-side-encryption-customer-algorithm=AES256&response-content-encoding=gzip&AWSAccessKeyId=AKIAIVMHKINF3K536SLA&Expires=1492305910&Signature=v29DCiSm6eeMFb7v3CYkNegBLvo%3D HTTP/1.1" 200 16801815
[2017-04-15 19:25:18,758] [250] [RequestExec-33] [snowflake.connector.network] [DEBUG] SUCCESS
[2017-04-15 19:25:18,858] [250] [RequestExec-34] [snowflake.connector.ssl_wrap_socket] [DEBUG] insecure_mode: True, OCSP response cache file name: None, PROXY_HOST: None, PROXY_PORT: None, PROXY_USER: None PROXY_PASSWORD: None
[2017-04-15 19:25:18,859] [250] [RequestExec-34] [snowflake.connector.ssl_wrap_socket] [INFO] THIS CONNECTION IS IN INSECURE MODE. IT MEANS THE CERTIFICATE WILL BE VALIDATED BUT THE CERTIFICATE REVOCATION STATUS WILL NOT BE CHECKED.
[2017-04-15 19:25:18,895] [250] [ChunkDownloader_32] [botocore.vendored.requests.packages.urllib3.util.retry] [DEBUG] Converted retries value: 5 -> Retry(total=5, connect=None, read=None, redirect=None)
[2017-04-15 19:25:18,895] [250] [ChunkDownloader_32] [botocore.vendored.requests.packages.urllib3.util.retry] [DEBUG] Converted retries value: 5 -> Retry(total=5, connect=None, read=None, redirect=None)
[2017-04-15 19:25:18,896] [250] [ChunkDownloader_32] [snowflake.connector.network] [DEBUG] using requests session [0 uses]: active 26, idle: 0
[2017-04-15 19:25:18,897] [250] [RequestExec-35] [botocore.vendored.requests.packages.urllib3.connectionpool] [INFO] Starting new HTTPS connection (1): sfc-ds2-customer-stage.s3.amazonaws.com
[2017-04-15 19:25:18,901] [250] [ChunkDownloader_32] [snowflake.connector.network] [DEBUG] request thread timeout: 60, rest of request timeout: 60, retry cnt: 1
[2017-04-15 19:25:19,061] [250] [RequestExec-34] [botocore.vendored.requests.packages.urllib3.connectionpool] [DEBUG] "GET /udoi-s-bounty2-/results/b1f97eed-9f52-4f6b-a14e-b5de377ae1bf/main/data_0_0_30?x-amz-server-side-encryption-customer-algorithm=AES256&response-content-encoding=gzip&AWSAccessKeyId=AKIAIVMHKINF3K536SLA&Expires=1492305910&Signature=4cVTakKtm%2B7SmPlqYJj73TPdftU%3D HTTP/1.1" 200 16812125
[2017-04-15 19:25:19,061] [250] [RequestExec-34] [snowflake.connector.network] [DEBUG] SUCCESS
[2017-04-15 19:25:19,147] [250] [ChunkDownloader_33] [botocore.vendored.requests.packages.urllib3.util.retry] [DEBUG] Converted retries value: 5 -> Retry(total=5, connect=None, read=None, redirect=None)
[2017-04-15 19:25:19,148] [250] [ChunkDownloader_33] [botocore.vendored.requests.packages.urllib3.util.retry] [DEBUG] Converted retries value: 5 -> Retry(total=5, connect=None, read=None, redirect=None)
[2017-04-15 19:25:19,149] [250] [ChunkDownloader_33] [snowflake.connector.network] [DEBUG] using requests session [0 uses]: active 27, idle: 0
[2017-04-15 19:25:19,150] [250] [RequestExec-36] [botocore.vendored.requests.packages.urllib3.connectionpool] [INFO] Starting new HTTPS connection (1): sfc-ds2-customer-stage.s3.amazonaws.com
[2017-04-15 19:25:19,150] [250] [ChunkDownloader_33] [snowflake.connector.network] [DEBUG] request thread timeout: 60, rest of request timeout: 60, retry cnt: 1
[2017-04-15 19:25:19,351] [250] [RequestExec-35] [snowflake.connector.ssl_wrap_socket] [DEBUG] insecure_mode: True, OCSP response cache file name: None, PROXY_HOST: None, PROXY_PORT: None, PROXY_USER: None PROXY_PASSWORD: None
[2017-04-15 19:25:19,352] [250] [RequestExec-35] [snowflake.connector.ssl_wrap_socket] [INFO] THIS CONNECTION IS IN INSECURE MODE. IT MEANS THE CERTIFICATE WILL BE VALIDATED BUT THE CERTIFICATE REVOCATION STATUS WILL NOT BE CHECKED.
[2017-04-15 19:25:19,378] [250] [RequestExec-36] [snowflake.connector.ssl_wrap_socket] [DEBUG] insecure_mode: True, OCSP response cache file name: None, PROXY_HOST: None, PROXY_PORT: None, PROXY_USER: None PROXY_PASSWORD: None
[2017-04-15 19:25:19,378] [250] [RequestExec-36] [snowflake.connector.ssl_wrap_socket] [INFO] THIS CONNECTION IS IN INSECURE MODE. IT MEANS THE CERTIFICATE WILL BE VALIDATED BUT THE CERTIFICATE REVOCATION STATUS WILL NOT BE CHECKED.
[2017-04-15 19:25:19,398] [250] [ChunkDownloader_34] [botocore.vendored.requests.packages.urllib3.util.retry] [DEBUG] Converted retries value: 5 -> Retry(total=5, connect=None, read=None, redirect=None)
[2017-04-15 19:25:19,398] [250] [ChunkDownloader_34] [botocore.vendored.requests.packages.urllib3.util.retry] [DEBUG] Converted retries value: 5 -> Retry(total=5, connect=None, read=None, redirect=None)
[2017-04-15 19:25:19,399] [250] [ChunkDownloader_34] [snowflake.connector.network] [DEBUG] using requests session [0 uses]: active 28, idle: 0
[2017-04-15 19:25:19,400] [250] [RequestExec-37] [botocore.vendored.requests.packages.urllib3.connectionpool] [INFO] Starting new HTTPS connection (1): sfc-ds2-customer-stage.s3.amazonaws.com
[2017-04-15 19:25:19,400] [250] [ChunkDownloader_34] [snowflake.connector.network] [DEBUG] request thread timeout: 60, rest of request timeout: 60, retry cnt: 1
[2017-04-15 19:25:19,491] [250] [RequestExec-35] [botocore.vendored.requests.packages.urllib3.connectionpool] [DEBUG] "GET /udoi-s-bounty2-/results/b1f97eed-9f52-4f6b-a14e-b5de377ae1bf/main/data_0_0_31?x-amz-server-side-encryption-customer-algorithm=AES256&response-content-encoding=gzip&AWSAccessKeyId=AKIAIVMHKINF3K536SLA&Expires=1492305910&Signature=VbxYTP5yoBbC0GtmnJgLIZU3cbc%3D HTTP/1.1" 200 16812609
[2017-04-15 19:25:19,492] [250] [RequestExec-35] [snowflake.connector.network] [DEBUG] SUCCESS
[2017-04-15 19:25:19,610] [250] [RequestExec-36] [botocore.vendored.requests.packages.urllib3.connectionpool] [DEBUG] "GET /udoi-s-bounty2-/results/b1f97eed-9f52-4f6b-a14e-b5de377ae1bf/main/data_0_0_32?x-amz-server-side-encryption-customer-algorithm=AES256&response-content-encoding=gzip&AWSAccessKeyId=AKIAIVMHKINF3K536SLA&Expires=1492305910&Signature=AXuTMqO%2Bi8QH18ZtjyT3QzP6WUg%3D HTTP/1.1" 200 16812522
[2017-04-15 19:25:19,611] [250] [RequestExec-36] [snowflake.connector.network] [DEBUG] SUCCESS
[2017-04-15 19:25:19,627] [250] [RequestExec-37] [snowflake.connector.ssl_wrap_socket] [DEBUG] insecure_mode: True, OCSP response cache file name: None, PROXY_HOST: None, PROXY_PORT: None, PROXY_USER: None PROXY_PASSWORD: None
[2017-04-15 19:25:19,627] [250] [RequestExec-37] [snowflake.connector.ssl_wrap_socket] [INFO] THIS CONNECTION IS IN INSECURE MODE. IT MEANS THE CERTIFICATE WILL BE VALIDATED BUT THE CERTIFICATE REVOCATION STATUS WILL NOT BE CHECKED.
[2017-04-15 19:25:19,649] [250] [ChunkDownloader_35] [botocore.vendored.requests.packages.urllib3.util.retry] [DEBUG] Converted retries value: 5 -> Retry(total=5, connect=None, read=None, redirect=None)
[2017-04-15 19:25:19,649] [250] [ChunkDownloader_35] [botocore.vendored.requests.packages.urllib3.util.retry] [DEBUG] Converted retries value: 5 -> Retry(total=5, connect=None, read=None, redirect=None)
[2017-04-15 19:25:19,650] [250] [ChunkDownloader_35] [snowflake.connector.network] [DEBUG] using requests session [0 uses]: active 29, idle: 0
[2017-04-15 19:25:19,651] [250] [RequestExec-38] [botocore.vendored.requests.packages.urllib3.connectionpool] [INFO] Starting new HTTPS connection (1): sfc-ds2-customer-stage.s3.amazonaws.com
[2017-04-15 19:25:19,653] [250] [ChunkDownloader_35] [snowflake.connector.network] [DEBUG] request thread timeout: 60, rest of request timeout: 60, retry cnt: 1
[2017-04-15 19:25:19,757] [250] [ChunkDownloader_9] [snowflake.connector.network] [DEBUG] request thread joined
[2017-04-15 19:25:19,759] [250] [ChunkDownloader_9] [snowflake.connector.network] [DEBUG] request thread returned object
[2017-04-15 19:25:19,760] [250] [ChunkDownloader_9] [snowflake.connector.network] [DEBUG] freed requests sessions: active 28, idle: 0
[2017-04-15 19:25:19,760] [250] [ChunkDownloader_9] [BENCHMARK] [INFO] Chunk download time: 5718 ms (0 ms latency)
[2017-04-15 19:25:19,767] [250] [ChunkDownloader_36] [snowflake.connector.network] [DEBUG] using requests session [1 uses]: active 29, idle: 0
[2017-04-15 19:25:19,773] [250] [ChunkDownloader_36] [snowflake.connector.network] [DEBUG] request thread timeout: 60, rest of request timeout: 60, retry cnt: 1
[2017-04-15 19:25:19,779] [250] [RequestExec-37] [botocore.vendored.requests.packages.urllib3.connectionpool] [DEBUG] "GET /udoi-s-bounty2-/results/b1f97eed-9f52-4f6b-a14e-b5de377ae1bf/main/data_0_0_33?x-amz-server-side-encryption-customer-algorithm=AES256&response-content-encoding=gzip&AWSAccessKeyId=AKIAIVMHKINF3K536SLA&Expires=1492305910&Signature=hMXVtdrIUKs4E7wP7SdYaPyCIuA%3D HTTP/1.1" 200 16805144
[2017-04-15 19:25:19,781] [250] [RequestExec-37] [snowflake.connector.network] [DEBUG] SUCCESS
[2017-04-15 19:25:19,900] [250] [ChunkDownloader_37] [botocore.vendored.requests.packages.urllib3.util.retry] [DEBUG] Converted retries value: 5 -> Retry(total=5, connect=None, read=None, redirect=None)
[2017-04-15 19:25:19,900] [250] [ChunkDownloader_37] [botocore.vendored.requests.packages.urllib3.util.retry] [DEBUG] Converted retries value: 5 -> Retry(total=5, connect=None, read=None, redirect=None)
[2017-04-15 19:25:19,901] [250] [ChunkDownloader_37] [snowflake.connector.network] [DEBUG] using requests session [0 uses]: active 30, idle: 0
[2017-04-15 19:25:19,902] [250] [RequestExec-40] [botocore.vendored.requests.packages.urllib3.connectionpool] [INFO] Starting new HTTPS connection (1): sfc-ds2-customer-stage.s3.amazonaws.com
[2017-04-15 19:25:19,905] [250] [ChunkDownloader_37] [snowflake.connector.network] [DEBUG] request thread timeout: 60, rest of request timeout: 60, retry cnt: 1
[2017-04-15 19:25:19,928] [250] [RequestExec-38] [snowflake.connector.ssl_wrap_socket] [DEBUG] insecure_mode: True, OCSP response cache file name: None, PROXY_HOST: None, PROXY_PORT: None, PROXY_USER: None PROXY_PASSWORD: None
[2017-04-15 19:25:19,928] [250] [RequestExec-38] [snowflake.connector.ssl_wrap_socket] [INFO] THIS CONNECTION IS IN INSECURE MODE. IT MEANS THE CERTIFICATE WILL BE VALIDATED BUT THE CERTIFICATE REVOCATION STATUS WILL NOT BE CHECKED.
[2017-04-15 19:25:19,959] [250] [RequestExec-39] [botocore.vendored.requests.packages.urllib3.connectionpool] [DEBUG] "GET /udoi-s-bounty2-/results/b1f97eed-9f52-4f6b-a14e-b5de377ae1bf/main/data_0_0_35?x-amz-server-side-encryption-customer-algorithm=AES256&response-content-encoding=gzip&AWSAccessKeyId=AKIAIVMHKINF3K536SLA&Expires=1492305910&Signature=8XYmFNmVulnP8oXf2PQQQf21vFI%3D HTTP/1.1" 200 16801989
[2017-04-15 19:25:19,960] [250] [RequestExec-39] [snowflake.connector.network] [DEBUG] SUCCESS
[2017-04-15 19:25:20,095] [250] [RequestExec-38] [botocore.vendored.requests.packages.urllib3.connectionpool] [DEBUG] "GET /udoi-s-bounty2-/results/b1f97eed-9f52-4f6b-a14e-b5de377ae1bf/main/data_0_0_34?x-amz-server-side-encryption-customer-algorithm=AES256&response-content-encoding=gzip&AWSAccessKeyId=AKIAIVMHKINF3K536SLA&Expires=1492305910&Signature=jwlFjSs66DfqekFNtMFDSdcROTU%3D HTTP/1.1" 200 16813423
[2017-04-15 19:25:20,096] [250] [RequestExec-38] [snowflake.connector.network] [DEBUG] SUCCESS
[2017-04-15 19:25:20,130] [250] [RequestExec-40] [snowflake.connector.ssl_wrap_socket] [DEBUG] insecure_mode: True, OCSP response cache file name: None, PROXY_HOST: None, PROXY_PORT: None, PROXY_USER: None PROXY_PASSWORD: None
[2017-04-15 19:25:20,130] [250] [RequestExec-40] [snowflake.connector.ssl_wrap_socket] [INFO] THIS CONNECTION IS IN INSECURE MODE. IT MEANS THE CERTIFICATE WILL BE VALIDATED BUT THE CERTIFICATE REVOCATION STATUS WILL NOT BE CHECKED.
[2017-04-15 19:25:20,151] [250] [ChunkDownloader_38] [botocore.vendored.requests.packages.urllib3.util.retry] [DEBUG] Converted retries value: 5 -> Retry(total=5, connect=None, read=None, redirect=None)
[2017-04-15 19:25:20,152] [250] [ChunkDownloader_38] [botocore.vendored.requests.packages.urllib3.util.retry] [DEBUG] Converted retries value: 5 -> Retry(total=5, connect=None, read=None, redirect=None)
[2017-04-15 19:25:20,152] [250] [ChunkDownloader_38] [snowflake.connector.network] [DEBUG] using requests session [0 uses]: active 31, idle: 0
[2017-04-15 19:25:20,154] [250] [RequestExec-41] [botocore.vendored.requests.packages.urllib3.connectionpool] [INFO] Starting new HTTPS connection (1): sfc-ds2-customer-stage.s3.amazonaws.com
[2017-04-15 19:25:20,154] [250] [ChunkDownloader_38] [snowflake.connector.network] [DEBUG] request thread timeout: 60, rest of request timeout: 60, retry cnt: 1
[2017-04-15 19:25:20,374] [250] [RequestExec-41] [snowflake.connector.ssl_wrap_socket] [DEBUG] insecure_mode: True, OCSP response cache file name: None, PROXY_HOST: None, PROXY_PORT: None, PROXY_USER: None PROXY_PASSWORD: None
[2017-04-15 19:25:20,374] [250] [RequestExec-41] [snowflake.connector.ssl_wrap_socket] [INFO] THIS CONNECTION IS IN INSECURE MODE. IT MEANS THE CERTIFICATE WILL BE VALIDATED BUT THE CERTIFICATE REVOCATION STATUS WILL NOT BE CHECKED.
[2017-04-15 19:25:20,403] [250] [ChunkDownloader_39] [botocore.vendored.requests.packages.urllib3.util.retry] [DEBUG] Converted retries value: 5 -> Retry(total=5, connect=None, read=None, redirect=None)
[2017-04-15 19:25:20,404] [250] [ChunkDownloader_39] [botocore.vendored.requests.packages.urllib3.util.retry] [DEBUG] Converted retries value: 5 -> Retry(total=5, connect=None, read=None, redirect=None)
[2017-04-15 19:25:20,407] [250] [ChunkDownloader_39] [snowflake.connector.network] [DEBUG] using requests session [0 uses]: active 32, idle: 0
[2017-04-15 19:25:20,408] [250] [RequestExec-42] [botocore.vendored.requests.packages.urllib3.connectionpool] [INFO] Starting new HTTPS connection (1): sfc-ds2-customer-stage.s3.amazonaws.com
[2017-04-15 19:25:20,409] [250] [ChunkDownloader_39] [snowflake.connector.network] [DEBUG] request thread timeout: 60, rest of request timeout: 60, retry cnt: 1
[2017-04-15 19:25:20,559] [250] [RequestExec-41] [botocore.vendored.requests.packages.urllib3.connectionpool] [DEBUG] "GET /udoi-s-bounty2-/results/b1f97eed-9f52-4f6b-a14e-b5de377ae1bf/main/data_0_0_37?x-amz-server-side-encryption-customer-algorithm=AES256&response-content-encoding=gzip&AWSAccessKeyId=AKIAIVMHKINF3K536SLA&Expires=1492305910&Signature=kU%2FnBrV8u4jIwt9hbMq%2BPifNAPM%3D HTTP/1.1" 200 16804530
[2017-04-15 19:25:20,560] [250] [RequestExec-41] [snowflake.connector.network] [DEBUG] SUCCESS
[2017-04-15 19:25:20,630] [250] [RequestExec-40] [botocore.vendored.requests.packages.urllib3.connectionpool] [DEBUG] "GET /udoi-s-bounty2-/results/b1f97eed-9f52-4f6b-a14e-b5de377ae1bf/main/data_0_0_36?x-amz-server-side-encryption-customer-algorithm=AES256&response-content-encoding=gzip&AWSAccessKeyId=AKIAIVMHKINF3K536SLA&Expires=1492305910&Signature=1ON5oNw4ISZpnlsO4RaKDO8n9z0%3D HTTP/1.1" 200 16802814
[2017-04-15 19:25:20,631] [250] [RequestExec-40] [snowflake.connector.network] [DEBUG] SUCCESS
[2017-04-15 19:25:20,643] [250] [RequestExec-42] [snowflake.connector.ssl_wrap_socket] [DEBUG] insecure_mode: True, OCSP response cache file name: None, PROXY_HOST: None, PROXY_PORT: None, PROXY_USER: None PROXY_PASSWORD: None
[2017-04-15 19:25:20,643] [250] [RequestExec-42] [snowflake.connector.ssl_wrap_socket] [INFO] THIS CONNECTION IS IN INSECURE MODE. IT MEANS THE CERTIFICATE WILL BE VALIDATED BUT THE CERTIFICATE REVOCATION STATUS WILL NOT BE CHECKED.
[2017-04-15 19:25:20,655] [250] [ChunkDownloader_40] [botocore.vendored.requests.packages.urllib3.util.retry] [DEBUG] Converted retries value: 5 -> Retry(total=5, connect=None, read=None, redirect=None)
[2017-04-15 19:25:20,655] [250] [ChunkDownloader_40] [botocore.vendored.requests.packages.urllib3.util.retry] [DEBUG] Converted retries value: 5 -> Retry(total=5, connect=None, read=None, redirect=None)
[2017-04-15 19:25:20,657] [250] [ChunkDownloader_40] [snowflake.connector.network] [DEBUG] using requests session [0 uses]: active 33, idle: 0
[2017-04-15 19:25:20,658] [250] [RequestExec-43] [botocore.vendored.requests.packages.urllib3.connectionpool] [INFO] Starting new HTTPS connection (1): sfc-ds2-customer-stage.s3.amazonaws.com
[2017-04-15 19:25:20,659] [250] [ChunkDownloader_40] [snowflake.connector.network] [DEBUG] request thread timeout: 60, rest of request timeout: 60, retry cnt: 1
[2017-04-15 19:25:20,793] [250] [RequestExec-42] [botocore.vendored.requests.packages.urllib3.connectionpool] [DEBUG] "GET /udoi-s-bounty2-/results/b1f97eed-9f52-4f6b-a14e-b5de377ae1bf/main/data_0_0_38?x-amz-server-side-encryption-customer-algorithm=AES256&response-content-encoding=gzip&AWSAccessKeyId=AKIAIVMHKINF3K536SLA&Expires=1492305910&Signature=QMwLYvBVGzY7U6iCkt%2Bwh7ilHEY%3D HTTP/1.1" 200 16805131
[2017-04-15 19:25:20,794] [250] [RequestExec-42] [snowflake.connector.network] [DEBUG] SUCCESS
[2017-04-15 19:25:20,882] [250] [RequestExec-43] [snowflake.connector.ssl_wrap_socket] [DEBUG] insecure_mode: True, OCSP response cache file name: None, PROXY_HOST: None, PROXY_PORT: None, PROXY_USER: None PROXY_PASSWORD: None
[2017-04-15 19:25:20,882] [250] [RequestExec-43] [snowflake.connector.ssl_wrap_socket] [INFO] THIS CONNECTION IS IN INSECURE MODE. IT MEANS THE CERTIFICATE WILL BE VALIDATED BUT THE CERTIFICATE REVOCATION STATUS WILL NOT BE CHECKED.
[2017-04-15 19:25:20,910] [250] [ChunkDownloader_41] [botocore.vendored.requests.packages.urllib3.util.retry] [DEBUG] Converted retries value: 5 -> Retry(total=5, connect=None, read=None, redirect=None)
[2017-04-15 19:25:20,910] [250] [ChunkDownloader_41] [botocore.vendored.requests.packages.urllib3.util.retry] [DEBUG] Converted retries value: 5 -> Retry(total=5, connect=None, read=None, redirect=None)
[2017-04-15 19:25:20,911] [250] [ChunkDownloader_41] [snowflake.connector.network] [DEBUG] using requests session [0 uses]: active 34, idle: 0
[2017-04-15 19:25:20,913] [250] [RequestExec-44] [botocore.vendored.requests.packages.urllib3.connectionpool] [INFO] Starting new HTTPS connection (1): sfc-ds2-customer-stage.s3.amazonaws.com
[2017-04-15 19:25:20,914] [250] [ChunkDownloader_41] [snowflake.connector.network] [DEBUG] request thread timeout: 60, rest of request timeout: 60, retry cnt: 1
[2017-04-15 19:25:21,045] [250] [RequestExec-43] [botocore.vendored.requests.packages.urllib3.connectionpool] [DEBUG] "GET /udoi-s-bounty2-/results/b1f97eed-9f52-4f6b-a14e-b5de377ae1bf/main/data_0_0_39?x-amz-server-side-encryption-customer-algorithm=AES256&response-content-encoding=gzip&AWSAccessKeyId=AKIAIVMHKINF3K536SLA&Expires=1492305910&Signature=1%2FjkJe3Ouw9AWwB50huRYCJiTD0%3D HTTP/1.1" 200 16813828
[2017-04-15 19:25:21,046] [250] [RequestExec-43] [snowflake.connector.network] [DEBUG] SUCCESS
[2017-04-15 19:25:21,130] [250] [RequestExec-44] [snowflake.connector.ssl_wrap_socket] [DEBUG] insecure_mode: True, OCSP response cache file name: None, PROXY_HOST: None, PROXY_PORT: None, PROXY_USER: None PROXY_PASSWORD: None
[2017-04-15 19:25:21,130] [250] [RequestExec-44] [snowflake.connector.ssl_wrap_socket] [INFO] THIS CONNECTION IS IN INSECURE MODE. IT MEANS THE CERTIFICATE WILL BE VALIDATED BUT THE CERTIFICATE REVOCATION STATUS WILL NOT BE CHECKED.
[2017-04-15 19:25:21,161] [250] [ChunkDownloader_42] [botocore.vendored.requests.packages.urllib3.util.retry] [DEBUG] Converted retries value: 5 -> Retry(total=5, connect=None, read=None, redirect=None)
[2017-04-15 19:25:21,161] [250] [ChunkDownloader_42] [botocore.vendored.requests.packages.urllib3.util.retry] [DEBUG] Converted retries value: 5 -> Retry(total=5, connect=None, read=None, redirect=None)
[2017-04-15 19:25:21,162] [250] [ChunkDownloader_42] [snowflake.connector.network] [DEBUG] using requests session [0 uses]: active 35, idle: 0
[2017-04-15 19:25:21,164] [250] [RequestExec-45] [botocore.vendored.requests.packages.urllib3.connectionpool] [INFO] Starting new HTTPS connection (1): sfc-ds2-customer-stage.s3.amazonaws.com
[2017-04-15 19:25:21,166] [250] [ChunkDownloader_42] [snowflake.connector.network] [DEBUG] request thread timeout: 60, rest of request timeout: 60, retry cnt: 1
[2017-04-15 19:25:21,317] [250] [RequestExec-44] [botocore.vendored.requests.packages.urllib3.connectionpool] [DEBUG] "GET /udoi-s-bounty2-/results/b1f97eed-9f52-4f6b-a14e-b5de377ae1bf/main/data_0_0_40?x-amz-server-side-encryption-customer-algorithm=AES256&response-content-encoding=gzip&AWSAccessKeyId=AKIAIVMHKINF3K536SLA&Expires=1492305910&Signature=KJ0yTlnQi0dbf%2Fw5H94OkLTYBss%3D HTTP/1.1" 200 16802340
[2017-04-15 19:25:21,318] [250] [RequestExec-44] [snowflake.connector.network] [DEBUG] SUCCESS
[2017-04-15 19:25:21,383] [250] [RequestExec-45] [snowflake.connector.ssl_wrap_socket] [DEBUG] insecure_mode: True, OCSP response cache file name: None, PROXY_HOST: None, PROXY_PORT: None, PROXY_USER: None PROXY_PASSWORD: None
[2017-04-15 19:25:21,384] [250] [RequestExec-45] [snowflake.connector.ssl_wrap_socket] [INFO] THIS CONNECTION IS IN INSECURE MODE. IT MEANS THE CERTIFICATE WILL BE VALIDATED BUT THE CERTIFICATE REVOCATION STATUS WILL NOT BE CHECKED.
[2017-04-15 19:25:21,412] [250] [ChunkDownloader_43] [botocore.vendored.requests.packages.urllib3.util.retry] [DEBUG] Converted retries value: 5 -> Retry(total=5, connect=None, read=None, redirect=None)
[2017-04-15 19:25:21,412] [250] [ChunkDownloader_43] [botocore.vendored.requests.packages.urllib3.util.retry] [DEBUG] Converted retries value: 5 -> Retry(total=5, connect=None, read=None, redirect=None)
[2017-04-15 19:25:21,413] [250] [ChunkDownloader_43] [snowflake.connector.network] [DEBUG] using requests session [0 uses]: active 36, idle: 0
[2017-04-15 19:25:21,415] [250] [RequestExec-46] [botocore.vendored.requests.packages.urllib3.connectionpool] [INFO] Starting new HTTPS connection (1): sfc-ds2-customer-stage.s3.amazonaws.com
[2017-04-15 19:25:21,416] [250] [ChunkDownloader_43] [snowflake.connector.network] [DEBUG] request thread timeout: 60, rest of request timeout: 60, retry cnt: 1
[2017-04-15 19:25:21,484] [250] [ChunkDownloader_8] [snowflake.connector.network] [DEBUG] request thread joined
[2017-04-15 19:25:21,486] [250] [ChunkDownloader_8] [snowflake.connector.network] [DEBUG] request thread returned object
[2017-04-15 19:25:21,490] [250] [ChunkDownloader_8] [snowflake.connector.network] [DEBUG] freed requests sessions: active 35, idle: 0
[2017-04-15 19:25:21,492] [250] [ChunkDownloader_8] [BENCHMARK] [INFO] Chunk download time: 7691 ms (0 ms latency)
[2017-04-15 19:25:21,493] [250] [ChunkDownloader_44] [snowflake.connector.network] [DEBUG] using requests session [0 uses]: active 36, idle: 0
[2017-04-15 19:25:21,515] [250] [ChunkDownloader_44] [snowflake.connector.network] [DEBUG] request thread timeout: 60, rest of request timeout: 60, retry cnt: 1
[2017-04-15 19:25:21,556] [250] [RequestExec-45] [botocore.vendored.requests.packages.urllib3.connectionpool] [DEBUG] "GET /udoi-s-bounty2-/results/b1f97eed-9f52-4f6b-a14e-b5de377ae1bf/main/data_0_0_41?x-amz-server-side-encryption-customer-algorithm=AES256&response-content-encoding=gzip&AWSAccessKeyId=AKIAIVMHKINF3K536SLA&Expires=1492305910&Signature=22YMVkEDaIqjAHF8qG78Qig0GGc%3D HTTP/1.1" 200 16811670
[2017-04-15 19:25:21,557] [250] [RequestExec-45] [snowflake.connector.network] [DEBUG] SUCCESS
[2017-04-15 19:25:21,668] [250] [RequestExec-46] [snowflake.connector.ssl_wrap_socket] [DEBUG] insecure_mode: True, OCSP response cache file name: None, PROXY_HOST: None, PROXY_PORT: None, PROXY_USER: None PROXY_PASSWORD: None
[2017-04-15 19:25:21,669] [250] [RequestExec-46] [snowflake.connector.ssl_wrap_socket] [INFO] THIS CONNECTION IS IN INSECURE MODE. IT MEANS THE CERTIFICATE WILL BE VALIDATED BUT THE CERTIFICATE REVOCATION STATUS WILL NOT BE CHECKED.
[2017-04-15 19:25:21,707] [250] [RequestExec-47] [botocore.vendored.requests.packages.urllib3.connectionpool] [DEBUG] "GET /udoi-s-bounty2-/results/b1f97eed-9f52-4f6b-a14e-b5de377ae1bf/main/data_0_0_43?x-amz-server-side-encryption-customer-algorithm=AES256&response-content-encoding=gzip&AWSAccessKeyId=AKIAIVMHKINF3K536SLA&Expires=1492305910&Signature=MxUYUm8eL%2Bm60UBvDRqqMTAqZPM%3D HTTP/1.1" 200 16804607
[2017-04-15 19:25:21,708] [250] [RequestExec-47] [snowflake.connector.network] [DEBUG] SUCCESS
[2017-04-15 19:25:21,909] [250] [RequestExec-46] [botocore.vendored.requests.packages.urllib3.connectionpool] [DEBUG] "GET /udoi-s-bounty2-/results/b1f97eed-9f52-4f6b-a14e-b5de377ae1bf/main/data_0_0_42?x-amz-server-side-encryption-customer-algorithm=AES256&response-content-encoding=gzip&AWSAccessKeyId=AKIAIVMHKINF3K536SLA&Expires=1492305910&Signature=FOSRHGuCK%2BoWx%2FrYF2H3FlSmcl8%3D HTTP/1.1" 200 16802534
[2017-04-15 19:25:21,910] [250] [RequestExec-46] [snowflake.connector.network] [DEBUG] SUCCESS
[2017-04-15 19:25:22,428] [250] [ChunkDownloader_15] [snowflake.connector.network] [DEBUG] request thread joined
[2017-04-15 19:25:22,430] [250] [ChunkDownloader_15] [snowflake.connector.network] [DEBUG] request thread returned object
[2017-04-15 19:25:22,431] [250] [ChunkDownloader_15] [snowflake.connector.network] [DEBUG] freed requests sessions: active 35, idle: 0
[2017-04-15 19:25:22,431] [250] [ChunkDownloader_15] [BENCHMARK] [INFO] Chunk download time: 7321 ms (0 ms latency)
[2017-04-15 19:25:23,043] [250] [ChunkDownloader_26] [snowflake.connector.network] [DEBUG] request thread joined
[2017-04-15 19:25:23,047] [250] [ChunkDownloader_26] [snowflake.connector.network] [DEBUG] request thread returned object
[2017-04-15 19:25:23,047] [250] [ChunkDownloader_26] [snowflake.connector.network] [DEBUG] freed requests sessions: active 34, idle: 1
[2017-04-15 19:25:23,051] [250] [ChunkDownloader_26] [BENCHMARK] [INFO] Chunk download time: 5414 ms (0 ms latency)
[2017-04-15 19:25:26,966] [250] [ChunkDownloader_14] [snowflake.connector.network] [DEBUG] request thread joined
[2017-04-15 19:25:26,967] [250] [ChunkDownloader_14] [snowflake.connector.network] [DEBUG] request thread returned object
[2017-04-15 19:25:26,967] [250] [ChunkDownloader_14] [snowflake.connector.network] [DEBUG] freed requests sessions: active 33, idle: 2
[2017-04-15 19:25:26,967] [250] [ChunkDownloader_14] [BENCHMARK] [INFO] Chunk download time: 11917 ms (0 ms latency)
[2017-04-15 19:25:27,870] [250] [ChunkDownloader_31] [snowflake.connector.network] [DEBUG] request thread joined
[2017-04-15 19:25:27,871] [250] [ChunkDownloader_31] [snowflake.connector.network] [DEBUG] request thread returned object
[2017-04-15 19:25:27,874] [250] [ChunkDownloader_31] [snowflake.connector.network] [DEBUG] freed requests sessions: active 32, idle: 3
[2017-04-15 19:25:27,875] [250] [ChunkDownloader_31] [BENCHMARK] [INFO] Chunk download time: 9232 ms (0 ms latency)
[2017-04-15 19:25:29,323] [250] [ChunkDownloader_21] [snowflake.connector.network] [DEBUG] request thread joined
[2017-04-15 19:25:29,329] [250] [ChunkDownloader_21] [snowflake.connector.network] [DEBUG] request thread returned object
[2017-04-15 19:25:29,330] [250] [ChunkDownloader_21] [snowflake.connector.network] [DEBUG] freed requests sessions: active 31, idle: 4
[2017-04-15 19:25:29,331] [250] [ChunkDownloader_21] [BENCHMARK] [INFO] Chunk download time: 12954 ms (0 ms latency)
[2017-04-15 19:25:29,479] [250] [ChunkDownloader_6] [snowflake.connector.network] [DEBUG] request thread joined
[2017-04-15 19:25:29,480] [250] [ChunkDownloader_6] [snowflake.connector.network] [DEBUG] request thread returned object
[2017-04-15 19:25:29,481] [250] [ChunkDownloader_6] [snowflake.connector.network] [DEBUG] freed requests sessions: active 30, idle: 5
[2017-04-15 19:25:29,481] [250] [ChunkDownloader_6] [BENCHMARK] [INFO] Chunk download time: 15934 ms (0 ms latency)
[2017-04-15 19:25:31,219] [250] [ChunkDownloader_36] [snowflake.connector.network] [DEBUG] request thread joined
[2017-04-15 19:25:31,222] [250] [ChunkDownloader_36] [snowflake.connector.network] [DEBUG] request thread returned object
[2017-04-15 19:25:31,227] [250] [ChunkDownloader_36] [snowflake.connector.network] [DEBUG] freed requests sessions: active 29, idle: 6
[2017-04-15 19:25:31,227] [250] [ChunkDownloader_36] [BENCHMARK] [INFO] Chunk download time: 11460 ms (0 ms latency)
[2017-04-15 19:25:32,093] [250] [ChunkDownloader_10] [snowflake.connector.network] [DEBUG] request thread joined
[2017-04-15 19:25:32,094] [250] [ChunkDownloader_10] [snowflake.connector.network] [DEBUG] request thread returned object
[2017-04-15 19:25:32,095] [250] [ChunkDownloader_10] [snowflake.connector.network] [DEBUG] freed requests sessions: active 28, idle: 7
[2017-04-15 19:25:32,095] [250] [ChunkDownloader_10] [BENCHMARK] [INFO] Chunk download time: 18051 ms (0 ms latency)
[2017-04-15 19:25:32,097] [250] [ChunkDownloader_45] [snowflake.connector.network] [DEBUG] using requests session [3 uses]: active 29, idle: 7
[2017-04-15 19:25:32,098] [250] [RequestExec-48] [botocore.vendored.requests.packages.urllib3.connectionpool] [INFO] Resetting dropped connection: sfc-ds2-customer-stage.s3.amazonaws.com
[2017-04-15 19:25:32,143] [250] [ChunkDownloader_45] [snowflake.connector.network] [DEBUG] request thread timeout: 60, rest of request timeout: 60, retry cnt: 1
[2017-04-15 19:25:32,364] [250] [ChunkDownloader_44] [snowflake.connector.network] [DEBUG] request thread joined
[2017-04-15 19:25:32,367] [250] [ChunkDownloader_44] [snowflake.connector.network] [DEBUG] request thread returned object
[2017-04-15 19:25:32,369] [250] [ChunkDownloader_44] [snowflake.connector.network] [DEBUG] freed requests sessions: active 28, idle: 7
[2017-04-15 19:25:32,370] [250] [ChunkDownloader_44] [BENCHMARK] [INFO] Chunk download time: 10876 ms (0 ms latency)
[2017-04-15 19:25:32,446] [250] [RequestExec-48] [snowflake.connector.ssl_wrap_socket] [DEBUG] insecure_mode: True, OCSP response cache file name: None, PROXY_HOST: None, PROXY_PORT: None, PROXY_USER: None PROXY_PASSWORD: None
[2017-04-15 19:25:32,447] [250] [RequestExec-48] [snowflake.connector.ssl_wrap_socket] [INFO] THIS CONNECTION IS IN INSECURE MODE. IT MEANS THE CERTIFICATE WILL BE VALIDATED BUT THE CERTIFICATE REVOCATION STATUS WILL NOT BE CHECKED.
[2017-04-15 19:25:32,589] [250] [RequestExec-48] [botocore.vendored.requests.packages.urllib3.connectionpool] [DEBUG] "GET /udoi-s-bounty2-/results/b1f97eed-9f52-4f6b-a14e-b5de377ae1bf/main/data_0_0_44?x-amz-server-side-encryption-customer-algorithm=AES256&response-content-encoding=gzip&AWSAccessKeyId=AKIAIVMHKINF3K536SLA&Expires=1492305910&Signature=6y3mE0uNunVH%2FvTjqzsaHlMA9Sk%3D HTTP/1.1" 200 16810062
[2017-04-15 19:25:32,589] [250] [RequestExec-48] [snowflake.connector.network] [DEBUG] SUCCESS
[2017-04-15 19:25:33,776] [250] [ChunkDownloader_11] [snowflake.connector.network] [DEBUG] request thread joined
[2017-04-15 19:25:33,780] [250] [ChunkDownloader_11] [snowflake.connector.network] [DEBUG] request thread returned object
[2017-04-15 19:25:33,782] [250] [ChunkDownloader_11] [snowflake.connector.network] [DEBUG] freed requests sessions: active 27, idle: 8
[2017-04-15 19:25:33,783] [250] [ChunkDownloader_11] [BENCHMARK] [INFO] Chunk download time: 19486 ms (0 ms latency)
[2017-04-15 19:25:34,307] [250] [ChunkDownloader_28] [snowflake.connector.network] [DEBUG] request thread joined
[2017-04-15 19:25:34,329] [250] [ChunkDownloader_28] [snowflake.connector.network] [DEBUG] request thread returned object
[2017-04-15 19:25:34,330] [250] [ChunkDownloader_28] [snowflake.connector.network] [DEBUG] freed requests sessions: active 26, idle: 9
[2017-04-15 19:25:34,330] [250] [ChunkDownloader_28] [BENCHMARK] [INFO] Chunk download time: 16190 ms (0 ms latency)
[2017-04-15 19:25:34,642] [250] [ChunkDownloader_35] [snowflake.connector.network] [DEBUG] request thread joined
[2017-04-15 19:25:34,646] [250] [ChunkDownloader_35] [snowflake.connector.network] [DEBUG] request thread returned object
[2017-04-15 19:25:34,646] [250] [ChunkDownloader_35] [snowflake.connector.network] [DEBUG] freed requests sessions: active 25, idle: 10
[2017-04-15 19:25:34,647] [250] [ChunkDownloader_35] [BENCHMARK] [INFO] Chunk download time: 14998 ms (0 ms latency)
[2017-04-15 19:25:34,887] [250] [ChunkDownloader_41] [snowflake.connector.network] [DEBUG] request thread joined
[2017-04-15 19:25:34,892] [250] [ChunkDownloader_41] [snowflake.connector.network] [DEBUG] request thread returned object
[2017-04-15 19:25:34,892] [250] [ChunkDownloader_41] [snowflake.connector.network] [DEBUG] freed requests sessions: active 24, idle: 11
[2017-04-15 19:25:34,893] [250] [ChunkDownloader_41] [BENCHMARK] [INFO] Chunk download time: 13983 ms (0 ms latency)
[2017-04-15 19:25:35,176] [250] [ChunkDownloader_42] [snowflake.connector.network] [DEBUG] request thread joined
[2017-04-15 19:25:35,177] [250] [ChunkDownloader_42] [snowflake.connector.network] [DEBUG] request thread returned object
[2017-04-15 19:25:35,177] [250] [ChunkDownloader_42] [snowflake.connector.network] [DEBUG] freed requests sessions: active 23, idle: 12
[2017-04-15 19:25:35,178] [250] [ChunkDownloader_42] [BENCHMARK] [INFO] Chunk download time: 14017 ms (0 ms latency)
[2017-04-15 19:25:35,303] [250] [ChunkDownloader_17] [snowflake.connector.network] [DEBUG] request thread joined
[2017-04-15 19:25:35,304] [250] [ChunkDownloader_17] [snowflake.connector.network] [DEBUG] request thread returned object
[2017-04-15 19:25:35,308] [250] [ChunkDownloader_17] [snowflake.connector.network] [DEBUG] freed requests sessions: active 22, idle: 13
[2017-04-15 19:25:35,312] [250] [ChunkDownloader_17] [BENCHMARK] [INFO] Chunk download time: 19946 ms (0 ms latency)
[2017-04-15 19:25:35,389] [250] [ChunkDownloader_27] [snowflake.connector.network] [DEBUG] request thread joined
[2017-04-15 19:25:35,391] [250] [ChunkDownloader_27] [snowflake.connector.network] [DEBUG] request thread returned object
[2017-04-15 19:25:35,393] [250] [ChunkDownloader_27] [snowflake.connector.network] [DEBUG] freed requests sessions: active 21, idle: 14
[2017-04-15 19:25:35,395] [250] [ChunkDownloader_27] [BENCHMARK] [INFO] Chunk download time: 17506 ms (0 ms latency)
[2017-04-15 19:25:36,577] [250] [ChunkDownloader_29] [snowflake.connector.network] [DEBUG] request thread joined
[2017-04-15 19:25:36,580] [250] [ChunkDownloader_29] [snowflake.connector.network] [DEBUG] request thread returned object
[2017-04-15 19:25:36,580] [250] [ChunkDownloader_29] [snowflake.connector.network] [DEBUG] freed requests sessions: active 20, idle: 15
[2017-04-15 19:25:36,581] [250] [ChunkDownloader_29] [BENCHMARK] [INFO] Chunk download time: 18349 ms (0 ms latency)
[2017-04-15 19:25:37,257] [250] [ChunkDownloader_19] [snowflake.connector.network] [DEBUG] request thread joined
[2017-04-15 19:25:37,258] [250] [ChunkDownloader_19] [snowflake.connector.network] [DEBUG] request thread returned object
[2017-04-15 19:25:37,260] [250] [ChunkDownloader_19] [snowflake.connector.network] [DEBUG] freed requests sessions: active 19, idle: 16
[2017-04-15 19:25:37,260] [250] [ChunkDownloader_19] [BENCHMARK] [INFO] Chunk download time: 21385 ms (0 ms latency)
[2017-04-15 19:25:38,241] [250] [ChunkDownloader_18] [snowflake.connector.network] [DEBUG] request thread joined
[2017-04-15 19:25:38,241] [250] [ChunkDownloader_18] [snowflake.connector.network] [DEBUG] request thread returned object
[2017-04-15 19:25:38,242] [250] [ChunkDownloader_18] [snowflake.connector.network] [DEBUG] freed requests sessions: active 18, idle: 17
[2017-04-15 19:25:38,243] [250] [ChunkDownloader_18] [BENCHMARK] [INFO] Chunk download time: 22623 ms (0 ms latency)
[2017-04-15 19:25:39,020] [250] [ChunkDownloader_20] [snowflake.connector.network] [DEBUG] request thread joined
[2017-04-15 19:25:39,027] [250] [ChunkDownloader_20] [snowflake.connector.network] [DEBUG] request thread returned object
[2017-04-15 19:25:39,029] [250] [ChunkDownloader_20] [snowflake.connector.network] [DEBUG] closing excess requests sessions: active 17, idle: 18
[2017-04-15 19:25:39,041] [250] [ChunkDownloader_20] [BENCHMARK] [INFO] Chunk download time: 22915 ms (0 ms latency)
[2017-04-15 19:25:39,994] [250] [ChunkDownloader_16] [snowflake.connector.network] [DEBUG] request thread joined
[2017-04-15 19:25:39,996] [250] [ChunkDownloader_16] [snowflake.connector.network] [DEBUG] request thread returned object
[2017-04-15 19:25:39,998] [250] [ChunkDownloader_16] [snowflake.connector.network] [DEBUG] closing excess requests sessions: active 16, idle: 18
[2017-04-15 19:25:40,006] [250] [ChunkDownloader_16] [BENCHMARK] [INFO] Chunk download time: 24895 ms (0 ms latency)
[2017-04-15 19:25:40,066] [250] [ChunkDownloader_13] [snowflake.connector.network] [DEBUG] request thread joined
[2017-04-15 19:25:40,068] [250] [ChunkDownloader_13] [snowflake.connector.network] [DEBUG] request thread returned object
[2017-04-15 19:25:40,071] [250] [ChunkDownloader_13] [snowflake.connector.network] [DEBUG] closing excess requests sessions: active 15, idle: 18
[2017-04-15 19:25:40,073] [250] [ChunkDownloader_13] [BENCHMARK] [INFO] Chunk download time: 25273 ms (0 ms latency)
[2017-04-15 19:25:40,779] [250] [ChunkDownloader_40] [snowflake.connector.network] [DEBUG] request thread joined
[2017-04-15 19:25:40,782] [250] [ChunkDownloader_40] [snowflake.connector.network] [DEBUG] request thread returned object
[2017-04-15 19:25:40,784] [250] [ChunkDownloader_40] [snowflake.connector.network] [DEBUG] closing excess requests sessions: active 14, idle: 18
[2017-04-15 19:25:40,787] [250] [ChunkDownloader_40] [BENCHMARK] [INFO] Chunk download time: 20132 ms (0 ms latency)
[2017-04-15 19:25:40,994] [250] [ChunkDownloader_38] [snowflake.connector.network] [DEBUG] request thread joined
[2017-04-15 19:25:40,997] [250] [ChunkDownloader_38] [snowflake.connector.network] [DEBUG] request thread returned object
[2017-04-15 19:25:41,049] [250] [ChunkDownloader_38] [snowflake.connector.network] [DEBUG] closing excess requests sessions: active 13, idle: 18
[2017-04-15 19:25:41,050] [250] [ChunkDownloader_25] [snowflake.connector.network] [DEBUG] request thread joined
[2017-04-15 19:25:41,052] [250] [ChunkDownloader_25] [snowflake.connector.network] [DEBUG] request thread returned object
[2017-04-15 19:25:41,055] [250] [ChunkDownloader_38] [BENCHMARK] [INFO] Chunk download time: 20903 ms (0 ms latency)
[2017-04-15 19:25:41,055] [250] [ChunkDownloader_25] [snowflake.connector.network] [DEBUG] closing excess requests sessions: active 12, idle: 18
[2017-04-15 19:25:41,060] [250] [ChunkDownloader_25] [BENCHMARK] [INFO] Chunk download time: 23676 ms (0 ms latency)
[2017-04-15 19:25:42,614] [250] [ChunkDownloader_12] [snowflake.connector.network] [DEBUG] request thread joined
[2017-04-15 19:25:42,615] [250] [ChunkDownloader_12] [snowflake.connector.network] [DEBUG] request thread returned object
[2017-04-15 19:25:42,615] [250] [ChunkDownloader_12] [snowflake.connector.network] [DEBUG] closing excess requests sessions: active 11, idle: 18
[2017-04-15 19:25:42,618] [250] [ChunkDownloader_12] [BENCHMARK] [INFO] Chunk download time: 28069 ms (0 ms latency)
[2017-04-15 19:25:42,659] [250] [MainThread] [BENCHMARK] [INFO] Processed rows 100000, 3195 r/s (total 3195 r/s)
[2017-04-15 19:25:42,711] [250] [ChunkDownloader_46] [snowflake.connector.network] [DEBUG] using requests session [0 uses]: active 12, idle: 17
[2017-04-15 19:25:42,755] [250] [ChunkDownloader_47] [snowflake.connector.network] [DEBUG] using requests session [0 uses]: active 13, idle: 16
[2017-04-15 19:25:42,757] [250] [RequestExec-49] [botocore.vendored.requests.packages.urllib3.connectionpool] [INFO] Resetting dropped connection: sfc-ds2-customer-stage.s3.amazonaws.com
[2017-04-15 19:25:42,801] [250] [ChunkDownloader_46] [snowflake.connector.network] [DEBUG] request thread timeout: 60, rest of request timeout: 60, retry cnt: 1
[2017-04-15 19:25:42,846] [250] [RequestExec-50] [botocore.vendored.requests.packages.urllib3.connectionpool] [INFO] Resetting dropped connection: sfc-ds2-customer-stage.s3.amazonaws.com
[2017-04-15 19:25:42,847] [250] [ChunkDownloader_47] [snowflake.connector.network] [DEBUG] request thread timeout: 60, rest of request timeout: 60, retry cnt: 1
[2017-04-15 19:25:42,847] [250] [ChunkDownloader_48] [snowflake.connector.network] [DEBUG] using requests session [0 uses]: active 14, idle: 15
[2017-04-15 19:25:42,876] [250] [MainThread] [BENCHMARK] [INFO] Processed rows 200000, 460983 r/s (total 6346 r/s)
[2017-04-15 19:25:42,904] [250] [RequestExec-51] [botocore.vendored.requests.packages.urllib3.connectionpool] [INFO] Resetting dropped connection: sfc-ds2-customer-stage.s3.amazonaws.com
[2017-04-15 19:25:42,905] [250] [ChunkDownloader_49] [snowflake.connector.network] [DEBUG] using requests session [0 uses]: active 15, idle: 14
[2017-04-15 19:25:42,953] [250] [ChunkDownloader_48] [snowflake.connector.network] [DEBUG] request thread timeout: 60, rest of request timeout: 60, retry cnt: 1
[2017-04-15 19:25:43,012] [250] [ChunkDownloader_50] [snowflake.connector.network] [DEBUG] using requests session [5 uses]: active 16, idle: 13
[2017-04-15 19:25:43,015] [250] [ChunkDownloader_49] [snowflake.connector.network] [DEBUG] request thread timeout: 60, rest of request timeout: 60, retry cnt: 1
[2017-04-15 19:25:43,015] [250] [ChunkDownloader_51] [snowflake.connector.network] [DEBUG] using requests session [2 uses]: active 17, idle: 12
[2017-04-15 19:25:43,017] [250] [RequestExec-53] [botocore.vendored.requests.packages.urllib3.connectionpool] [INFO] Resetting dropped connection: sfc-ds2-customer-stage.s3.amazonaws.com
[2017-04-15 19:25:43,017] [250] [ChunkDownloader_52] [snowflake.connector.network] [DEBUG] using requests session [0 uses]: active 18, idle: 11
[2017-04-15 19:25:43,020] [250] [ChunkDownloader_50] [snowflake.connector.network] [DEBUG] request thread timeout: 60, rest of request timeout: 60, retry cnt: 1
[2017-04-15 19:25:43,021] [250] [RequestExec-54] [botocore.vendored.requests.packages.urllib3.connectionpool] [INFO] Resetting dropped connection: sfc-ds2-customer-stage.s3.amazonaws.com
[2017-04-15 19:25:43,022] [250] [ChunkDownloader_51] [snowflake.connector.network] [DEBUG] request thread timeout: 60, rest of request timeout: 60, retry cnt: 1
[2017-04-15 19:25:43,030] [250] [RequestExec-55] [botocore.vendored.requests.packages.urllib3.connectionpool] [INFO] Resetting dropped connection: sfc-ds2-customer-stage.s3.amazonaws.com
[2017-04-15 19:25:43,031] [250] [ChunkDownloader_52] [snowflake.connector.network] [DEBUG] request thread timeout: 60, rest of request timeout: 60, retry cnt: 1
[2017-04-15 19:25:43,076] [250] [ChunkDownloader_53] [snowflake.connector.network] [DEBUG] using requests session [1 uses]: active 19, idle: 10
[2017-04-15 19:25:43,079] [250] [RequestExec-52] [botocore.vendored.requests.packages.urllib3.connectionpool] [INFO] Resetting dropped connection: sfc-ds2-customer-stage.s3.amazonaws.com
[2017-04-15 19:25:43,122] [250] [MainThread] [BENCHMARK] [INFO] Processed rows 300000, 407023 r/s (total 9446 r/s)
[2017-04-15 19:25:43,138] [250] [RequestExec-56] [botocore.vendored.requests.packages.urllib3.connectionpool] [INFO] Resetting dropped connection: sfc-ds2-customer-stage.s3.amazonaws.com
[2017-04-15 19:25:43,138] [250] [ChunkDownloader_53] [snowflake.connector.network] [DEBUG] request thread timeout: 60, rest of request timeout: 60, retry cnt: 1
[2017-04-15 19:25:43,151] [250] [RequestExec-49] [snowflake.connector.ssl_wrap_socket] [DEBUG] insecure_mode: True, OCSP response cache file name: None, PROXY_HOST: None, PROXY_PORT: None, PROXY_USER: None PROXY_PASSWORD: None
[2017-04-15 19:25:43,152] [250] [RequestExec-49] [snowflake.connector.ssl_wrap_socket] [INFO] THIS CONNECTION IS IN INSECURE MODE. IT MEANS THE CERTIFICATE WILL BE VALIDATED BUT THE CERTIFICATE REVOCATION STATUS WILL NOT BE CHECKED.
[2017-04-15 19:25:43,199] [250] [RequestExec-50] [snowflake.connector.ssl_wrap_socket] [DEBUG] insecure_mode: True, OCSP response cache file name: None, PROXY_HOST: None, PROXY_PORT: None, PROXY_USER: None PROXY_PASSWORD: None
[2017-04-15 19:25:43,199] [250] [RequestExec-50] [snowflake.connector.ssl_wrap_socket] [INFO] THIS CONNECTION IS IN INSECURE MODE. IT MEANS THE CERTIFICATE WILL BE VALIDATED BUT THE CERTIFICATE REVOCATION STATUS WILL NOT BE CHECKED.
[2017-04-15 19:25:43,226] [250] [RequestExec-51] [snowflake.connector.ssl_wrap_socket] [DEBUG] insecure_mode: True, OCSP response cache file name: None, PROXY_HOST: None, PROXY_PORT: None, PROXY_USER: None PROXY_PASSWORD: None
[2017-04-15 19:25:43,226] [250] [RequestExec-51] [snowflake.connector.ssl_wrap_socket] [INFO] THIS CONNECTION IS IN INSECURE MODE. IT MEANS THE CERTIFICATE WILL BE VALIDATED BUT THE CERTIFICATE REVOCATION STATUS WILL NOT BE CHECKED.
[2017-04-15 19:25:43,290] [250] [RequestExec-53] [snowflake.connector.ssl_wrap_socket] [DEBUG] insecure_mode: True, OCSP response cache file name: None, PROXY_HOST: None, PROXY_PORT: None, PROXY_USER: None PROXY_PASSWORD: None
[2017-04-15 19:25:43,291] [250] [RequestExec-53] [snowflake.connector.ssl_wrap_socket] [INFO] THIS CONNECTION IS IN INSECURE MODE. IT MEANS THE CERTIFICATE WILL BE VALIDATED BUT THE CERTIFICATE REVOCATION STATUS WILL NOT BE CHECKED.
[2017-04-15 19:25:43,306] [250] [RequestExec-54] [snowflake.connector.ssl_wrap_socket] [DEBUG] insecure_mode: True, OCSP response cache file name: None, PROXY_HOST: None, PROXY_PORT: None, PROXY_USER: None PROXY_PASSWORD: None
[2017-04-15 19:25:43,307] [250] [RequestExec-54] [snowflake.connector.ssl_wrap_socket] [INFO] THIS CONNECTION IS IN INSECURE MODE. IT MEANS THE CERTIFICATE WILL BE VALIDATED BUT THE CERTIFICATE REVOCATION STATUS WILL NOT BE CHECKED.
[2017-04-15 19:25:43,319] [250] [RequestExec-49] [botocore.vendored.requests.packages.urllib3.connectionpool] [DEBUG] "GET /udoi-s-bounty2-/results/b1f97eed-9f52-4f6b-a14e-b5de377ae1bf/main/data_0_0_45?x-amz-server-side-encryption-customer-algorithm=AES256&response-content-encoding=gzip&AWSAccessKeyId=AKIAIVMHKINF3K536SLA&Expires=1492305910&Signature=NXYpM1jl3LpFKrhMdi7Tj8lyVFk%3D HTTP/1.1" 200 16811589
[2017-04-15 19:25:43,320] [250] [RequestExec-49] [snowflake.connector.network] [DEBUG] SUCCESS
[2017-04-15 19:25:43,352] [250] [RequestExec-55] [snowflake.connector.ssl_wrap_socket] [DEBUG] insecure_mode: True, OCSP response cache file name: None, PROXY_HOST: None, PROXY_PORT: None, PROXY_USER: None PROXY_PASSWORD: None
[2017-04-15 19:25:43,353] [250] [RequestExec-55] [snowflake.connector.ssl_wrap_socket] [INFO] THIS CONNECTION IS IN INSECURE MODE. IT MEANS THE CERTIFICATE WILL BE VALIDATED BUT THE CERTIFICATE REVOCATION STATUS WILL NOT BE CHECKED.
[2017-04-15 19:25:43,355] [250] [RequestExec-52] [snowflake.connector.ssl_wrap_socket] [DEBUG] insecure_mode: True, OCSP response cache file name: None, PROXY_HOST: None, PROXY_PORT: None, PROXY_USER: None PROXY_PASSWORD: None
[2017-04-15 19:25:43,356] [250] [RequestExec-52] [snowflake.connector.ssl_wrap_socket] [INFO] THIS CONNECTION IS IN INSECURE MODE. IT MEANS THE CERTIFICATE WILL BE VALIDATED BUT THE CERTIFICATE REVOCATION STATUS WILL NOT BE CHECKED.
[2017-04-15 19:25:43,365] [250] [RequestExec-56] [snowflake.connector.ssl_wrap_socket] [DEBUG] insecure_mode: True, OCSP response cache file name: None, PROXY_HOST: None, PROXY_PORT: None, PROXY_USER: None PROXY_PASSWORD: None
[2017-04-15 19:25:43,365] [250] [RequestExec-56] [snowflake.connector.ssl_wrap_socket] [INFO] THIS CONNECTION IS IN INSECURE MODE. IT MEANS THE CERTIFICATE WILL BE VALIDATED BUT THE CERTIFICATE REVOCATION STATUS WILL NOT BE CHECKED.
[2017-04-15 19:25:43,388] [250] [RequestExec-50] [botocore.vendored.requests.packages.urllib3.connectionpool] [DEBUG] "GET /udoi-s-bounty2-/results/b1f97eed-9f52-4f6b-a14e-b5de377ae1bf/main/data_0_0_46?x-amz-server-side-encryption-customer-algorithm=AES256&response-content-encoding=gzip&AWSAccessKeyId=AKIAIVMHKINF3K536SLA&Expires=1492305910&Signature=eCmg7h6bEdTomLwzRJRE9opBHdQ%3D HTTP/1.1" 200 16813640
[2017-04-15 19:25:43,389] [250] [RequestExec-50] [snowflake.connector.network] [DEBUG] SUCCESS
[2017-04-15 19:25:43,498] [250] [ChunkDownloader_33] [snowflake.connector.network] [DEBUG] request thread joined
[2017-04-15 19:25:43,499] [250] [RequestExec-51] [botocore.vendored.requests.packages.urllib3.connectionpool] [DEBUG] "GET /udoi-s-bounty2-/results/b1f97eed-9f52-4f6b-a14e-b5de377ae1bf/main/data_0_0_47?x-amz-server-side-encryption-customer-algorithm=AES256&response-content-encoding=gzip&AWSAccessKeyId=AKIAIVMHKINF3K536SLA&Expires=1492305910&Signature=SfaRpg%2FO%2B0F4mGelXbWq5umEAk4%3D HTTP/1.1" 200 16807127
[2017-04-15 19:25:43,500] [250] [RequestExec-54] [botocore.vendored.requests.packages.urllib3.connectionpool] [DEBUG] "GET /udoi-s-bounty2-/results/b1f97eed-9f52-4f6b-a14e-b5de377ae1bf/main/data_0_0_50?x-amz-server-side-encryption-customer-algorithm=AES256&response-content-encoding=gzip&AWSAccessKeyId=AKIAIVMHKINF3K536SLA&Expires=1492305910&Signature=EBPDi4FcSmnBjxa9VbC3vRO0%2FqI%3D HTTP/1.1" 200 16805092
[2017-04-15 19:25:43,501] [250] [RequestExec-53] [botocore.vendored.requests.packages.urllib3.connectionpool] [DEBUG] "GET /udoi-s-bounty2-/results/b1f97eed-9f52-4f6b-a14e-b5de377ae1bf/main/data_0_0_49?x-amz-server-side-encryption-customer-algorithm=AES256&response-content-encoding=gzip&AWSAccessKeyId=AKIAIVMHKINF3K536SLA&Expires=1492305910&Signature=30EhMtco6HccXGvjNocl5KZbfIc%3D HTTP/1.1" 200 16813546
[2017-04-15 19:25:43,502] [250] [ChunkDownloader_33] [snowflake.connector.network] [DEBUG] request thread returned object
[2017-04-15 19:25:43,505] [250] [ChunkDownloader_33] [snowflake.connector.network] [DEBUG] freed requests sessions: active 18, idle: 10
[2017-04-15 19:25:43,503] [250] [RequestExec-54] [snowflake.connector.network] [DEBUG] SUCCESS
[2017-04-15 19:25:43,505] [250] [RequestExec-53] [snowflake.connector.network] [DEBUG] SUCCESS
[2017-04-15 19:25:43,503] [250] [RequestExec-51] [snowflake.connector.network] [DEBUG] SUCCESS
[2017-04-15 19:25:43,506] [250] [ChunkDownloader_33] [BENCHMARK] [INFO] Chunk download time: 24359 ms (0 ms latency)
[2017-04-15 19:25:43,509] [250] [RequestExec-52] [botocore.vendored.requests.packages.urllib3.connectionpool] [DEBUG] "GET /udoi-s-bounty2-/results/b1f97eed-9f52-4f6b-a14e-b5de377ae1bf/main/data_0_0_48?x-amz-server-side-encryption-customer-algorithm=AES256&response-content-encoding=gzip&AWSAccessKeyId=AKIAIVMHKINF3K536SLA&Expires=1492305910&Signature=0izMeTSF6WJBLHA2HO4bS9Jzih0%3D HTTP/1.1" 200 16802314
[2017-04-15 19:25:43,509] [250] [RequestExec-52] [snowflake.connector.network] [DEBUG] SUCCESS
[2017-04-15 19:25:43,513] [250] [RequestExec-56] [botocore.vendored.requests.packages.urllib3.connectionpool] [DEBUG] "GET /udoi-s-bounty2-/results/b1f97eed-9f52-4f6b-a14e-b5de377ae1bf/main/data_0_0_52?x-amz-server-side-encryption-customer-algorithm=AES256&response-content-encoding=gzip&AWSAccessKeyId=AKIAIVMHKINF3K536SLA&Expires=1492305910&Signature=%2FUx%2FE3T49E2lrZD1PeNFqxj%2BjMc%3D HTTP/1.1" 200 16805895
[2017-04-15 19:25:43,514] [250] [RequestExec-56] [snowflake.connector.network] [DEBUG] SUCCESS
[2017-04-15 19:25:43,538] [250] [RequestExec-55] [botocore.vendored.requests.packages.urllib3.connectionpool] [DEBUG] "GET /udoi-s-bounty2-/results/b1f97eed-9f52-4f6b-a14e-b5de377ae1bf/main/data_0_0_51?x-amz-server-side-encryption-customer-algorithm=AES256&response-content-encoding=gzip&AWSAccessKeyId=AKIAIVMHKINF3K536SLA&Expires=1492305910&Signature=39SqYqVaO38%2BTCm1aSGnt9nIVUE%3D HTTP/1.1" 200 16812341
[2017-04-15 19:25:43,539] [250] [RequestExec-55] [snowflake.connector.network] [DEBUG] SUCCESS
[2017-04-15 19:25:45,035] [250] [ChunkDownloader_22] [snowflake.connector.network] [DEBUG] request thread joined
[2017-04-15 19:25:45,043] [250] [ChunkDownloader_22] [snowflake.connector.network] [DEBUG] request thread returned object
[2017-04-15 19:25:45,044] [250] [ChunkDownloader_22] [snowflake.connector.network] [DEBUG] freed requests sessions: active 17, idle: 11
[2017-04-15 19:25:45,049] [250] [ChunkDownloader_22] [BENCHMARK] [INFO] Chunk download time: 28421 ms (0 ms latency)
[2017-04-15 19:25:45,051] [250] [ChunkDownloader_54] [snowflake.connector.network] [DEBUG] using requests session [0 uses]: active 18, idle: 11
[2017-04-15 19:25:45,097] [250] [ChunkDownloader_54] [snowflake.connector.network] [DEBUG] request thread timeout: 60, rest of request timeout: 60, retry cnt: 1
[2017-04-15 19:25:45,098] [250] [RequestExec-57] [botocore.vendored.requests.packages.urllib3.connectionpool] [INFO] Resetting dropped connection: sfc-ds2-customer-stage.s3.amazonaws.com
[2017-04-15 19:25:45,318] [250] [RequestExec-57] [snowflake.connector.ssl_wrap_socket] [DEBUG] insecure_mode: True, OCSP response cache file name: None, PROXY_HOST: None, PROXY_PORT: None, PROXY_USER: None PROXY_PASSWORD: None
[2017-04-15 19:25:45,319] [250] [RequestExec-57] [snowflake.connector.ssl_wrap_socket] [INFO] THIS CONNECTION IS IN INSECURE MODE. IT MEANS THE CERTIFICATE WILL BE VALIDATED BUT THE CERTIFICATE REVOCATION STATUS WILL NOT BE CHECKED.
[2017-04-15 19:25:45,393] [250] [ChunkDownloader_23] [snowflake.connector.network] [DEBUG] request thread joined
[2017-04-15 19:25:45,396] [250] [ChunkDownloader_23] [snowflake.connector.network] [DEBUG] request thread returned object
[2017-04-15 19:25:45,396] [250] [ChunkDownloader_23] [snowflake.connector.network] [DEBUG] freed requests sessions: active 17, idle: 11
[2017-04-15 19:25:45,396] [250] [ChunkDownloader_23] [BENCHMARK] [INFO] Chunk download time: 28516 ms (0 ms latency)
[2017-04-15 19:25:45,399] [250] [ChunkDownloader_55] [snowflake.connector.network] [DEBUG] using requests session [0 uses]: active 18, idle: 11
[2017-04-15 19:25:45,446] [250] [ChunkDownloader_55] [snowflake.connector.network] [DEBUG] request thread timeout: 60, rest of request timeout: 60, retry cnt: 1
[2017-04-15 19:25:45,452] [250] [RequestExec-58] [botocore.vendored.requests.packages.urllib3.connectionpool] [INFO] Resetting dropped connection: sfc-ds2-customer-stage.s3.amazonaws.com
[2017-04-15 19:25:45,467] [250] [RequestExec-57] [botocore.vendored.requests.packages.urllib3.connectionpool] [DEBUG] "GET /udoi-s-bounty2-/results/b1f97eed-9f52-4f6b-a14e-b5de377ae1bf/main/data_0_0_53?x-amz-server-side-encryption-customer-algorithm=AES256&response-content-encoding=gzip&AWSAccessKeyId=AKIAIVMHKINF3K536SLA&Expires=1492305910&Signature=HBHmHZaYvlXGfUtewRVn3JXvpcE%3D HTTP/1.1" 200 16812795
[2017-04-15 19:25:45,468] [250] [RequestExec-57] [snowflake.connector.network] [DEBUG] SUCCESS
[2017-04-15 19:25:45,672] [250] [RequestExec-58] [snowflake.connector.ssl_wrap_socket] [DEBUG] insecure_mode: True, OCSP response cache file name: None, PROXY_HOST: None, PROXY_PORT: None, PROXY_USER: None PROXY_PASSWORD: None
[2017-04-15 19:25:45,673] [250] [RequestExec-58] [snowflake.connector.ssl_wrap_socket] [INFO] THIS CONNECTION IS IN INSECURE MODE. IT MEANS THE CERTIFICATE WILL BE VALIDATED BUT THE CERTIFICATE REVOCATION STATUS WILL NOT BE CHECKED.
[2017-04-15 19:25:45,757] [250] [ChunkDownloader_45] [snowflake.connector.network] [DEBUG] request thread joined
[2017-04-15 19:25:45,757] [250] [ChunkDownloader_45] [snowflake.connector.network] [DEBUG] request thread returned object
[2017-04-15 19:25:45,758] [250] [ChunkDownloader_45] [snowflake.connector.network] [DEBUG] freed requests sessions: active 17, idle: 11
[2017-04-15 19:25:45,758] [250] [ChunkDownloader_45] [BENCHMARK] [INFO] Chunk download time: 13661 ms (0 ms latency)
[2017-04-15 19:25:45,829] [250] [RequestExec-58] [botocore.vendored.requests.packages.urllib3.connectionpool] [DEBUG] "GET /udoi-s-bounty2-/results/b1f97eed-9f52-4f6b-a14e-b5de377ae1bf/main/data_0_0_54?x-amz-server-side-encryption-customer-algorithm=AES256&response-content-encoding=gzip&AWSAccessKeyId=AKIAIVMHKINF3K536SLA&Expires=1492305910&Signature=4HBjA39Ydc9AXSZjuhZ0P%2BMBoZQ%3D HTTP/1.1" 200 16811613
[2017-04-15 19:25:45,830] [250] [RequestExec-58] [snowflake.connector.network] [DEBUG] SUCCESS
[2017-04-15 19:25:48,561] [250] [ChunkDownloader_43] [snowflake.connector.network] [DEBUG] request thread joined
[2017-04-15 19:25:48,561] [250] [ChunkDownloader_43] [snowflake.connector.network] [DEBUG] request thread returned object
[2017-04-15 19:25:48,562] [250] [ChunkDownloader_43] [snowflake.connector.network] [DEBUG] freed requests sessions: active 16, idle: 12
[2017-04-15 19:25:48,563] [250] [ChunkDownloader_43] [BENCHMARK] [INFO] Chunk download time: 27151 ms (0 ms latency)
[2017-04-15 19:25:48,947] [250] [ChunkDownloader_34] [snowflake.connector.network] [DEBUG] request thread joined
[2017-04-15 19:25:48,947] [250] [ChunkDownloader_34] [snowflake.connector.network] [DEBUG] request thread returned object
[2017-04-15 19:25:48,949] [250] [ChunkDownloader_34] [snowflake.connector.network] [DEBUG] freed requests sessions: active 15, idle: 13
[2017-04-15 19:25:48,949] [250] [ChunkDownloader_34] [BENCHMARK] [INFO] Chunk download time: 29551 ms (0 ms latency)
[2017-04-15 19:25:50,136] [250] [ChunkDownloader_50] [snowflake.connector.network] [DEBUG] request thread joined
[2017-04-15 19:25:50,137] [250] [ChunkDownloader_50] [snowflake.connector.network] [DEBUG] request thread returned object
[2017-04-15 19:25:50,138] [250] [ChunkDownloader_50] [snowflake.connector.network] [DEBUG] freed requests sessions: active 14, idle: 14
[2017-04-15 19:25:50,139] [250] [ChunkDownloader_50] [BENCHMARK] [INFO] Chunk download time: 7235 ms (0 ms latency)
[2017-04-15 19:25:50,638] [250] [ChunkDownloader_53] [snowflake.connector.network] [DEBUG] request thread joined
[2017-04-15 19:25:50,640] [250] [ChunkDownloader_53] [snowflake.connector.network] [DEBUG] request thread returned object
[2017-04-15 19:25:50,640] [250] [ChunkDownloader_53] [snowflake.connector.network] [DEBUG] closing excess requests sessions: active 13, idle: 15
[2017-04-15 19:25:50,647] [250] [ChunkDownloader_53] [BENCHMARK] [INFO] Chunk download time: 7570 ms (0 ms latency)
[2017-04-15 19:25:51,635] [250] [ChunkDownloader_47] [snowflake.connector.network] [DEBUG] request thread joined
[2017-04-15 19:25:51,635] [250] [ChunkDownloader_47] [snowflake.connector.network] [DEBUG] request thread returned object
[2017-04-15 19:25:51,635] [250] [ChunkDownloader_47] [snowflake.connector.network] [DEBUG] closing excess requests sessions: active 12, idle: 15
[2017-04-15 19:25:51,641] [250] [ChunkDownloader_47] [BENCHMARK] [INFO] Chunk download time: 8886 ms (0 ms latency)
[2017-04-15 19:25:53,017] [250] [ChunkDownloader_49] [snowflake.connector.network] [DEBUG] request thread joined
[2017-04-15 19:25:53,018] [250] [ChunkDownloader_49] [snowflake.connector.network] [DEBUG] request thread returned object
[2017-04-15 19:25:53,019] [250] [ChunkDownloader_49] [snowflake.connector.network] [DEBUG] closing excess requests sessions: active 11, idle: 15
[2017-04-15 19:25:53,021] [250] [ChunkDownloader_49] [BENCHMARK] [INFO] Chunk download time: 10169 ms (0 ms latency)
[2017-04-15 19:25:53,154] [250] [ChunkDownloader_37] [snowflake.connector.network] [DEBUG] request thread joined
[2017-04-15 19:25:53,155] [250] [ChunkDownloader_37] [snowflake.connector.network] [DEBUG] request thread returned object
[2017-04-15 19:25:53,155] [250] [ChunkDownloader_37] [snowflake.connector.network] [DEBUG] closing excess requests sessions: active 10, idle: 15
[2017-04-15 19:25:53,157] [250] [ChunkDownloader_37] [BENCHMARK] [INFO] Chunk download time: 33257 ms (0 ms latency)
[2017-04-15 19:25:55,731] [250] [ChunkDownloader_46] [snowflake.connector.network] [DEBUG] request thread joined
[2017-04-15 19:25:55,731] [250] [ChunkDownloader_46] [snowflake.connector.network] [DEBUG] request thread returned object
[2017-04-15 19:25:55,731] [250] [ChunkDownloader_46] [snowflake.connector.network] [DEBUG] closing excess requests sessions: active 9, idle: 15
[2017-04-15 19:25:55,733] [250] [ChunkDownloader_46] [BENCHMARK] [INFO] Chunk download time: 13021 ms (0 ms latency)
[2017-04-15 19:25:57,291] [250] [ChunkDownloader_51] [snowflake.connector.network] [DEBUG] request thread joined
[2017-04-15 19:25:57,293] [250] [ChunkDownloader_51] [snowflake.connector.network] [DEBUG] request thread returned object
[2017-04-15 19:25:57,294] [250] [ChunkDownloader_51] [snowflake.connector.network] [DEBUG] closing excess requests sessions: active 8, idle: 15
[2017-04-15 19:25:57,296] [250] [ChunkDownloader_51] [BENCHMARK] [INFO] Chunk download time: 14339 ms (0 ms latency)
[2017-04-15 19:25:57,341] [250] [ChunkDownloader_54] [snowflake.connector.network] [DEBUG] request thread joined
[2017-04-15 19:25:57,342] [250] [ChunkDownloader_54] [snowflake.connector.network] [DEBUG] request thread returned object
[2017-04-15 19:25:57,342] [250] [ChunkDownloader_54] [snowflake.connector.network] [DEBUG] closing excess requests sessions: active 7, idle: 15
[2017-04-15 19:25:57,344] [250] [ChunkDownloader_54] [BENCHMARK] [INFO] Chunk download time: 12292 ms (0 ms latency)
[2017-04-15 19:25:58,189] [250] [ChunkDownloader_52] [snowflake.connector.network] [DEBUG] request thread joined
[2017-04-15 19:25:58,190] [250] [ChunkDownloader_52] [snowflake.connector.network] [DEBUG] request thread returned object
[2017-04-15 19:25:58,190] [250] [ChunkDownloader_52] [snowflake.connector.network] [DEBUG] closing excess requests sessions: active 6, idle: 15
[2017-04-15 19:25:58,355] [250] [ChunkDownloader_48] [snowflake.connector.network] [DEBUG] request thread joined
[2017-04-15 19:25:58,357] [250] [ChunkDownloader_48] [snowflake.connector.network] [DEBUG] request thread returned object
[2017-04-15 19:25:58,357] [250] [ChunkDownloader_48] [snowflake.connector.network] [DEBUG] closing excess requests sessions: active 5, idle: 15
[2017-04-15 19:25:58,357] [250] [ChunkDownloader_52] [BENCHMARK] [INFO] Chunk download time: 15344 ms (0 ms latency)
[2017-04-15 19:25:58,360] [250] [ChunkDownloader_48] [BENCHMARK] [INFO] Chunk download time: 15559 ms (0 ms latency)
[2017-04-15 19:26:17,135] [250] [ChunkDownloader_24] [snowflake.connector.network] [DEBUG] request thread joined
[2017-04-15 19:26:18,395] [250] [ChunkDownloader_30] [snowflake.connector.network] [DEBUG] request thread joined
[2017-04-15 19:26:18,901] [250] [ChunkDownloader_32] [snowflake.connector.network] [DEBUG] request thread joined
[2017-04-15 19:26:20,409] [250] [ChunkDownloader_39] [snowflake.connector.network] [DEBUG] request thread joined
[2017-04-15 19:26:32,135] [250] [ChunkDownloader_24] [snowflake.connector.network] [ERROR] Failed to get the response. Hanging? method: get, url: https://sfc-ds2-customer-stage.s3.amazonaws.com/udoi-s-bounty2-/results/b1f97eed-9f52-4f6b-a14e-b5de377ae1bf/main/data_0_0_23?x-amz-server-side-encryption-customer-algorithm=AES256&response-content-encoding=gzip&AWSAccessKeyId=AKIAIVMHKINF3K536SLA&Expires=1492305910&Signature=pWtKL7kgriRwMpOGe5uAzy7XcgI%3D, headers:{'x-amz-server-side-encryption-customer-key': 'lZSvKoOkdXa7JLAqt7fxbCxQMjJxWpV1nEqORajHM6k=', 'x-amz-server-side-encryption-customer-key-md5': '9CT9WM8c4lok375GoQKsjQ=='}, data: None, proxies: None
[2017-04-15 19:26:32,136] [250] [ChunkDownloader_24] [snowflake.connector.network] [DEBUG] closing excess requests sessions: active 4, idle: 15
[2017-04-15 19:26:32,274] [250] [MainThread] [root] [CRITICAL] FINISHED: FETCHONE() ITER
FETCHONE() ITER...
TOOK 80.91179311299993
USER 14.149999999999999
SYS 2.539999999999999
IOWAIT 0.02999999999999936
IDLE 141.62000000000035
Command 'main' error, traceback...
Traceback (most recent call last)
17. File "/usr/local/lib/python3.6/site-packages/shellish/session.py", line 115, in execute
raise e
16. File "/usr/local/lib/python3.6/site-packages/shellish/session.py", line 112, in execute
result = command.run_wrap(args)
15. File "/usr/local/lib/python3.6/site-packages/shellish/command/command.py", line 168, in run_wrap
raise e
14. File "/usr/local/lib/python3.6/site-packages/shellish/command/command.py", line 164, in run_wrap
result = self.run(args)
13. File "/usr/local/lib/python3.6/site-packages/shellish/command/autocommand.py", line 52, in run
return self.func(*positionals, **keywords)
12. File "/bench/bench.py", line 171, in main
while q.fetchone() is not None:
11. File "/usr/local/lib/python3.6/site-packages/snowflake_connector_python-1.3.15-py3.6.egg/snowflake/connector/cursor.py", line 654, in fetchone
return self._row_to_python(next(self._row_iter))
10. File "/usr/local/lib/python3.6/site-packages/snowflake_connector_python-1.3.15-py3.6.egg/snowflake/connector/cursor.py", line 664, in _chunk_row_gen
for chunk in downloader:
9. File "/usr/local/lib/python3.6/site-packages/snowflake_connector_python-1.3.15-py3.6.egg/snowflake/connector/chunk_downloader.py", line 91, in __next__
rows = fut.result(timeout=0.250)
8. File "/usr/local/lib/python3.6/concurrent/futures/_base.py", line 405, in result
return self.__get_result()
7. File "/usr/local/lib/python3.6/concurrent/futures/_base.py", line 357, in __get_result
raise self._exception
6. File "/usr/local/lib/python3.6/site-packages/snowflake_connector_python-1.3.15-py3.6.egg/snowflake/connector/chunk_downloader.py", line 174, in _fetch_chunk_worker_runner
rows = self._fetch_chunk_worker(chunk)
5. File "/bench/bench.py", line 73, in _fetch_chunk_worker
is_raw_binary_iterator=False, use_ijson=self._use_ijson)
4. File "/usr/local/lib/python3.6/site-packages/snowflake_connector_python-1.3.15-py3.6.egg/snowflake/connector/network.py", line 665, in fetch
proxies, timeouts, **kwargs)
3. File "/usr/local/lib/python3.6/site-packages/snowflake_connector_python-1.3.15-py3.6.egg/snowflake/connector/network.py", line 895, in _fetch
u'errno': ER_FAILED_TO_REQUEST,
2. File "/usr/local/lib/python3.6/site-packages/snowflake_connector_python-1.3.15-py3.6.egg/snowflake/connector/errors.py", line 99, in errorhandler_wrapper
connection.errorhandler(connection, cursor, errorclass, errorvalue)
1. File "/usr/local/lib/python3.6/site-packages/snowflake_connector_python-1.3.15-py3.6.egg/snowflake/connector/errors.py", line 72, in default_errorhandler
done_format_msg=errorvalue.get(u'done_format_msg'))
OperationalError: 250003: None: Failed to get the response. Hanging? method: get, url: https://sfc-ds2-customer-stage.s3.amazonaws.com/udoi-s-bounty2-/results/b1f97eed-9f52-4f6b-a14e-b5de377ae1bf/main/data_0_0_23?x-amz-server-side-encryption-customer-algorithm=AES256&response-content-encoding=gzip&AWSAccessKeyId=AKIAIVMHKINF3K536SLA&Expires=1492305910&Signature=pWtKL7kgriRwMpOGe5uAzy7XcgI%3D, proxies: None
1.4.4 caused the regression in 3.6.
@JDFagan, Thanks for reporting an issue.
It seems that there is no easy way to set the download concurrency settings for the connector.
This results in very slow downloads for large result sets (as the default values are quite small).
From what I understand the smaller between DEFAULT_CLIENT_RESULT_PREFETCH_SLOTS
and DEFAULT_CLIENT_RESULT_PREFETCH_THREADS
controls the concurrency.
At runtime those parameters are also received from the query response, but there doesn't seem to be a way to set them. Executing ALTER SESSION SET CLIENT_PREFETCH_THREADS = 10
has no effect, and ALTER SESSION SET CLIENT_RESULT_PREFETCH_SLOTS = 10
results in invalid parameter 'CLIENT_RESULT_PREFETCH_SLOTS'
error.
What this causes is a very slow download, because if some chunk is slow to download or the connection is dropped, it delays the download of all other chunks.
Monkey patching the relevant parts of the code to increase those values at runtime results in much faster download speeds.
The questions are:
I have MFA enabled on my account. Every time I run a Python script that called the Snowflake module, it triggers a Duo Mobile push notification to approve. This page seems to detail ways to connect using MFA, but doesn't mention Python. My assumption is that once authorized, I should be able to run my script multiple times within a given period without needing to approve every run.
Is there a way around this?
Example:
This works:
import boto3
client = boto3.Session(profile_name = 'aws_profile').client('s3')
client.list_objects(Bucket = 'bucky-mcbucketface')
This doesn't
import boto3
import snowflake.connector
client = boto3.Session(profile_name = 'aws_profile').client('s3')
client.list_objects(Bucket = 'bucky-mcbucketface')
There is no error or warning message, it just hangs and never completes.
Simple access convention error when handling cursor errors..
Traceback (most recent call last)
2. File "/usr/lib/python3.5/site-packages/snowflake_connector_python-1.3.14-py3.5.egg/snowflake/connector/cursor.py", line 856, in _row_to_python
row[idx] = conv_method(col, conv_ctx)
1. File "/usr/lib/python3.5/site-packages/snowflake_connector_python-1.3.14-py3.5.egg/snowflake/connector/converter.py", line 295, in _TIMESTAMP_NTZ_to_python
t = ZERO_EPOCH + timedelta(seconds=(microseconds / float(1000000)))
OverflowError: date value out of range
During handling of the above exception, another exception occurred:
Traceback (most recent call last)
8. File "/home/mayfield/project/shellish/shellish/session.py", line 119, in execute
raise e
7. File "/home/mayfield/project/shellish/shellish/session.py", line 116, in execute
result = command.run_wrap(args)
6. File "/home/mayfield/project/shellish/shellish/command/command.py", line 169, in run_wrap
raise e
5. File "/home/mayfield/project/shellish/shellish/command/command.py", line 165, in run_wrap
result = self.run(args)
4. File "/home/mayfield/project/shellish/shellish/command/autocommand.py", line 52, in run
return self.func(*positionals, **keywords)
3. File "bench.py", line 54, in main
while q.fetchone() is not None:
2. File "/usr/lib/python3.5/site-packages/snowflake_connector_python-1.3.14-py3.5.egg/snowflake/connector/cursor.py", line 689, in fetchone
return self._row_to_python(row) if row is not None else None
1. File "/usr/lib/python3.5/site-packages/snowflake_connector_python-1.3.14-py3.5.egg/snowflake/connector/cursor.py", line 863, in _row_to_python
type=FIELD_ID_TO_NAME(col_desc[1]),
TypeError: 'collections.defaultdict' object is not callable
I am trying to use the snowflake-connector in a flask application with flask-socketio and eventlet. I run into an issue when eventlet is monkey patched.
import snowflake.connector as scon
from flask import Flask
from flask_socketio import SocketIO
import eventlet
eventlet.monkey_patch()
def query():
# Gets the version
ctx = scon.connect(
user='xxx',
password='xxxxxxxx',
account='xxxxxx',
)
cs = ctx.cursor()
try:
cs.execute("SELECT current_version()")
one = cs.fetchone()
return 'Snowflake version={}'.format(one[0])
finally:
cs.close()
app = Flask(__name__)
socketio = SocketIO(app)
@app.route('/snowflake')
def query_route():
return query()
@app.route('/')
def default_route():
return 'Success'
socketio.run(app,debug=False, host='localhost', port=5000)
Error Received:
Traceback (most recent call last):
File "/home/chad/miniconda2/lib/python2.7/threading.py", line 801, in __bootstrap_inner
self.run()
File "/home/chad/miniconda2/lib/python2.7/threading.py", line 754, in run
self.__target(*self.__args, **self.__kwargs)
File "/home/chad/miniconda2/lib/python2.7/multiprocessing/pool.py", line 327, in _handle_workers
while thread._state == RUN or (pool._cache and thread._state != TERMINATE):
AttributeError: '_MainThread' object has no attribute '_state'
Exception in thread Thread-6:
Traceback (most recent call last):
File "/home/chad/miniconda2/lib/python2.7/threading.py", line 801, in __bootstrap_inner
self.run()
File "/home/chad/miniconda2/lib/python2.7/threading.py", line 754, in run
self.__target(*self.__args, **self.__kwargs)
File "/home/chad/miniconda2/lib/python2.7/multiprocessing/pool.py", line 363, in _handle_tasks
cache[job]._set(ind + 1, (False, ex))
TypeError: unsupported operand type(s) for +: 'NoneType' and 'int'
Environment details:
OS: Ubuntu 16.04.3 LTS
Python Version: Python 2.7.14 :: Anaconda, Inc.
From https://www.python.org/dev/peps/pep-0249/#commit:
Note that if the database supports an auto-commit feature, this must be initially off. An interface method may be provided to turn it back on.
$ pip freeze
asn1crypto==0.24.0
azure-common==1.1.12
azure-nspkg==2.0.0
azure-storage==0.36.0
boto3==1.7.48
botocore==1.10.63
certifi==2018.4.16
cffi==1.11.5
chardet==3.0.4
cryptography==2.2.2
docutils==0.14
future==0.16.0
idna==2.7
ijson==2.3
jmespath==0.9.3
numpy==1.14.5
pandas==0.23.1
pip-autoremove==0.9.1
psutil==5.4.6
pyasn1==0.4.3
pyasn1-modules==0.2.2
pycparser==2.18
pycryptodome==3.6.3
PyJWT==1.6.4
pyOpenSSL==17.5.0
python-dateutil==2.7.3
pytz==2018.5
PyYAML==3.12
requests==2.19.1
s3transfer==0.1.13
six==1.11.0
snowflake-connector-python==1.6.5
teradata==15.10.0.10
urllib3==1.23
virtualenv==16.0.0
[DEBUG:snowflake.connector.connection:678]--:--sql=[USE DATABASE AIDM_EDW_DEV], sequence_id=[2], is_file_transfer=[None]
[DEBUG:snowflake.connector.network:649]--:--socket timeout: 60
[DEBUG:snowflake.connector.network:790]--:--Active requests sessions: 1, idle: 0
[DEBUG:snowflake.connector.network:530]--:--remaining request timeout: None, retry cnt: 1
[DEBUG:snowflake.connector.network:649]--:--socket timeout: 60
[DEBUG:snowflake.connector.network:802]--:--Active requests sessions: 0, idle: 1
[DEBUG:snowflake.connector.network:802]--:--Active requests sessions: 0, idle: 1
[INFO:snowflake.connector.telemetry:106]--:--Failed to upload metrics to telemetry.
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/botocore/vendored/requests/packages/urllib3/connectionpool.py", line 372, in _make_request
httplib_response = conn.getresponse(buffering=True)
TypeError: getresponse() got an unexpected keyword argument 'buffering'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/snowflake/connector/ssl_wrap_socket.py", line 220, in recv_into
return self.connection.recv_into(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/OpenSSL/SSL.py", line 1715, in recv_into
self._raise_ssl_error(self._ssl, result)
File "/usr/local/lib/python3.6/site-packages/OpenSSL/SSL.py", line 1521, in _raise_ssl_error
raise WantReadError()
OpenSSL.SSL.WantReadError
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/snowflake/connector/telemetry.py", line 96, in send_batch
method='post', client=None)
File "/usr/local/lib/python3.6/site-packages/snowflake/connector/network.py", line 282, in request
timeout=self._connection.network_timeout)
File "/usr/local/lib/python3.6/site-packages/snowflake/connector/network.py", line 449, in _post_request
no_retry=no_retry, socket_timeout=socket_timeout)
File "/usr/local/lib/python3.6/site-packages/snowflake/connector/network.py", line 516, in fetch
**kwargs)
File "/usr/local/lib/python3.6/site-packages/snowflake/connector/network.py", line 579, in _request_exec_wrapper
raise e
File "/usr/local/lib/python3.6/site-packages/snowflake/connector/network.py", line 542, in _request_exec_wrapper
**kwargs)
File "/usr/local/lib/python3.6/site-packages/snowflake/connector/network.py", line 770, in _request_exec
raise err
File "/usr/local/lib/python3.6/site-packages/snowflake/connector/network.py", line 675, in _request_exec
auth=SnowflakeAuth(token),
File "/usr/local/lib/python3.6/site-packages/botocore/vendored/requests/sessions.py", line 465, in request
resp = self.send(prep, **send_kwargs)
File "/usr/local/lib/python3.6/site-packages/botocore/vendored/requests/sessions.py", line 573, in send
r = adapter.send(request, **kwargs)
File "/usr/local/lib/python3.6/site-packages/botocore/vendored/requests/adapters.py", line 370, in send
timeout=timeout
File "/usr/local/lib/python3.6/site-packages/botocore/vendored/requests/packages/urllib3/connectionpool.py", line 544, in urlopen
body=body, headers=headers)
File "/usr/local/lib/python3.6/site-packages/botocore/vendored/requests/packages/urllib3/connectionpool.py", line 374, in _make_request
httplib_response = conn.getresponse()
File "/usr/lib64/python3.6/http/client.py", line 1331, in getresponse
response.begin()
File "/usr/lib64/python3.6/http/client.py", line 297, in begin
version, status, reason = self._read_status()
File "/usr/lib64/python3.6/http/client.py", line 258, in _read_status
line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
File "/usr/lib64/python3.6/socket.py", line 586, in readinto
return self._sock.recv_into(b)
File "/usr/local/lib/python3.6/site-packages/snowflake/connector/ssl_wrap_socket.py", line 236, in recv_into
return self.recv_into(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/snowflake/connector/ssl_wrap_socket.py", line 220, in recv_into
return self.connection.recv_into(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/OpenSSL/SSL.py", line 1715, in recv_into
self._raise_ssl_error(self._ssl, result)
File "/usr/local/lib/python3.6/site-packages/OpenSSL/SSL.py", line 1546, in _raise_ssl_error
_raise_current_error()
File "/usr/local/lib/python3.6/site-packages/OpenSSL/_util.py", line 54, in exception_from_error_queue
raise exception_type(errors)
OpenSSL.SSL.Error: [('SSL routines', 'ssl3_get_record', 'decryption failed or bad record mac')]
[DEBUG:snowflake.connector.network:580]--:--Ignored error
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/botocore/vendored/requests/packages/urllib3/connectionpool.py", line 372, in _make_request
httplib_response = conn.getresponse(buffering=True)
TypeError: getresponse() got an unexpected keyword argument 'buffering'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/snowflake/connector/ssl_wrap_socket.py", line 220, in recv_into
return self.connection.recv_into(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/OpenSSL/SSL.py", line 1715, in recv_into
self._raise_ssl_error(self._ssl, result)
File "/usr/local/lib/python3.6/site-packages/OpenSSL/SSL.py", line 1521, in _raise_ssl_error
raise WantReadError()
OpenSSL.SSL.WantReadError
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/snowflake/connector/ssl_wrap_socket.py", line 220, in recv_into
return self.connection.recv_into(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/OpenSSL/SSL.py", line 1715, in recv_into
self._raise_ssl_error(self._ssl, result)
File "/usr/local/lib/python3.6/site-packages/OpenSSL/SSL.py", line 1521, in _raise_ssl_error
raise WantReadError()
OpenSSL.SSL.WantReadError
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/snowflake/connector/network.py", line 542, in _request_exec_wrapper
**kwargs)
File "/usr/local/lib/python3.6/site-packages/snowflake/connector/network.py", line 770, in _request_exec
raise err
File "/usr/local/lib/python3.6/site-packages/snowflake/connector/network.py", line 675, in _request_exec
auth=SnowflakeAuth(token),
File "/usr/local/lib/python3.6/site-packages/botocore/vendored/requests/sessions.py", line 465, in request
resp = self.send(prep, **send_kwargs)
File "/usr/local/lib/python3.6/site-packages/botocore/vendored/requests/sessions.py", line 573, in send
r = adapter.send(request, **kwargs)
File "/usr/local/lib/python3.6/site-packages/botocore/vendored/requests/adapters.py", line 370, in send
timeout=timeout
File "/usr/local/lib/python3.6/site-packages/botocore/vendored/requests/packages/urllib3/connectionpool.py", line 544, in urlopen
body=body, headers=headers)
File "/usr/local/lib/python3.6/site-packages/botocore/vendored/requests/packages/urllib3/connectionpool.py", line 374, in _make_request
httplib_response = conn.getresponse()
File "/usr/lib64/python3.6/http/client.py", line 1331, in getresponse
response.begin()
File "/usr/lib64/python3.6/http/client.py", line 297, in begin
version, status, reason = self._read_status()
File "/usr/lib64/python3.6/http/client.py", line 258, in _read_status
line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
File "/usr/lib64/python3.6/socket.py", line 586, in readinto
return self._sock.recv_into(b)
File "/usr/local/lib/python3.6/site-packages/snowflake/connector/ssl_wrap_socket.py", line 236, in recv_into
return self.recv_into(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/snowflake/connector/ssl_wrap_socket.py", line 236, in recv_into
return self.recv_into(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/snowflake/connector/ssl_wrap_socket.py", line 220, in recv_into
return self.connection.recv_into(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/OpenSSL/SSL.py", line 1715, in recv_into
self._raise_ssl_error(self._ssl, result)
File "/usr/local/lib/python3.6/site-packages/OpenSSL/SSL.py", line 1546, in _raise_ssl_error
_raise_current_error()
File "/usr/local/lib/python3.6/site-packages/OpenSSL/_util.py", line 54, in exception_from_error_queue
raise exception_type(errors)
OpenSSL.SSL.Error: [('SSL routines', 'ssl3_get_record', 'decryption failed or bad record mac')]
[INFO:snowflake.connector.telemetry:106]--:--Failed to upload metrics to telemetry.
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/botocore/vendored/requests/packages/urllib3/connectionpool.py", line 372, in _make_request
httplib_response = conn.getresponse(buffering=True)
TypeError: getresponse() got an unexpected keyword argument 'buffering'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/snowflake/connector/ssl_wrap_socket.py", line 220, in recv_into
return self.connection.recv_into(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/OpenSSL/SSL.py", line 1715, in recv_into
self._raise_ssl_error(self._ssl, result)
File "/usr/local/lib/python3.6/site-packages/OpenSSL/SSL.py", line 1521, in _raise_ssl_error
raise WantReadError()
OpenSSL.SSL.WantReadError
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/snowflake/connector/telemetry.py", line 96, in send_batch
method='post', client=None)
File "/usr/local/lib/python3.6/site-packages/snowflake/connector/network.py", line 282, in request
timeout=self._connection.network_timeout)
File "/usr/local/lib/python3.6/site-packages/snowflake/connector/network.py", line 449, in _post_request
no_retry=no_retry, socket_timeout=socket_timeout)
File "/usr/local/lib/python3.6/site-packages/snowflake/connector/network.py", line 516, in fetch
**kwargs)
File "/usr/local/lib/python3.6/site-packages/snowflake/connector/network.py", line 579, in _request_exec_wrapper
raise e
File "/usr/local/lib/python3.6/site-packages/snowflake/connector/network.py", line 542, in _request_exec_wrapper
**kwargs)
File "/usr/local/lib/python3.6/site-packages/snowflake/connector/network.py", line 770, in _request_exec
raise err
File "/usr/local/lib/python3.6/site-packages/snowflake/connector/network.py", line 675, in _request_exec
auth=SnowflakeAuth(token),
File "/usr/local/lib/python3.6/site-packages/botocore/vendored/requests/sessions.py", line 465, in request
resp = self.send(prep, **send_kwargs)
File "/usr/local/lib/python3.6/site-packages/botocore/vendored/requests/sessions.py", line 573, in send
r = adapter.send(request, **kwargs)
File "/usr/local/lib/python3.6/site-packages/botocore/vendored/requests/adapters.py", line 370, in send
timeout=timeout
File "/usr/local/lib/python3.6/site-packages/botocore/vendored/requests/packages/urllib3/connectionpool.py", line 544, in urlopen
body=body, headers=headers)
File "/usr/local/lib/python3.6/site-packages/botocore/vendored/requests/packages/urllib3/connectionpool.py", line 374, in _make_request
httplib_response = conn.getresponse()
File "/usr/lib64/python3.6/http/client.py", line 1331, in getresponse
response.begin()
File "/usr/lib64/python3.6/http/client.py", line 297, in begin
version, status, reason = self._read_status()
File "/usr/lib64/python3.6/http/client.py", line 258, in _read_status
line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
File "/usr/lib64/python3.6/socket.py", line 586, in readinto
return self._sock.recv_into(b)
File "/usr/local/lib/python3.6/site-packages/snowflake/connector/ssl_wrap_socket.py", line 236, in recv_into
return self.recv_into(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/snowflake/connector/ssl_wrap_socket.py", line 220, in recv_into
return self.connection.recv_into(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/OpenSSL/SSL.py", line 1715, in recv_into
self._raise_ssl_error(self._ssl, result)
File "/usr/local/lib/python3.6/site-packages/OpenSSL/SSL.py", line 1546, in _raise_ssl_error
_raise_current_error()
File "/usr/local/lib/python3.6/site-packages/OpenSSL/_util.py", line 54, in exception_from_error_queue
raise exception_type(errors)
OpenSSL.SSL.Error: [('SSL routines', 'ssl3_get_record', 'decryption failed or bad record mac')]
Python 3.6.6
pycrypto 2.6.1
pycryptodome 3.6.1
snowflake-connector-python 1.6.5
here is stack trace for Error --
File "/Users/sref/anaconda3/lib/python3.6/site-packages/botocore/vendored/requests/packages/urllib3/connection.py", line 238, in connect ssl_version=resolved_ssl_version)
File "/Users/sref/anaconda3/lib/python3.6/site-packages/snowflake/connector/ssl_wrap_socket.py", line 383, in ssl_wrap_socket_with_ocsp from .ocsp_asn1crypto import SnowflakeOCSP
File "/Users/sref/anaconda3/lib/python3.6/site-packages/snowflake/connector/ocsp_asn1crypto.py", line 21, in <module> from Crypto.Hash import SHA256, SHA384, SHA1, SHA512
File "/Users/sref/anaconda3/lib/python3.6/site-packages/Crypto/Hash/SHA1.py", line 23, in <module> from Crypto.Util._raw_api import (load_pycryptodome_raw_lib,
File "/Users/sref/anaconda3/lib/python3.6/site-packages/Crypto/Util/_raw_api.py", line 32, in <module> from Crypto.Util.py3compat import byte_string
ImportError: cannot import name 'byte_string'
As user not intending to use boto or azure modules, e.g. in case of using the connector for simple queries to Snowflake, is it be possible to relax the dependency requirements for these particular modules?
To repro:
import snowflake.connector as snowflake_connector
import boto3
sqs = boto3.resource('sqs')
if __name__ == '__main__':
queue = sqs.get_queue_by_name(QueueName='<queue-name>')
Will raise snowflake.connector.errors.OperationalError: 254007: The certificate attached is not for OCSP signing.
Getting an asn1crypto certificate error. A partial stacktrace follows. Haven't determined exact steps to reproduce other than trying to establish a connection.
File "../python2.7/site-packages/snowflake/connector/connection.py", line 123, in __init__
self.connect(**kwargs)
File "../python2.7/site-packages/snowflake/connector/connection.py", line 316, in connect
'password_callback'))
File "../python2.7/site-packages/snowflake/connector/connection.py", line 494, in __open_connection
session_parameters=self._session_parameters,
File "../python2.7/site-packages/snowflake/connector/auth.py", line 179, in authenticate
socket_timeout=self._rest._connection._login_timeout)
File "../python2.7/site-packages/snowflake/connector/network.py", line 387, in _post_request
no_retry=no_retry, socket_timeout=socket_timeout)
File "../python2.7/site-packages/snowflake/connector/network.py", line 454, in fetch
**kwargs)
File "../python2.7/site-packages/snowflake/connector/network.py", line 498, in _request_exec_wrapper
conn, full_url, cause)
File "../python2.7/site-packages/snowflake/connector/network.py", line 551, in handle_invalid_certificate_error
u'errno': ER_FAILED_TO_REQUEST,
File "../python2.7/site-packages/snowflake/connector/errors.py", line 100, in errorhandler_wrapper
connection.errorhandler(connection, cursor, errorclass, errorvalue)
File "../python2.7/site-packages/snowflake/connector/errors.py", line 73, in default_errorhandler
done_format_msg=errorvalue.get(u'done_format_msg'))
OperationalError: 250003: Failed to execute request: Error parsing asn1crypto.ocsp.Certificates - explicitly-tagged class should have been context, but universal was found
while parsing asn1crypto.ocsp.BasicOCSPResponse
This line of converter.py is trying to call utcoffset
on the tzinfo
object attached to a datetime. The python standard library implementation of tzinfo
doesn't take any keyword arguments to the utcoffset
method. The pytz
library has a separate implementation of utcoffset
which takes an optional kw argument of is_dst
.
None of the documentation I've seen states that datetimes need to have pytz
tzinfo objects instead of python datetime tzinfo objects.
The line in question has been a part of the repo since the initial release, I'm surprised that no one else has reported an issue with it.
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/snowflake/connector/cursor.py", line 868, in __process_params_dict
c = to_snowflake(c)
File "/usr/lib/python3.6/site-packages/snowflake/connector/converter.py", line 548, in to_snowflake
type_name=type_name))(value)
File "/usr/lib/python3.6/site-packages/snowflake/connector/converter.py", line 582, in _datetime_to_snowflake
td = tzinfo_value.utcoffset(value, is_dst=False)
TypeError: utcoffset() takes no keyword arguments
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1193, in _execute_context
context)
File "/usr/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 507, in do_execute
cursor.execute(statement, parameters)
File "/usr/lib/python3.6/site-packages/snowflake/connector/cursor.py", line 442, in execute
processed_params = self.__process_params(params)
File "/usr/lib/python3.6/site-packages/snowflake/connector/cursor.py", line 886, in __process_params
return self.__process_params_dict(params)
File "/usr/lib/python3.6/site-packages/snowflake/connector/cursor.py", line 880, in __process_params_dict
self.connection, self, ProgrammingError, errorvalue)
File "/usr/lib/python3.6/site-packages/snowflake/connector/errors.py", line 97, in errorhandler_wrapper
cursor.errorhandler(connection, cursor, errorclass, errorvalue)
File "/usr/lib/python3.6/site-packages/snowflake/connector/errors.py", line 73, in default_errorhandler
done_format_msg=errorvalue.get(u'done_format_msg'))
snowflake.connector.errors.ProgrammingError: 252004: None: Failed processing pyformat-parameters; utcoffset() takes no keyword arguments
Can we get a heartbeat implemented here to eliminate the authentication bug?
with self.snowflake.cursor() as cursor:
cursor.execute("PUT file://{} @~/test.csv".format(file))
Results in the following warnings:
./usr/local/lib/python3.6/site-packages/snowflake/connector/file_transfer_agent.py:153: ResourceWarning: unclosed <socket.socket fd=5, family=AddressFamily.AF_INET, type=2049, proto=6, laddr=('x.x.x.x', 50992), raddr=('y.y.y.y', 443)>
self._transfer_accelerate_config()
/usr/local/lib/python3.6/site-packages/snowflake/connector/file_transfer_agent.py:341: ResourceWarning: unclosed file <_io.BufferedReader name='/tmp/tmpg2_7tg90/test.csv_c.gz'>
meta[u'real_src_file_name'])
Is this a bug in the connector or user error? The cursor and connection are closed properly.
When PUT command is executed from a cursor, the connector creates a temporary gzip file using Python's tmpfile
standard library. However, since these files are created in memory, it causes out of memory errors, if the file is large and there is not enough memory on the system to accomodate the file in /tmp
.
Please add an option to specify alternate locations for the temporary files so larger files can be created on disk instead of in memory.
It is not considered to be good practice to pin versions in the install_requires section of setup.py:
https://packaging.python.org/requirements/?highlight=install_requires
Pinned versions are best left to requirements.txt files for the purpose of achieving repeatable installations. The install_requires option is used to specify "abstract" dependencies.
This lack of flexibility is causing me some problems because other packages I use have different version requirements for packages like boto3.
If it is desired to place some limitations on the versions of packages that are depended upon, it is better to use either >= and/or >= to specify min/max or ranges. Pinning to specific versions is far too restrictive for install_requires.
Please update the setup.py to fix this so that developers that are using your package can have more control over their installations.
Hi,
When uploading a couple of hundred ORC data files with sizes around 50-100MB with a single PUT statement, multiple uploads fail with a 'NoneType' object has no attribute 'upload_file'
error.
In the debug log, the error happens when an S3 HeadObject request is performed with a return code of 400 - Bad request, the stack trace is:
Traceback (most recent call last):
File "snowflake/connector/file_transfer_agent.py", line 330, in upload_one_file
File "snowflake/connector/s3_util.py", line 634, in upload_one_file_to_s3_with_retry
File "snowflake/connector/s3_util.py", line 193, in upload_one_file_to_s3
AttributeError: 'NoneType' object has no attribute 'upload_file'
This error happens because in in line 193 of s3_util.py
, the akey
variable is None
, because when it is initialised in line 153 by the
akey = SnowflakeS3Util.get_s3_file_object(meta, meta[u'dst_file_name'])
statement, the get_s3_file_object
function returns None in line 660 in case of a metadata retrieval error (return code 400), which happened during the request.
Cryptography 2.3 contains fixes for security vulnerabilities. (See https://cryptography.io/en/latest/changelog/#v2-3 ) This version pin is potentially dangerous:
https://github.com/snowflakedb/snowflake-connector-python/blob/v1.6.5/setup.py#L48
Am using version 1.4.5 of the Snowflake-Connector-Python with Snowflake-SqlAlchemy version 1.0.7 on top of Python 3.6.2. Looks like my ETL code properly loaded in the data but it hangs after loading in data (code never finishes and gets below stack trace - scroll towards bottom to see errors). Not sure yet if this is a Snowflake-Connector-Python issue or a Snowflake-SqlAlchemy issue - let me know if I should also post this in the Snowflake-SqlAlchemy project issues.
==================== 1 passed, 23 warnings in 15.32 seconds ====================Directory for raw extracted csv files: /Users/jdfagan/tmp/property_ledger/METLIFE
Directory for transformed csv files for loading into database: /Users/jdfagan/tmp/property_ledger/METLIFE/transform
Loading source: /Users/jdfagan/tmp/property_ledger/METLIFE/transform/transform/CommercialBuilding.csv ...
database table: METLIFE_CommercialBuilding
| column | data_type |
| -------------------- | --------- |
| BuildingID | Text |
| BuildingName | Text |
| BuildingGLA | Number |
| Address | Text |
| Address2 | Text |
| City | Text |
| State | Text |
| ZipCode | Number |
| EntityID | Text |
| UseNaturalBreakpoint | Boolean |
| SqftType | Text |
| BuildingSuites | Number |
| OccupiedGLA | Number |
| AlternateAddress1 | Text |
| AlternateAddress2 | Text |
| AlternateCity | Text |
| AlternateState | Text |
| AlternateZipCode | Text |
| Active | Boolean |
| ManagerID | Text |
Creating transformer_loader for /Users/jdfagan/tmp/property_ledger/METLIFE/transform/CommercialBuildingSquareFootage.csv ...
Loading source: /Users/jdfagan/tmp/property_ledger/METLIFE/transform/transform/CommercialBuildingSquareFootage.csv ...
database table: METLIFE_CommercialBuildingSquareFootage
| column | data_type |
| ------------------ | --------- |
| BuildingID | Text |
| LeasableSquareFeet | Number |
| SquareFootType | Text |
| EffectiveDate | DateTime |
PASSEDException during reset or similar
Traceback (most recent call last):
File "/Users/jdfagan/miniconda3/envs/waypoint_extraction/lib/python3.6/site-packages/snowflake/connector/cursor.py", line 356, in _execute_helper
_no_results=_no_results)
File "/Users/jdfagan/miniconda3/envs/waypoint_extraction/lib/python3.6/site-packages/snowflake/connector/connection.py", line 607, in cmd_query
data, client=client, _no_results=_no_results)
File "/Users/jdfagan/miniconda3/envs/waypoint_extraction/lib/python3.6/site-packages/snowflake/connector/network.py", line 226, in request
timeout=self._connection._network_timeout)
File "/Users/jdfagan/miniconda3/envs/waypoint_extraction/lib/python3.6/site-packages/snowflake/connector/network.py", line 347, in _post_request
is_single_thread=is_single_thread)
File "/Users/jdfagan/miniconda3/envs/waypoint_extraction/lib/python3.6/site-packages/snowflake/connector/network.py", line 406, in fetch
proxies, timeouts, **kwargs)
File "/Users/jdfagan/miniconda3/envs/waypoint_extraction/lib/python3.6/site-packages/snowflake/connector/network.py", line 638, in _fetch
u'errno': ER_FAILED_TO_REQUEST,
File "/Users/jdfagan/miniconda3/envs/waypoint_extraction/lib/python3.6/site-packages/snowflake/connector/errors.py", line 100, in errorhandler_wrapper
connection.errorhandler(connection, cursor, errorclass, errorvalue)
File "/Users/jdfagan/miniconda3/envs/waypoint_extraction/lib/python3.6/site-packages/snowflake/connector/errors.py", line 73, in default_errorhandler
done_format_msg=errorvalue.get(u'done_format_msg'))
snowflake.connector.errors.OperationalError: 250003: Failed to get the response. Hanging? method: post, url: https://waypointbuilding.snowflakecomputing.com:443/queries/v1/query-request?requestId=2ba0faf9-7041-47d5-a037-4249f2dd03ee, proxies: None
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/jdfagan/miniconda3/envs/waypoint_extraction/lib/python3.6/site-packages/sqlalchemy/pool.py", line 687, in _finalize_fairy
fairy._reset(pool)
File "/Users/jdfagan/miniconda3/envs/waypoint_extraction/lib/python3.6/site-packages/sqlalchemy/pool.py", line 829, in _reset
pool._dialect.do_rollback(self)
File "/Users/jdfagan/miniconda3/envs/waypoint_extraction/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 440, in do_rollback
dbapi_connection.rollback()
File "/Users/jdfagan/miniconda3/envs/waypoint_extraction/lib/python3.6/site-packages/snowflake/connector/connection.py", line 371, in rollback
self.cursor().execute("ROLLBACK")
File "/Users/jdfagan/miniconda3/envs/waypoint_extraction/lib/python3.6/site-packages/snowflake/connector/cursor.py", line 453, in execute
_is_put_get=_is_put_get)
File "/Users/jdfagan/miniconda3/envs/waypoint_extraction/lib/python3.6/site-packages/snowflake/connector/cursor.py", line 359, in _execute_helper
signal.signal(signal.SIGINT, original_sigint)
File "/Users/jdfagan/miniconda3/envs/waypoint_extraction/lib/python3.6/signal.py", line 47, in signal
handler = _signal.signal(_enum_to_int(signalnum), _enum_to_int(handler))
TypeError: signal handler must be signal.SIG_IGN, signal.SIG_DFL, or a callable object
/Users/jdfagan/miniconda3/envs/waypoint_extraction/lib/python3.6/site-packages/snowflake/connector/ssl_wrap_socket.py:337: DeprecationWarning: str for cipher_list is no longer accepted, use bytes
/Users/jdfagan/miniconda3/envs/waypoint_extraction/lib/python3.6/site-packages/pyasn1/type/univ.py:740: DeprecationWarning: __int__ returned non-int (type SizedInteger). The ability to return an instance of a strict subclass of int is deprecated, and may be removed in a future version of Python.
/Users/jdfagan/miniconda3/envs/waypoint_extraction/lib/python3.6/site-packages/pyasn1/type/univ.py:740: DeprecationWarning: __int__ returned non-int (type SizedInteger). The ability to return an instance of a strict subclass of int is deprecated, and may be removed in a future version of Python.
/Users/jdfagan/miniconda3/envs/waypoint_extraction/lib/python3.6/site-packages/pyasn1/type/univ.py:740: DeprecationWarning: __int__ returned non-int (type SizedInteger). The ability to return an instance of a strict subclass of int is deprecated, and may be removed in a future version of Python.
/Users/jdfagan/miniconda3/envs/waypoint_extraction/lib/python3.6/site-packages/pyasn1/type/univ.py:740: DeprecationWarning: __int__ returned non-int (type SizedInteger). The ability to return an instance of a strict subclass of int is deprecated, and may be removed in a future version of Python.
Attempting to stop PyCharm pytest then has this continued error log:
/Users/jdfagan/miniconda3/envs/waypoint_extraction/lib/python3.6/site-packages/snowflake/connector/network.py:311: ResourceWarning: unclosed <socket.socket fd=7, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('192.168.1.250', 50017), raddr=('52.41.7.166', 443)>
/Users/jdfagan/miniconda3/envs/waypoint_extraction/lib/python3.6/site-packages/snowflake/connector/ssl_wrap_socket.py:337: DeprecationWarning: str for cipher_list is no longer accepted, use bytes
/Users/jdfagan/miniconda3/envs/waypoint_extraction/lib/python3.6/site-packages/pyasn1/type/univ.py:740: DeprecationWarning: int returned non-int (type SizedInteger). The ability to return an instance of a strict subclass of int is deprecated, and may be removed in a future version of Python.
/Users/jdfagan/miniconda3/envs/waypoint_extraction/lib/python3.6/site-packages/pyasn1/type/univ.py:740: DeprecationWarning: int returned non-int (type SizedInteger). The ability to return an instance of a strict subclass of int is deprecated, and may be removed in a future version of Python.
/Users/jdfagan/miniconda3/envs/waypoint_extraction/lib/python3.6/site-packages/pyasn1/type/univ.py:740: DeprecationWarning: int returned non-int (type SizedInteger). The ability to return an instance of a strict subclass of int is deprecated, and may be removed in a future version of Python.
/Users/jdfagan/miniconda3/envs/waypoint_extraction/lib/python3.6/site-packages/pyasn1/type/univ.py:740: DeprecationWarning: int returned non-int (type SizedInteger). The ability to return an instance of a strict subclass of int is deprecated, and may be removed in a future version of Python.
Finally could only stop the test by killing it:
Process finished with exit code 137 (interrupted by signal 9: SIGKILL)
See my test below, test_snowflake_with_sqlalchemy_one_row, that causes following stack trace:
Stack trace:
F2017-09-17 12:36:02,249 INFO sqlalchemy.engine.base.Engine SELECT CAST('test plain returns' AS VARCHAR(60)) AS anon_1
2017-09-17 12:36:02,249 INFO sqlalchemy.engine.base.Engine {}
2017-09-17 12:36:02,378 INFO sqlalchemy.engine.base.Engine SELECT CAST('test unicode returns' AS VARCHAR(60)) AS anon_1
2017-09-17 12:36:02,378 INFO sqlalchemy.engine.base.Engine {}
2017-09-17 12:36:02,643 INFO sqlalchemy.engine.base.Engine DESC TABLE /* sqlalchemy:_has_object */ users
2017-09-17 12:36:02,643 INFO sqlalchemy.engine.base.Engine {}
2017-09-17 12:36:02,783 INFO sqlalchemy.engine.base.Engine DESC TABLE /* sqlalchemy:_has_object */ addresses
2017-09-17 12:36:02,783 INFO sqlalchemy.engine.base.Engine {}
2017-09-17 12:36:03,073 INFO sqlalchemy.engine.base.Engine INSERT INTO users (id, name, age, password) VALUES (%(id)s, %(name)s, %(age)s, %(password)s)
2017-09-17 12:36:03,073 INFO sqlalchemy.engine.base.Engine {'id': None, 'name': 'Nora', 'age': 0, 'password': 'wolf'}
2017-09-17 12:36:04,065 INFO sqlalchemy.engine.base.Engine ROLLBACK
test_snowflake.py:52 (test_snowflake_with_sqlalchemy_one_row)
self = <sqlalchemy.engine.base.Connection object at 0x10de49d30>
dialect = <snowflake.sqlalchemy.base.SnowflakeDialect object at 0x10dec1e10>
constructor = <bound method DefaultExecutionContext._init_compiled of <class 'snowflake.sqlalchemy.base.SnowflakeExecutionContext'>>
statement = 'INSERT INTO users (id, name, age, password) VALUES (%(id)s, %(name)s, %(age)s, %(password)s)'
parameters = {'age': 0, 'id': None, 'name': 'Nora', 'password': 'wolf'}
args = (<snowflake.sqlalchemy.base.SnowflakeCompiler object at 0x10dec6c50>, [{'age': 0, 'name': 'Nora', 'password': 'wolf'}])
conn = <sqlalchemy.pool._ConnectionFairy object at 0x10de49208>
context = <snowflake.sqlalchemy.base.SnowflakeExecutionContext object at 0x10dec15f8>
def _execute_context(self, dialect, constructor,
statement, parameters,
*args):
"""Create an :class:`.ExecutionContext` and execute, returning
a :class:`.ResultProxy`."""
try:
try:
conn = self.__connection
except AttributeError:
conn = self._revalidate_connection()
context = constructor(dialect, self, conn, *args)
except BaseException as e:
self._handle_dbapi_exception(
e,
util.text_type(statement), parameters,
None, None)
if context.compiled:
context.pre_exec()
cursor, statement, parameters = context.cursor, \
context.statement, \
context.parameters
if not context.executemany:
parameters = parameters[0]
if self._has_events or self.engine._has_events:
for fn in self.dispatch.before_cursor_execute:
statement, parameters = \
fn(self, cursor, statement, parameters,
context, context.executemany)
if self._echo:
self.engine.logger.info(statement)
self.engine.logger.info(
"%r",
sql_util._repr_params(parameters, batches=10)
)
evt_handled = False
try:
if context.executemany:
if self.dialect._has_events:
for fn in self.dialect.dispatch.do_executemany:
if fn(cursor, statement, parameters, context):
evt_handled = True
break
if not evt_handled:
self.dialect.do_executemany(
cursor,
statement,
parameters,
context)
elif not parameters and context.no_parameters:
if self.dialect._has_events:
for fn in self.dialect.dispatch.do_execute_no_params:
if fn(cursor, statement, context):
evt_handled = True
break
if not evt_handled:
self.dialect.do_execute_no_params(
cursor,
statement,
context)
else:
if self.dialect._has_events:
for fn in self.dialect.dispatch.do_execute:
if fn(cursor, statement, parameters, context):
evt_handled = True
break
if not evt_handled:
self.dialect.do_execute(
cursor,
statement,
parameters,
> context)
/Users/jdfagan/miniconda3/envs/waypoint_extraction/lib/python3.6/site-packages/sqlalchemy/engine/base.py:1182:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <snowflake.sqlalchemy.base.SnowflakeDialect object at 0x10dec1e10>
cursor = <snowflake.connector.cursor.SnowflakeCursor object at 0x10dec6e80>
statement = 'INSERT INTO users (id, name, age, password) VALUES (%(id)s, %(name)s, %(age)s, %(password)s)'
parameters = {'age': 0, 'id': None, 'name': 'Nora', 'password': 'wolf'}
context = <snowflake.sqlalchemy.base.SnowflakeExecutionContext object at 0x10dec15f8>
def do_execute(self, cursor, statement, parameters, context=None):
> cursor.execute(statement, parameters)
/Users/jdfagan/miniconda3/envs/waypoint_extraction/lib/python3.6/site-packages/sqlalchemy/engine/default.py:470:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <snowflake.connector.cursor.SnowflakeCursor object at 0x10dec6e80>
command = 'INSERT INTO users (id, name, age, password) VALUES (%(id)s, %(name)s, %(age)s, %(password)s)'
params = {'age': 0, 'id': None, 'name': 'Nora', 'password': 'wolf'}
timeout = None, _do_reset = True, _put_callback = None
_put_callback_output_stream = <_pytest.capture.EncodedFile object at 0x10c6ab6a0>
_get_callback = None
_get_callback_output_stream = <_pytest.capture.EncodedFile object at 0x10c6ab6a0>
_statement_params = None, _is_internal = False, _no_results = False
_use_ijson = False, _is_put_get = None
def execute(self, command, params=None, timeout=None,
_do_reset=True,
_put_callback=None,
_put_callback_output_stream=sys.stdout,
_get_callback=None,
_get_callback_output_stream=sys.stdout,
_statement_params=None,
_is_internal=False,
_no_results=False,
_use_ijson=False,
_is_put_get=None):
u"""
Executes a command/query
"""
logger.debug(u'executing SQL/command')
if self.is_closed():
Error.errorhandler_wrapper(
self.connection, self,
DatabaseError,
{u'msg': u"Cursor is closed in execute.",
u'errno': ER_CURSOR_IS_CLOSED})
if _do_reset:
self.reset()
command = command.strip(u' \t\n\r') if command else None
if not command:
logger.warning(u'execute: no query is given to execute')
return
processed_params = self.__process_params(params)
logger.debug(u'binding: %s with input=%s, processed=%s',
command,
params, processed_params)
if len(processed_params) > 0:
query = command % processed_params
else:
query = command
if logger.getEffectiveLevel() <= logging.DEBUG:
logger.debug(
u'query: [%s]',
u' '.join(line.strip() for line in query.split(u'\n')))
m = DESC_TABLE_RE.match(query)
if m:
query1 = u'describe table {0}'.format(m.group(1))
if logger.getEffectiveLevel() <= logging.WARNING:
logger.warning(
u'query was rewritten: org=%s, new=%s',
u' '.join(line.strip() for line in query.split(u'\n')),
query1
)
query = query1
ret = self._execute_helper(query, timeout=timeout,
statement_params=_statement_params,
is_internal=_is_internal,
_no_results=_no_results,
_is_put_get=_is_put_get)
self._sfqid = ret[u'data'][
u'queryId'] if u'data' in ret and u'queryId' in ret[
u'data'] else None
self._sqlstate = ret[u'data'][
u'sqlState'] if u'data' in ret and u'sqlState' in ret[
u'data'] else None
logger.debug('sfqid: %s', self.sfqid)
if ret[u'success']:
logger.debug(u'SUCCESS')
data = ret[u'data']
if u'finalDatabaseName' in data:
self._connection._database = data[u'finalDatabaseName']
if u'finalSchemaName' in data:
self._connection._schema = data[u'finalSchemaName']
if u'finalWarehouseName' in data:
self._connection._warehouse = data[u'finalWarehouseName']
if u'finalRoleName' in data:
self._connection._role = data[u'finalRoleName']
# logger.debug(ret)
logger.debug(u"PUT OR GET: %s", self.is_file_transfer)
if self.is_file_transfer:
sf_file_transfer_agent = SnowflakeFileTransferAgent(
self, query, ret,
put_callback=_put_callback,
put_callback_output_stream=_put_callback_output_stream,
get_callback=_get_callback,
get_callback_output_stream=_get_callback_output_stream)
sf_file_transfer_agent.execute()
data = sf_file_transfer_agent.result()
self._total_rowcount = len(data[u'rowset']) if \
u'rowset' in data else -1
m = self.ALTER_SESSION_RE.match(query)
if m:
# session parameters
param = m.group(1).upper()
value = m.group(2)
self._connection.converter.set_parameter(param, value)
if _no_results:
self._total_rowcount = ret[u'data'][
u'total'] if u'data' in ret and u'total' in ret[
u'data'] else -1
return data
self.chunk_info(data, use_ijson=_use_ijson)
else:
self._total_rowcount = ret[u'data'][
u'total'] if u'data' in ret and u'total' in ret[u'data'] else -1
logger.debug(ret)
err = ret[u'message']
code = ret.get(u'code', -1)
if u'data' in ret and u'errorMessage' in ret[u'data']:
err += ret[u'data'][u'errorMessage']
errvalue = {
u'msg': err,
u'errno': int(code),
u'sqlstate': self._sqlstate,
u'sfqid': self._sfqid
}
Error.errorhandler_wrapper(self.connection, self,
ProgrammingError,
> errvalue)
/Users/jdfagan/miniconda3/envs/waypoint_extraction/lib/python3.6/site-packages/snowflake/connector/cursor.py:526:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
connection = <snowflake.connector.connection.SnowflakeConnection object at 0x10dec6400>
cursor = <snowflake.connector.cursor.SnowflakeCursor object at 0x10dec6e80>
errorclass = <class 'snowflake.connector.errors.ProgrammingError'>
errorvalue = {'done_format_msg': False, 'errno': 100072, 'msg': 'NULL result in a non-nullable column', 'sfqid': '571cba58-7910-46e9-b975-505ce4aac5d4', ...}
@staticmethod
def errorhandler_wrapper(connection, cursor, errorclass, errorvalue=None):
u"""
Error handler wrapper that calls the errorhandler method
"""
if errorvalue is None:
# no value indicates errorclass is errorobject
errorobject = errorclass
errorclass = type(errorobject)
errorvalue = {
u'msg': errorobject.msg,
u'errno': errorobject.errno,
u'sqlstate': errorobject.sqlstate,
u'done_format_msg': True
}
else:
errorvalue[u'done_format_msg'] = False
if connection is not None:
connection.messages.append((errorclass, errorvalue))
if cursor is not None:
cursor.messages.append((errorclass, errorvalue))
> cursor.errorhandler(connection, cursor, errorclass, errorvalue)
/Users/jdfagan/miniconda3/envs/waypoint_extraction/lib/python3.6/site-packages/snowflake/connector/errors.py:97:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
connection = <snowflake.connector.connection.SnowflakeConnection object at 0x10dec6400>
cursor = <snowflake.connector.cursor.SnowflakeCursor object at 0x10dec6e80>
errorclass = <class 'snowflake.connector.errors.ProgrammingError'>
errorvalue = {'done_format_msg': False, 'errno': 100072, 'msg': 'NULL result in a non-nullable column', 'sfqid': '571cba58-7910-46e9-b975-505ce4aac5d4', ...}
@staticmethod
def default_errorhandler(connection, cursor, errorclass, errorvalue):
u"""
Default error handler that raises an error
"""
raise errorclass(
msg=errorvalue.get(u'msg'),
errno=errorvalue.get(u'errno'),
sqlstate=errorvalue.get(u'sqlstate'),
sfqid=errorvalue.get(u'sfqid'),
> done_format_msg=errorvalue.get(u'done_format_msg'))
E snowflake.connector.errors.ProgrammingError: 100072 (22000): NULL result in a non-nullable column
/Users/jdfagan/miniconda3/envs/waypoint_extraction/lib/python3.6/site-packages/snowflake/connector/errors.py:73: ProgrammingError
The above exception was the direct cause of the following exception:
def test_snowflake_with_sqlalchemy_one_row():
sqlalchemy_engine = create_engine(URL(
account='**********',
user='jdfagan',
password='**********',
database='qa',
schema='market_benchmark_report',
warehouse='x_small'
))
sqlalchemy_engine.echo = True
# Create tables in database
metadata = MetaData(bind=sqlalchemy_engine)
users_table = Table('users', metadata,
Column('id', Integer, primary_key=True),
Column('name', String(40)),
Column('age', Integer),
Column('password', String),
)
addresses_table = Table('addresses', metadata,
Column('id', Integer, primary_key=True),
Column('user_id', None, ForeignKey('users.id')),
Column('email_address', String, nullable=False)
)
metadata.create_all()
# Populate tables
# Option 1) Brittle way where connection forgotten to be closed
conn = sqlalchemy_engine.connect()
conn.execute(users_table.insert(), [
> {"name": "Nora", "age": 0, "password": "wolf"},
])
test_snowflake.py:83:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/Users/jdfagan/miniconda3/envs/waypoint_extraction/lib/python3.6/site-packages/sqlalchemy/engine/base.py:945: in execute
return meth(self, multiparams, params)
/Users/jdfagan/miniconda3/envs/waypoint_extraction/lib/python3.6/site-packages/sqlalchemy/sql/elements.py:263: in _execute_on_connection
return connection._execute_clauseelement(self, multiparams, params)
/Users/jdfagan/miniconda3/envs/waypoint_extraction/lib/python3.6/site-packages/sqlalchemy/engine/base.py:1053: in _execute_clauseelement
compiled_sql, distilled_params
/Users/jdfagan/miniconda3/envs/waypoint_extraction/lib/python3.6/site-packages/sqlalchemy/engine/base.py:1189: in _execute_context
context)
/Users/jdfagan/miniconda3/envs/waypoint_extraction/lib/python3.6/site-packages/sqlalchemy/engine/base.py:1402: in _handle_dbapi_exception
exc_info
/Users/jdfagan/miniconda3/envs/waypoint_extraction/lib/python3.6/site-packages/sqlalchemy/util/compat.py:203: in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
/Users/jdfagan/miniconda3/envs/waypoint_extraction/lib/python3.6/site-packages/sqlalchemy/util/compat.py:186: in reraise
raise value.with_traceback(tb)
/Users/jdfagan/miniconda3/envs/waypoint_extraction/lib/python3.6/site-packages/sqlalchemy/engine/base.py:1182: in _execute_context
context)
/Users/jdfagan/miniconda3/envs/waypoint_extraction/lib/python3.6/site-packages/sqlalchemy/engine/default.py:470: in do_execute
cursor.execute(statement, parameters)
/Users/jdfagan/miniconda3/envs/waypoint_extraction/lib/python3.6/site-packages/snowflake/connector/cursor.py:526: in execute
errvalue)
/Users/jdfagan/miniconda3/envs/waypoint_extraction/lib/python3.6/site-packages/snowflake/connector/errors.py:97: in errorhandler_wrapper
cursor.errorhandler(connection, cursor, errorclass, errorvalue)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
connection = <snowflake.connector.connection.SnowflakeConnection object at 0x10dec6400>
cursor = <snowflake.connector.cursor.SnowflakeCursor object at 0x10dec6e80>
errorclass = <class 'snowflake.connector.errors.ProgrammingError'>
errorvalue = {'done_format_msg': False, 'errno': 100072, 'msg': 'NULL result in a non-nullable column', 'sfqid': '571cba58-7910-46e9-b975-505ce4aac5d4', ...}
@staticmethod
def default_errorhandler(connection, cursor, errorclass, errorvalue):
u"""
Default error handler that raises an error
"""
raise errorclass(
msg=errorvalue.get(u'msg'),
errno=errorvalue.get(u'errno'),
sqlstate=errorvalue.get(u'sqlstate'),
sfqid=errorvalue.get(u'sfqid'),
> done_format_msg=errorvalue.get(u'done_format_msg'))
E sqlalchemy.exc.ProgrammingError: (snowflake.connector.errors.ProgrammingError) 100072 (22000): NULL result in a non-nullable column [SQL: 'INSERT INTO users (id, name, age, password) VALUES (%(id)s, %(name)s, %(age)s, %(password)s)'] [parameters: {'id': None, 'name': 'Nora', 'age': 0, 'password': 'wolf'}]
/Users/jdfagan/miniconda3/envs/waypoint_extraction/lib/python3.6/site-packages/snowflake/connector/errors.py:73: ProgrammingError
Test code (I'm using pytest to run these tests - only one fails, the *_one_row test):
import logging
import snowflake.connector
from snowflake.sqlalchemy import URL
from sqlalchemy import *
from sqlalchemy.pool import QueuePool
# from sqlalchemy.pool import NullPool
# from sqlalchemy.pool import SingletonThreadPool
def test_snowflake_with_sqlalchemy():
sqlalchemy_engine = create_engine(URL(
account='**********',
user='jdfagan',
password='**********',
database='qa',
schema='market_benchmark_report',
warehouse='x_small'
))
sqlalchemy_engine.echo = True
# Create tables in database
metadata = MetaData(bind=sqlalchemy_engine)
users_table = Table('users', metadata,
Column('id', Integer, primary_key=True),
Column('name', String(40)),
Column('age', Integer),
Column('password', String),
)
addresses_table = Table('addresses', metadata,
Column('id', Integer, primary_key=True),
Column('user_id', None, ForeignKey('users.id')),
Column('email_address', String, nullable=False)
)
metadata.create_all()
# Populate tables
# Option 1) Brittle way where connection forgotten to be closed
conn = sqlalchemy_engine.connect()
conn.execute(users_table.insert(), [
{"name": "Nora", "age": 0, "password": "wolf"},
{"name": "Ted", "age": 10, "password": "dink"},
])
# The bug that is easy to create (i.e., not closing the connection using next line)
# conn.close()
# Query tables
s = select([users_table])
result = s.execute()
for row in result:
print(row)
def test_snowflake_with_sqlalchemy_one_row():
sqlalchemy_engine = create_engine(URL(
account='**********',
user='jdfagan',
password='**********',
database='qa',
schema='market_benchmark_report',
warehouse='x_small'
))
sqlalchemy_engine.echo = True
# Create tables in database
metadata = MetaData(bind=sqlalchemy_engine)
users_table = Table('users', metadata,
Column('id', Integer, primary_key=True),
Column('name', String(40)),
Column('age', Integer),
Column('password', String),
)
addresses_table = Table('addresses', metadata,
Column('id', Integer, primary_key=True),
Column('user_id', None, ForeignKey('users.id')),
Column('email_address', String, nullable=False)
)
metadata.create_all()
# Populate tables
# Option 1) Brittle way where connection forgotten to be closed
conn = sqlalchemy_engine.connect()
conn.execute(users_table.insert(), [
{"name": "Nora", "age": 0, "password": "wolf"},
])
# The bug that is easy to create (i.e., not closing the connection using next line)
# conn.close()
# Query tables
s = select([users_table])
result = s.execute()
for row in result:
print(row)
def test_snowflake_with_sqlalchemy_with_context_manager():
sqlalchemy_engine = create_engine(URL(
account='**********',
user='jdfagan',
password='**********',
database='qa',
schema='market_benchmark_report',
warehouse='x_small'
))
sqlalchemy_engine.echo = True
# Create tables in database
metadata = MetaData(bind=sqlalchemy_engine)
users_table = Table('users', metadata,
Column('id', Integer, primary_key=True),
Column('name', String(40)),
Column('age', Integer),
Column('password', String),
)
addresses_table = Table('addresses', metadata,
Column('id', Integer, primary_key=True),
Column('user_id', None, ForeignKey('users.id')),
Column('email_address', String, nullable=False)
)
metadata.create_all()
# Populate tables
# Option 2) Preferred way to clean up open connections
with sqlalchemy_engine.connect() as conn:
conn.execute(users_table.insert(), [
{"name": "Conor", "age": 1, "password": "giraffe"},
{"name": "Asahina", "age": 25, "password": "nippon"},
])
# Query tables
s = select([users_table])
result = s.execute()
for row in result:
print(row)
def test_snowflake_with_sqlalchemy_with_dispose():
sqlalchemy_engine = create_engine(URL(
account='**********',
user='jdfagan',
password='**********',
database='qa',
schema='market_benchmark_report',
warehouse='x_small'
))
try:
sqlalchemy_engine.echo = True
# Create tables in database
metadata = MetaData(bind=sqlalchemy_engine)
users_table = Table('users', metadata,
Column('id', Integer, primary_key=True),
Column('name', String(40)),
Column('age', Integer),
Column('password', String),
)
addresses_table = Table('addresses', metadata,
Column('id', Integer, primary_key=True),
Column('user_id', None, ForeignKey('users.id')),
Column('email_address', String, nullable=False)
)
metadata.create_all()
# Populate tables
# Option 1) Brittle way where connection forgotten to be closed
conn = sqlalchemy_engine.connect()
conn.execute(users_table.insert(), [
{"name": "Arthur", "age": 2, "password": "zebra"},
{"name": "Evan", "age": 40, "password": "macaca"},
])
# The bug that is easy to create (i.e., not closing the connection using next line)
# conn.close()
# Query tables
s = select([users_table])
result = s.execute()
for row in result:
print(row)
finally:
print('dispose started')
sqlalchemy_engine.dispose()
print('dispose ended')
def test_snowflake_with_sqlalchemy_with_context_manager_with_dispose():
sqlalchemy_engine = create_engine(URL(
account='**********',
user='jdfagan',
password='**********',
database='qa',
schema='market_benchmark_report',
warehouse='x_small'
))
try:
sqlalchemy_engine.echo = True
# Create tables in database
metadata = MetaData(bind=sqlalchemy_engine)
users_table = Table('users', metadata,
Column('id', Integer, primary_key=True),
Column('name', String(40)),
Column('age', Integer),
Column('password', String),
)
addresses_table = Table('addresses', metadata,
Column('id', Integer, primary_key=True),
Column('user_id', None, ForeignKey('users.id')),
Column('email_address', String, nullable=False)
)
metadata.create_all()
# Populate tables
# Option 2) Preferred way to clean up open connections
with sqlalchemy_engine.connect() as conn:
conn.execute(users_table.insert(), [
{"name": "Lola", "age": 4, "password": "lamb"},
{"name": "Ted", "age": 10, "password": "dink"},
])
# Query tables
s = select([users_table])
result = s.execute()
for row in result:
print(row)
finally:
print('dispose started')
sqlalchemy_engine.dispose()
print('dispose ended')
def test_snowflake_with_sqlalchemy_with_connection_pool():
for logger_name in ['snowflake.connector', 'botocore']:
logger = logging.getLogger(logger_name)
logger.setLevel(logging.DEBUG)
ch = logging.FileHandler('log')
ch.setLevel(logging.DEBUG)
ch.setFormatter(logging.Formatter(
'%(asctime)s - %(threadName)s %(filename)s:%(lineno)d - %(funcName)s() - %(levelname)s - %(message)s'))
logger.addHandler(ch)
sqlalchemy_engine = create_engine(URL(
account='**********',
user='jdfagan',
password='**********',
database='qa',
schema='market_benchmark_report',
warehouse='x_small',
), poolclass=QueuePool)
# poolclass=SingletonThreadPool, pool_size=5)
# poolclass=NullPool)
# poolclass=NewQueuePool)
sqlalchemy_engine.echo = True
# Create tables in database
metadata = MetaData(bind=sqlalchemy_engine)
users_table = Table('users', metadata,
Column('id', Integer, primary_key=True),
Column('name', String(40)),
Column('age', Integer),
Column('password', String),
)
addresses_table = Table('addresses', metadata,
Column('id', Integer, primary_key=True),
Column('user_id', None, ForeignKey('users.id')),
Column('email_address', String, nullable=False)
)
metadata.create_all()
# Populate tables
# Option 1) Brittle way where connection forgotten to be closed
conn = sqlalchemy_engine.connect()
conn.execute(users_table.insert(), [
{"name": "Penelope", "age": 4, "password": "lion"},
{"name": "Asahina", "age": 25, "password": "nippon"},
])
# The bug that is easy to create (i.e., not closing the connection using next line)
# conn.close()
# Query tables
s = select([users_table])
result = s.execute()
for row in result:
print(row)
def test_snowflake_with_sqlalchemy_with_connection_pool_with_context_manager():
for logger_name in ['snowflake.connector', 'botocore']:
logger = logging.getLogger(logger_name)
logger.setLevel(logging.DEBUG)
ch = logging.FileHandler('log')
ch.setLevel(logging.DEBUG)
ch.setFormatter(logging.Formatter(
'%(asctime)s - %(threadName)s %(filename)s:%(lineno)d - %(funcName)s() - %(levelname)s - %(message)s'))
logger.addHandler(ch)
sqlalchemy_engine = create_engine(URL(
account='**********',
user='jdfagan',
password='**********',
database='qa',
schema='market_benchmark_report',
warehouse='x_small',
), poolclass=QueuePool)
# poolclass=SingletonThreadPool, pool_size=5)
# poolclass=NullPool)
# poolclass=NewQueuePool)
sqlalchemy_engine.echo = True
# Create tables in database
metadata = MetaData(bind=sqlalchemy_engine)
users_table = Table('users', metadata,
Column('id', Integer, primary_key=True),
Column('name', String(40)),
Column('age', Integer),
Column('password', String),
)
addresses_table = Table('addresses', metadata,
Column('id', Integer, primary_key=True),
Column('user_id', None, ForeignKey('users.id')),
Column('email_address', String, nullable=False)
)
metadata.create_all()
# Populate tables
# Option 2) Preferred way to clean up open connections
with sqlalchemy_engine.connect() as conn:
conn.execute(users_table.insert(), [
{"name": "Drew", "age": 5, "password": "bear"},
{"name": "Evan", "age": 40, "password": "macaca"},
])
# Query tables
s = select([users_table])
result = s.execute()
for row in result:
print(row)
def test_snowflake_with_sqlalchemy_with_connection_pool_with_dispose():
for logger_name in ['snowflake.connector', 'botocore']:
logger = logging.getLogger(logger_name)
logger.setLevel(logging.DEBUG)
ch = logging.FileHandler('log')
ch.setLevel(logging.DEBUG)
ch.setFormatter(logging.Formatter(
'%(asctime)s - %(threadName)s %(filename)s:%(lineno)d - %(funcName)s() - %(levelname)s - %(message)s'))
logger.addHandler(ch)
sqlalchemy_engine = create_engine(URL(
account='**********',
user='jdfagan',
password='**********',
database='qa',
schema='market_benchmark_report',
warehouse='x_small',
), poolclass=QueuePool)
# poolclass=SingletonThreadPool, pool_size=5)
# poolclass=NullPool)
# poolclass=NewQueuePool)
try:
sqlalchemy_engine.echo = True
# Create tables in database
metadata = MetaData(bind=sqlalchemy_engine)
users_table = Table('users', metadata,
Column('id', Integer, primary_key=True),
Column('name', String(40)),
Column('age', Integer),
Column('password', String),
)
addresses_table = Table('addresses', metadata,
Column('id', Integer, primary_key=True),
Column('user_id', None, ForeignKey('users.id')),
Column('email_address', String, nullable=False)
)
metadata.create_all()
# Populate tables
# Option 1) Brittle way where connection forgotten to be closed
conn = sqlalchemy_engine.connect()
conn.execute(users_table.insert(), [
{"name": "Henry", "age": 6, "password": "monkey"},
{"name": "Ted", "age": 10, "password": "dink"},
])
# The bug that is easy to create (i.e., not closing the connection using next line)
# conn.close()
# Query tables
s = select([users_table])
result = s.execute()
for row in result:
print(row)
finally:
print('dispose started')
sqlalchemy_engine.dispose()
print('dispose ended')
def test_snowflake_with_sqlalchemy_with_connection_pool_with_context_manager_with_dispose():
for logger_name in ['snowflake.connector', 'botocore']:
logger = logging.getLogger(logger_name)
logger.setLevel(logging.DEBUG)
ch = logging.FileHandler('log')
ch.setLevel(logging.DEBUG)
ch.setFormatter(logging.Formatter(
'%(asctime)s - %(threadName)s %(filename)s:%(lineno)d - %(funcName)s() - %(levelname)s - %(message)s'))
logger.addHandler(ch)
sqlalchemy_engine = create_engine(URL(
account='**********',
user='jdfagan',
password='**********',
database='qa',
schema='market_benchmark_report',
warehouse='x_small',
), poolclass=QueuePool)
# poolclass=SingletonThreadPool, pool_size=5)
# poolclass=NullPool)
# poolclass=NewQueuePool)
try:
sqlalchemy_engine.echo = True
# Create tables in database
metadata = MetaData(bind=sqlalchemy_engine)
users_table = Table('users', metadata,
Column('id', Integer, primary_key=True),
Column('name', String(40)),
Column('age', Integer),
Column('password', String),
)
addresses_table = Table('addresses', metadata,
Column('id', Integer, primary_key=True),
Column('user_id', None, ForeignKey('users.id')),
Column('email_address', String, nullable=False)
)
metadata.create_all()
# Populate tables
# Option 2) Preferred way to clean up open connections
with sqlalchemy_engine.connect() as conn:
conn.execute(users_table.insert(), [
{"name": "Molly", "age": 8, "password": "elephant"},
{"name": "Asahina", "age": 25, "password": "nippon"},
])
# Query tables
s = select([users_table])
result = s.execute()
for row in result:
print(row)
finally:
print('dispose started')
sqlalchemy_engine.dispose()
print('dispose ended')
Here is my environment info:
$ cat environment.yml
name: waypoint_extraction
channels:
- conda-forge
- defaults
- asmeurer
- snowflakedb
dependencies:
- agate=1.6.0=py36_2
- agate-dbf=0.2.0=py36_0
- agate-excel=0.2.1=py36_0
- agate-sql=0.5.2=py36_0
- asn1crypto=0.22.0=py36_0
- babel=2.4.0=py36_0
- ca-certificates=2017.7.27.1=0
- certifi=2017.7.27.1=py36_0
- cffi=1.10.0=py36_0
- chardet=3.0.4=py36_0
- coverage=4.4.1=py36_0
- cryptography=2.0.3=py36_0
- csvkit=1.0.2=py36_0
- dbfread=2.0.7=py36_0
- et_xmlfile=1.0.1=py36_0
- isodate=0.5.4=py36_0
- jdcal=1.3=py36_0
- leather=0.3.3=py36_0
- libffi=3.2.1=3
- mysql-connector-python=2.2.2=py36_0
- ncurses=5.9=10
- openpyxl=2.5.0a3=py36_0
- openssl=1.0.2l=0
- parsedatetime=2.4=py36_0
- pip=9.0.1=py36_0
- py=1.4.34=py36_0
- pycparser=2.18=py36_0
- pyopenssl=17.2.0=py36_0
- pysocks=1.6.7=py36_0
- pytest=3.2.2=py_0
- pytest-cov=2.5.1=py36_0
- python=3.6.2=0
- python-slugify=1.2.4=py36_1
- pytimeparse=1.1.7=py36_0
- pytz=2017.2=py36_0
- readline=6.2=0
- requests=2.18.4=py36_1
- requests-toolbelt=0.8.0=py36_0
- setuptools=36.3.0=py36_0
- six=1.10.0=py36_1
- sqlalchemy=1.1.13=py36_0
- sqlite=3.13.0=1
- tk=8.5.19=2
- unidecode=0.04.21=py36_0
- urllib3=1.22=py36_0
- wheel=0.29.0=py36_0
- xlrd=1.1.0=py36_1
- xz=5.2.3=0
- zlib=1.2.11=0
- idna=2.6=py36_0
- pymysql=0.7.9=py36_0
- pip:
- boto3==1.4.6
- botocore==1.6.8
- docutils==0.14
- et-xmlfile==1.0.1
- future==0.16.0
- hvac==0.2.17
- ijson==2.3
- jmespath==0.9.3
- pyasn1==0.2.3
- pyasn1-modules==0.0.9
- pycryptodome==3.4.7
- python-dateutil==2.6.1
- s3transfer==0.1.11
- snowflake-connector-python==1.4.6
- snowflake-sqlalchemy==1.0.7
prefix: /Users/jdfagan/miniconda3/envs/waypoint_extraction
When enabling numpy on my connection I run into errors parsing numbers with precision. They are interpreted as FIXED type and passed to _FIXED_numpy_to_python
as a string value; This function simply runs numpy.int64
which will never return a precision based value (eg float/decimal).
In this trace the column is a NUMBER(38,1) and the value is 1234.1
.
It appears that FIXED types should result in a precision-able type when precision != 0
. When numpy is disabled the high precision Decimal
type is used. Should this not also be the case in numpy? I don't see an equivalent in numpy to Decimal
so perhaps the same logic used without numpy could apply except it would return numpy.int64 when precision == 0
?
(Note that even this approach could be considered suspect because NUMBER can be 38 digits which far exceeds the int64 type (but not builtin int
). One might argue that numpy.int64 is always inadequate for NUMBER types.)
Command 'main' error, traceback...
Traceback (most recent call last)
2. File "/usr/lib/python3.5/site-packages/snowflake_connector_python-1.3.14-py3.5.egg/snowflake/connector/cursor.py", line 856, in _row_to_python
row[idx] = conv_method(col, conv_ctx)
1. File "/usr/lib/python3.5/site-packages/snowflake_connector_python-1.3.14-py3.5.egg/snowflake/connector/converter.py", line 115, in _FIXED_numpy_to_python
return numpy.int64(value)
ValueError: invalid literal for int() with base 10: '1234.1'
During handling of the above exception, another exception occurred:
Traceback (most recent call last)
10. File "/home/mayfield/project/shellish/shellish/session.py", line 119, in execute
raise e
9. File "/home/mayfield/project/shellish/shellish/session.py", line 116, in execute
result = command.run_wrap(args)
8. File "/home/mayfield/project/shellish/shellish/command/command.py", line 169, in run_wrap
raise e
7. File "/home/mayfield/project/shellish/shellish/command/command.py", line 165, in run_wrap
result = self.run(args)
6. File "/home/mayfield/project/shellish/shellish/command/autocommand.py", line 52, in run
return self.func(*positionals, **keywords)
5. File "bench_scale.py", line 69, in main
row = q.fetchone()
4. File "/usr/lib/python3.5/site-packages/snowflake_connector_python-1.3.14-py3.5.egg/snowflake/connector/cursor.py", line 689, in fetchone
return self._row_to_python(row) if row is not None else None
3. File "/usr/lib/python3.5/site-packages/snowflake_connector_python-1.3.14-py3.5.egg/snowflake/connector/cursor.py", line 871, in _row_to_python
u'errno': ER_FAILED_TO_CONVERT_ROW_TO_PYTHON_TYPE,
2. File "/usr/lib/python3.5/site-packages/snowflake_connector_python-1.3.14-py3.5.egg/snowflake/connector/errors.py", line 95, in errorhandler_wrapper
cursor.errorhandler(connection, cursor, errorclass, errorvalue)
1. File "/usr/lib/python3.5/site-packages/snowflake_connector_python-1.3.14-py3.5.egg/snowflake/connector/errors.py", line 71, in default_errorhandler
done_format_msg=errorvalue.get(u'done_format_msg'))
InterfaceError: 252005: None: Failed to convert: field DEC2: FIXED::1234.1, Error: invalid literal for int() with base 10: '1234.1'
Hello,
I'm getting this error below when trying to process a csv file where one of the values in a field has quotes in it naturally, so when properly quoted to deal with quotes, it makes the field size bit larger (but shouldn't count against field size length since Snowflake should properly decode the quotes to lower the field length back):
Query: put file:///Users/jdfagan/tmp/mri_cabinet/test/transform/BLDG.csv @"%test_BLDG"
['source', 'target', 'source_size', 'target_size', 'source_compression', 'target_compression', 'status', 'message']
('BLDG.csv', 'BLDG.csv.gz', 867679, 80410, 'NONE', 'GZIP', 'UPLOADED', '')
FAILED
tests/waypoint/etl/mri/test_cabinet_etl.py:104 (test_MriCabinetETL_load[test])
self = <sqlalchemy.engine.base.Connection object at 0x10ec7ec18>
dialect = <snowflake.sqlalchemy.base.SnowflakeDialect object at 0x110498240>
constructor = <bound method DefaultExecutionContext._init_statement of <class 'snowflake.sqlalchemy.base.SnowflakeExecutionContext'>>
statement = 'copy into "test_BLDG"', parameters = {}
args = ('copy into "test_BLDG"', [])
conn = <sqlalchemy.pool._ConnectionFairy object at 0x10d500390>
context = <snowflake.sqlalchemy.base.SnowflakeExecutionContext object at 0x10d50eba8>
def _execute_context(self, dialect, constructor,
statement, parameters,
*args):
"""Create an :class:`.ExecutionContext` and execute, returning
a :class:`.ResultProxy`."""
try:
try:
conn = self.__connection
except AttributeError:
conn = self._revalidate_connection()
context = constructor(dialect, self, conn, *args)
except BaseException as e:
self._handle_dbapi_exception(
e,
util.text_type(statement), parameters,
None, None)
if context.compiled:
context.pre_exec()
cursor, statement, parameters = context.cursor, \
context.statement, \
context.parameters
if not context.executemany:
parameters = parameters[0]
if self._has_events or self.engine._has_events:
for fn in self.dispatch.before_cursor_execute:
statement, parameters = \
fn(self, cursor, statement, parameters,
context, context.executemany)
if self._echo:
self.engine.logger.info(statement)
self.engine.logger.info(
"%r",
sql_util._repr_params(parameters, batches=10)
)
evt_handled = False
try:
if context.executemany:
if self.dialect._has_events:
for fn in self.dialect.dispatch.do_executemany:
if fn(cursor, statement, parameters, context):
evt_handled = True
break
if not evt_handled:
self.dialect.do_executemany(
cursor,
statement,
parameters,
context)
elif not parameters and context.no_parameters:
if self.dialect._has_events:
for fn in self.dialect.dispatch.do_execute_no_params:
if fn(cursor, statement, context):
evt_handled = True
break
if not evt_handled:
self.dialect.do_execute_no_params(
cursor,
statement,
context)
else:
if self.dialect._has_events:
for fn in self.dialect.dispatch.do_execute:
if fn(cursor, statement, parameters, context):
evt_handled = True
break
if not evt_handled:
self.dialect.do_execute(
cursor,
statement,
parameters,
> context)
../../../miniconda3/envs/data_warehouse/lib/python3.6/site-packages/sqlalchemy/engine/base.py:1182:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <snowflake.sqlalchemy.base.SnowflakeDialect object at 0x110498240>
cursor = <snowflake.connector.cursor.SnowflakeCursor object at 0x10ef87a90>
statement = 'copy into "test_BLDG"', parameters = {}
context = <snowflake.sqlalchemy.base.SnowflakeExecutionContext object at 0x10d50eba8>
def do_execute(self, cursor, statement, parameters, context=None):
> cursor.execute(statement, parameters)
../../../miniconda3/envs/data_warehouse/lib/python3.6/site-packages/sqlalchemy/engine/default.py:470:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <snowflake.connector.cursor.SnowflakeCursor object at 0x10ef87a90>
command = 'copy into "test_BLDG"', params = {}, timeout = None, _do_reset = True
_put_callback = None
_put_callback_output_stream = <_io.TextIOWrapper name='<stdout>' mode='w' encoding='UTF-8'>
_get_callback = None
_get_callback_output_stream = <_io.TextIOWrapper name='<stdout>' mode='w' encoding='UTF-8'>
_statement_params = None, _is_internal = False, _no_results = False
_use_ijson = False, _is_put_get = None
def execute(self, command, params=None, timeout=None,
_do_reset=True,
_put_callback=None,
_put_callback_output_stream=sys.stdout,
_get_callback=None,
_get_callback_output_stream=sys.stdout,
_statement_params=None,
_is_internal=False,
_no_results=False,
_use_ijson=False,
_is_put_get=None):
u"""
Executes a command/query
"""
logger.debug(u'executing SQL/command')
if self.is_closed():
Error.errorhandler_wrapper(
self.connection, self,
DatabaseError,
{u'msg': u"Cursor is closed in execute.",
u'errno': ER_CURSOR_IS_CLOSED})
if _do_reset:
self.reset()
command = command.strip(u' \t\n\r') if command else None
if not command:
logger.warning(u'execute: no query is given to execute')
return
processed_params = self.__process_params(params)
logger.debug(u'binding: %s with input=%s, processed=%s',
command,
params, processed_params)
if len(processed_params) > 0:
query = command % processed_params
else:
query = command
if logger.getEffectiveLevel() <= logging.DEBUG:
logger.debug(
u'query: [%s]',
u' '.join(line.strip() for line in query.split(u'\n')))
m = DESC_TABLE_RE.match(query)
if m:
query1 = u'describe table {0}'.format(m.group(1))
if logger.getEffectiveLevel() <= logging.WARNING:
logger.warning(
u'query was rewritten: org=%s, new=%s',
u' '.join(line.strip() for line in query.split(u'\n')),
query1
)
query = query1
ret = self._execute_helper(query, timeout=timeout,
statement_params=_statement_params,
is_internal=_is_internal,
_no_results=_no_results,
_is_put_get=_is_put_get)
self._sfqid = ret[u'data'][
u'queryId'] if u'data' in ret and u'queryId' in ret[
u'data'] else None
self._sqlstate = ret[u'data'][
u'sqlState'] if u'data' in ret and u'sqlState' in ret[
u'data'] else None
logger.debug('sfqid: %s', self.sfqid)
if ret[u'success']:
logger.debug(u'SUCCESS')
data = ret[u'data']
if u'finalDatabaseName' in data:
self._connection._database = data[u'finalDatabaseName']
if u'finalSchemaName' in data:
self._connection._schema = data[u'finalSchemaName']
if u'finalWarehouseName' in data:
self._connection._warehouse = data[u'finalWarehouseName']
if u'finalRoleName' in data:
self._connection._role = data[u'finalRoleName']
# logger.debug(ret)
logger.debug(u"PUT OR GET: %s", self.is_file_transfer)
if self.is_file_transfer:
sf_file_transfer_agent = SnowflakeFileTransferAgent(
self, query, ret,
put_callback=_put_callback,
put_callback_output_stream=_put_callback_output_stream,
get_callback=_get_callback,
get_callback_output_stream=_get_callback_output_stream)
sf_file_transfer_agent.execute()
data = sf_file_transfer_agent.result()
self._total_rowcount = len(data[u'rowset']) if \
u'rowset' in data else -1
m = self.ALTER_SESSION_RE.match(query)
if m:
# session parameters
param = m.group(1).upper()
value = m.group(2)
self._connection.converter.set_parameter(param, value)
if _no_results:
self._total_rowcount = ret[u'data'][
u'total'] if u'data' in ret and u'total' in ret[
u'data'] else -1
return data
self.chunk_info(data, use_ijson=_use_ijson)
else:
self._total_rowcount = ret[u'data'][
u'total'] if u'data' in ret and u'total' in ret[u'data'] else -1
logger.debug(ret)
err = ret[u'message']
code = ret.get(u'code', -1)
if u'data' in ret:
err += ret[u'data'].get(u'errorMessage', '')
errvalue = {
u'msg': err,
u'errno': int(code),
u'sqlstate': self._sqlstate,
u'sfqid': self._sfqid
}
Error.errorhandler_wrapper(self.connection, self,
ProgrammingError,
> errvalue)
../../../miniconda3/envs/data_warehouse/lib/python3.6/site-packages/snowflake/connector/cursor.py:526:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
connection = <snowflake.connector.connection.SnowflakeConnection object at 0x110497940>
cursor = <snowflake.connector.cursor.SnowflakeCursor object at 0x10ef87a90>
errorclass = <class 'snowflake.connector.errors.ProgrammingError'>
errorvalue = {'done_format_msg': False, 'errno': 100074, 'msg': 'User character length limit (30) exceeded by string \'"EVERETT COM...ding options, please run \'info loading_data\' in a SQL client.', 'sfqid': '08f751b4-3520-41e8-b0c7-8a5a20645315', ...}
@staticmethod
def errorhandler_wrapper(connection, cursor, errorclass, errorvalue=None):
u"""
Error handler wrapper that calls the errorhandler method
"""
if errorvalue is None:
# no value indicates errorclass is errorobject
errorobject = errorclass
errorclass = type(errorobject)
errorvalue = {
u'msg': errorobject.msg,
u'errno': errorobject.errno,
u'sqlstate': errorobject.sqlstate,
u'done_format_msg': True
}
else:
errorvalue[u'done_format_msg'] = False
if connection is not None:
connection.messages.append((errorclass, errorvalue))
if cursor is not None:
cursor.messages.append((errorclass, errorvalue))
> cursor.errorhandler(connection, cursor, errorclass, errorvalue)
../../../miniconda3/envs/data_warehouse/lib/python3.6/site-packages/snowflake/connector/errors.py:97:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
connection = <snowflake.connector.connection.SnowflakeConnection object at 0x110497940>
cursor = <snowflake.connector.cursor.SnowflakeCursor object at 0x10ef87a90>
errorclass = <class 'snowflake.connector.errors.ProgrammingError'>
errorvalue = {'done_format_msg': False, 'errno': 100074, 'msg': 'User character length limit (30) exceeded by string \'"EVERETT COM...ding options, please run \'info loading_data\' in a SQL client.', 'sfqid': '08f751b4-3520-41e8-b0c7-8a5a20645315', ...}
@staticmethod
def default_errorhandler(connection, cursor, errorclass, errorvalue):
u"""
Default error handler that raises an error
"""
raise errorclass(
msg=errorvalue.get(u'msg'),
errno=errorvalue.get(u'errno'),
sqlstate=errorvalue.get(u'sqlstate'),
sfqid=errorvalue.get(u'sfqid'),
> done_format_msg=errorvalue.get(u'done_format_msg'))
E snowflake.connector.errors.ProgrammingError: 100074 (54000): User character length limit (30) exceeded by string '"EVERETT COMMERCE CENTER ""A"""'
E File '@TEST_BLDG/BLDG.csv.gz', line 762, character 9
E Row 762, column "TEST_BLDG"[" BLDGNAME ":2]
E If you would like to continue loading when an error is encountered, use other values such as 'SKIP_FILE' or 'CONTINUE' for the ON_ERROR option. For more information on loading options, please run 'info loading_data' in a SQL client.
../../../miniconda3/envs/data_warehouse/lib/python3.6/site-packages/snowflake/connector/errors.py:73: ProgrammingError
The above exception was the direct cause of the following exception:
client = 'test'
@pytest.mark.etl
@pytest.mark.tl
@pytest.mark.cabinet
@pytest.mark.parametrize("client", clients)
def test_MriCabinetETL_load(client: str):
etl = MriCabinetETL(client_name=client, etl_dir="{}{}".format(ETL_DIR_BASE, client), delimiter='^')
> etl.load()
../../../Repositories/Waypoint/data_warehouse/tests/waypoint/etl/mri/test_cabinet_etl.py:111:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
../../../Repositories/Waypoint/data_warehouse/waypoint/etl/__init__.py:88: in load
loader.from_csv().to_sql(self.database_url, self.client_code)
../../../Repositories/Waypoint/data_warehouse/waypoint/etl/mri/cabinet_transformer_loader.py:55: in to_sql
result_proxy = engine.execute(query)
../../../miniconda3/envs/data_warehouse/lib/python3.6/site-packages/sqlalchemy/engine/base.py:2064: in execute
return connection.execute(statement, *multiparams, **params)
../../../miniconda3/envs/data_warehouse/lib/python3.6/site-packages/sqlalchemy/engine/base.py:939: in execute
return self._execute_text(object, multiparams, params)
../../../miniconda3/envs/data_warehouse/lib/python3.6/site-packages/sqlalchemy/engine/base.py:1097: in _execute_text
statement, parameters
../../../miniconda3/envs/data_warehouse/lib/python3.6/site-packages/sqlalchemy/engine/base.py:1189: in _execute_context
context)
../../../miniconda3/envs/data_warehouse/lib/python3.6/site-packages/sqlalchemy/engine/base.py:1402: in _handle_dbapi_exception
exc_info
../../../miniconda3/envs/data_warehouse/lib/python3.6/site-packages/sqlalchemy/util/compat.py:203: in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
../../../miniconda3/envs/data_warehouse/lib/python3.6/site-packages/sqlalchemy/util/compat.py:186: in reraise
raise value.with_traceback(tb)
../../../miniconda3/envs/data_warehouse/lib/python3.6/site-packages/sqlalchemy/engine/base.py:1182: in _execute_context
context)
../../../miniconda3/envs/data_warehouse/lib/python3.6/site-packages/sqlalchemy/engine/default.py:470: in do_execute
cursor.execute(statement, parameters)
../../../miniconda3/envs/data_warehouse/lib/python3.6/site-packages/snowflake/connector/cursor.py:526: in execute
errvalue)
../../../miniconda3/envs/data_warehouse/lib/python3.6/site-packages/snowflake/connector/errors.py:97: in errorhandler_wrapper
cursor.errorhandler(connection, cursor, errorclass, errorvalue)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
connection = <snowflake.connector.connection.SnowflakeConnection object at 0x110497940>
cursor = <snowflake.connector.cursor.SnowflakeCursor object at 0x10ef87a90>
errorclass = <class 'snowflake.connector.errors.ProgrammingError'>
errorvalue = {'done_format_msg': False, 'errno': 100074, 'msg': 'User character length limit (30) exceeded by string \'"EVERETT COM...ding options, please run \'info loading_data\' in a SQL client.', 'sfqid': '08f751b4-3520-41e8-b0c7-8a5a20645315', ...}
@staticmethod
def default_errorhandler(connection, cursor, errorclass, errorvalue):
u"""
Default error handler that raises an error
"""
raise errorclass(
msg=errorvalue.get(u'msg'),
errno=errorvalue.get(u'errno'),
sqlstate=errorvalue.get(u'sqlstate'),
sfqid=errorvalue.get(u'sfqid'),
> done_format_msg=errorvalue.get(u'done_format_msg'))
E sqlalchemy.exc.ProgrammingError: (snowflake.connector.errors.ProgrammingError) 100074 (54000): User character length limit (30) exceeded by string '"EVERETT COMMERCE CENTER ""A"""'
E File '@TEST_BLDG/BLDG.csv.gz', line 762, character 9
E Row 762, column "TEST_BLDG"[" BLDGNAME ":2]
E If you would like to continue loading when an error is encountered, use other values such as 'SKIP_FILE' or 'CONTINUE' for the ON_ERROR option. For more information on loading options, please run 'info loading_data' in a SQL client. [SQL: 'copy into "test_BLDG"']
../../../miniconda3/envs/data_warehouse/lib/python3.6/site-packages/snowflake/connector/errors.py:73: ProgrammingError
Example csv data that causes problem:
BLDGID , BLDGNAME , CITY , STATE , ZIPCODE , LLRDID , PAYTO , BLDGGLA , SDINTRST , ADDRESS1 , ADDRESS2 , PHONENO , BILLDATE , NMBRUNIT , MNGRID , MGRSTDAT , MGREXDAT , PRTSTAT , DAYDUE , DELQDAY , DELQOPT , LATEFLAT , LATERATE , LATECALC , MGMTRATE , NCFLAG , LASTDATE , USERID , INACTIVE , ICASHTYP , DCASHTYP , CPIINDEX , PTWID , ENTITYID , ALTADDR1 , ALTADDR2 , ALTCITY , ALTSTATE , ALTZIP , CLOSEDAY , FEDIDTYP , FEEMETH , FLATAMT , MGMTACC , MGMTCASH , REFNDACC , REFNDCAS , SECURACC , SECURCAS , COUNTY , ALTCOUNT , PROCDATE , CMINTRAC , OCCSUITE , OCCGLA , GLDETAIL , STARTINV , FASB13 , SQFTTYPE , FASBSTRT , PROFILE , WOINCCAT , MARKUP , MINCCAT , TICKFORM , SCANCODEID , RPTYPRI1 , RPTYPRI2 , RPTYPRI3 , RPTYPRI4 , RPTYPRI5 , BILLEVEL , ESTLEVEL , RECLEVEL , NATURAL , NINCCAT1 , NINCCAT2 , NINCCAT3 , ESTMETH , BILLPNT , CALCEST , REQRPTY , FORETYPE , PERCCHNG , RETAIL , NRECID , BASIS , MGMTNONT , MGMTSECAPP , MGMTCRAPP , MGMTVENDID , LTFEEDAYS , LTFCAMT , RANGETYPE , MGMTDEPT , MGMTRTAXGRP , MGMTBILLFLAT , MERCHANTID , MERCHANTPASSWORD , RECPTTYPEID , BAPPLYBYPRIORITY , MARKET , NUMBLDGS , YRBUILT , PROPTYPE , PROPID , ACQYIELD , TARGETNOI , TARGETDATE , EXITCAPRATE , YR5BASEIRR , YR7BASEIRR , YR5UPSIDEIRR , YR7UPSIDEIRR , YR5DOWNSIDEIRR , YR7DOWNSIDEIRR , NEXTAPPRAISALDATE , SELLER , HASFlashReport , URL , BLDGMATRIXOPTION , MATRIXID , BASEUOM , ALTERNATEUOM , USEALTERNATEUOM , MAP , SRVCCRED , SRVCDEBT , ALLOWPARTPMNT , ALLOWCREDIT , ALLOWCREDITAPPLY , STATEID
R3ECC ,"EVERETT COMMERCE CENTER ""A""", Renton , WA , 98057 , UNENCU , LBA Realty Fund III-Company VI ,91046.00,, LBA Realty , 660 S.W. 39th St. Suite 255 , (425) 272-0260 ,2011-10-01T00:00:00,1, ZSOLD ,,,True,True,, A ,,,,1.08,True,2015-10-26T16:13:00, CPLOSKER ,True, BA , BA ,,False, R3ECC , LBA Realty Fund III LP , PO Box 745805 , Los Angeles , CA , 900745805 , EOM ,1, C ,945.46, MR450100 , BA , MR110100 , BA , MR207100 , BA ,,,2011-09-20T00:00:00, MR111633 ,0,0.00, B ,1,False, GLA ,1998-01-01T00:00:00,,,,,,,,,,,,,,,False,,,,, C ,True,,,,False,, B ,False,False,False, REA ,365,0.00, P , @ ,,False,,, CHK ,,,,,,,,,,,,,,,,,,,,,False,,,,False,,,,,,,
Note, the second field in row 2 is really just the escaped quoted version of this value:
EVERETT COMMERCE CENTER "A"
The schema definition I have is as follows (note, the 2nd field's definition is VARCHAR(30) which fits the originating data length which is less than 30 characters without the escaped quotes being counted as part of length of field:
CREATE TABLE "test_BLDG" (
"BLDGID " VARCHAR(7) NOT NULL,
" BLDGNAME " VARCHAR(30),
" CITY " VARCHAR(19),
" STATE " VARCHAR(5),
" ZIPCODE " VARCHAR(11),
" LLRDID " VARCHAR(8) NOT NULL,
" PAYTO " VARCHAR(34),
" BLDGGLA " DECIMAL,
" SDINTRST " DECIMAL,
" ADDRESS1 " VARCHAR(33),
" ADDRESS2 " VARCHAR(32),
" PHONENO " VARCHAR(17),
" BILLDATE " TIMESTAMP,
" NMBRUNIT " DECIMAL,
" MNGRID " VARCHAR(8) NOT NULL,
" MGRSTDAT " TIMESTAMP,
" MGREXDAT " TIMESTAMP,
" PRTSTAT " BOOLEAN NOT NULL,
" DAYDUE " BOOLEAN NOT NULL,
" DELQDAY " DECIMAL,
" DELQOPT " VARCHAR(3) NOT NULL,
" LATEFLAT " DECIMAL,
" LATERATE " DECIMAL,
" LATECALC " VARCHAR(3),
" MGMTRATE " DECIMAL,
" NCFLAG " BOOLEAN NOT NULL,
" LASTDATE " TIMESTAMP,
" USERID " VARCHAR(22) NOT NULL,
" INACTIVE " BOOLEAN NOT NULL,
" ICASHTYP " VARCHAR(4),
" DCASHTYP " VARCHAR(4),
" CPIINDEX " BOOLEAN,
" PTWID " BOOLEAN,
" ENTITYID " VARCHAR(8) NOT NULL,
" ALTADDR1 " VARCHAR(34),
" ALTADDR2 " VARCHAR(32),
" ALTCITY " VARCHAR(19),
" ALTSTATE " VARCHAR(5),
" ALTZIP " VARCHAR(11),
" CLOSEDAY " VARCHAR(5) NOT NULL,
" FEDIDTYP " DECIMAL NOT NULL,
" FEEMETH " VARCHAR(3) NOT NULL,
" FLATAMT " DECIMAL,
" MGMTACC " VARCHAR(14),
" MGMTCASH " VARCHAR(4),
" REFNDACC " VARCHAR(14),
" REFNDCAS " VARCHAR(4),
" SECURACC " VARCHAR(14),
" SECURCAS " VARCHAR(4),
" COUNTY " VARCHAR(17),
" ALTCOUNT " VARCHAR,
" PROCDATE " TIMESTAMP,
" CMINTRAC " VARCHAR(14),
" OCCSUITE " DECIMAL,
" OCCGLA " DECIMAL,
" GLDETAIL " VARCHAR(3) NOT NULL,
" STARTINV " DECIMAL,
" FASB13 " BOOLEAN NOT NULL,
" SQFTTYPE " VARCHAR(5),
" FASBSTRT " TIMESTAMP,
" PROFILE " BOOLEAN,
" WOINCCAT " BOOLEAN,
" MARKUP " DECIMAL,
" MINCCAT " BOOLEAN,
" TICKFORM " BOOLEAN,
" SCANCODEID " BOOLEAN,
" RPTYPRI1 " BOOLEAN,
" RPTYPRI2 " BOOLEAN,
" RPTYPRI3 " BOOLEAN,
" RPTYPRI4 " BOOLEAN,
" RPTYPRI5 " BOOLEAN,
" BILLEVEL " BOOLEAN,
" ESTLEVEL " BOOLEAN,
" RECLEVEL " BOOLEAN,
" NATURAL " BOOLEAN NOT NULL,
" NINCCAT1 " BOOLEAN,
" NINCCAT2 " BOOLEAN,
" NINCCAT3 " BOOLEAN,
" ESTMETH " BOOLEAN,
" BILLPNT " VARCHAR(3) NOT NULL,
" CALCEST " BOOLEAN NOT NULL,
" REQRPTY " BOOLEAN,
" FORETYPE " BOOLEAN,
" PERCCHNG " DECIMAL,
" RETAIL " BOOLEAN,
" NRECID " BOOLEAN,
" BASIS " VARCHAR(3),
" MGMTNONT " BOOLEAN NOT NULL,
" MGMTSECAPP " BOOLEAN NOT NULL,
" MGMTCRAPP " BOOLEAN NOT NULL,
" MGMTVENDID " VARCHAR(8),
" LTFEEDAYS " DECIMAL,
" LTFCAMT " DECIMAL,
" RANGETYPE " VARCHAR(3) NOT NULL,
" MGMTDEPT " VARCHAR(3) NOT NULL,
" MGMTRTAXGRP " BOOLEAN,
" MGMTBILLFLAT " BOOLEAN NOT NULL,
" MERCHANTID " BOOLEAN,
" MERCHANTPASSWORD " BOOLEAN,
" RECPTTYPEID " VARCHAR(5),
" BAPPLYBYPRIORITY " BOOLEAN,
" MARKET " BOOLEAN,
" NUMBLDGS " BOOLEAN,
" YRBUILT " DECIMAL,
" PROPTYPE " BOOLEAN,
" PROPID " BOOLEAN,
" ACQYIELD " DECIMAL,
" TARGETNOI " DECIMAL,
" TARGETDATE " TIMESTAMP,
" EXITCAPRATE " DECIMAL,
" YR5BASEIRR " DECIMAL,
" YR7BASEIRR " DECIMAL,
" YR5UPSIDEIRR " DECIMAL,
" YR7UPSIDEIRR " DECIMAL,
" YR5DOWNSIDEIRR " DECIMAL,
" YR7DOWNSIDEIRR " DECIMAL,
" NEXTAPPRAISALDATE " TIMESTAMP,
" SELLER " VARCHAR(17),
" HASFlashReport " BOOLEAN,
" URL " BOOLEAN,
" BLDGMATRIXOPTION " BOOLEAN,
" MATRIXID " BOOLEAN,
" BASEUOM " BOOLEAN,
" ALTERNATEUOM " BOOLEAN,
" USEALTERNATEUOM " BOOLEAN,
" MAP " BOOLEAN,
" SRVCCRED " BOOLEAN,
" SRVCDEBT " BOOLEAN,
" ALLOWPARTPMNT " BOOLEAN,
" ALLOWCREDIT " BOOLEAN,
" ALLOWCREDITAPPLY " BOOLEAN,
" STATEID" VARCHAR(21)
)
Conda list of packages I have installed (using Snowflake Connector version 1.4.10):
$ conda list
# packages in environment at /Users/jdfagan/miniconda3/envs/data_warehouse:
#
agate 1.6.0 py36_2 conda-forge
agate-dbf 0.2.0 py36_0 conda-forge
agate-excel 0.2.1 py36_0 conda-forge
agate-sql 0.5.2 py36_0 conda-forge
asn1crypto 0.23.0 <pip>
asn1crypto 0.22.0 py36_0 conda-forge
azure-common 1.1.8 <pip>
azure-nspkg 2.0.0 <pip>
azure-storage 0.34.2 <pip>
babel 2.5.1 py36_0 conda-forge
boto3 1.4.7 <pip>
botocore 1.7.39 <pip>
ca-certificates 2017.7.27.1 0 conda-forge
certifi 2017.7.27.1 py36_0 conda-forge
cffi 1.11.2 <pip>
cffi 1.10.0 py36_0 conda-forge
chardet 3.0.4 py36_0 conda-forge
chardet 3.0.4 <pip>
click 6.7 py36_0 conda-forge
coverage 4.4.1 py36_0 conda-forge
cryptography 2.0.3 <pip>
cryptography 2.0.3 py36_0 conda-forge
cryptography 2.1.3 <pip>
dbfread 2.0.7 py36_0 conda-forge
docutils 0.14 <pip>
et_xmlfile 1.0.1 py36_0 conda-forge
future 0.16.0 <pip>
hvac 0.2.17 <pip>
idna 2.6 <pip>
idna 2.6 py36_1 conda-forge
ijson 2.3 <pip>
isodate 0.6.0 py_0 conda-forge
jdcal 1.3 py36_0 conda-forge
jmespath 0.9.3 <pip>
leather 0.3.3 py36_0 conda-forge
libffi 3.2.1 3 conda-forge
mysql-connector-python 2.2.2 py36_0 conda-forge
ncurses 5.9 10 conda-forge
openpyxl 2.5.0b1 py36_0 conda-forge
openssl 1.0.2l 0 conda-forge
parsedatetime 2.4 py36_0 conda-forge
pip 9.0.1 py36_0 conda-forge
py 1.4.34 py36_0 conda-forge
pyasn1 0.3.7 <pip>
pyasn1-modules 0.1.5 <pip>
pycparser 2.18 py36_0 conda-forge
pycparser 2.18 <pip>
pycryptodome 3.4.7 <pip>
pymysql 0.7.11 py_0 conda-forge
pyOpenSSL 17.3.0 <pip>
pyopenssl 17.2.0 py36_0 conda-forge
pysocks 1.6.7 py36_0 conda-forge
pytest 3.2.3 py36_0 conda-forge
pytest-cov 2.5.1 py36_0 conda-forge
python 3.6.3 1 conda-forge
python-dateutil 2.6.1 <pip>
python-slugify 1.2.4 py36_1 conda-forge
pytimeparse 1.1.7 py36_0 conda-forge
pytz 2017.3 <pip>
pytz 2017.3 py_1 conda-forge
readline 6.2 0 conda-forge
requests 2.18.4 py36_1 conda-forge
requests 2.18.4 <pip>
requests-toolbelt 0.8.0 py36_0 conda-forge
s3transfer 0.1.11 <pip>
setuptools 36.6.0 py36_1 conda-forge
six 1.11.0 py36_1 conda-forge
six 1.11.0 <pip>
snowflake-connector-python 1.4.10 <pip>
snowflake-sqlalchemy 1.0.7 <pip>
SQLAlchemy 1.1.13 <pip>
sqlalchemy 1.1.13 py36_0 conda-forge
sqlite 3.13.0 1 conda-forge
tk 8.5.19 2 conda-forge
unidecode 0.04.21 py36_0 conda-forge
urllib3 1.22 <pip>
urllib3 1.22 py36_0 conda-forge
wheel 0.30.0 py_1 conda-forge
xlrd 1.1.0 py36_1 conda-forge
xz 5.2.3 0 conda-forge
zlib 1.2.11 0 conda-forge
pyasn1 v0.3.1 was released today, and it changes the constructor APIs of some underlying types:
https://github.com/etingof/pyasn1/blob/v0.3.1/CHANGES.rst
This makes it incompatible with pyasn-modules v0.0.9. Locking pyasn1 to v0.2.3 seems to fix the issue for me.
Stack trace with pyasn1==0.3.1:
File "/usr/src/app/.tox/integration-snowflake-py36/lib/python3.6/site-packages/snowflake/connector/__init__.py", line 21, in <module>
from .connection import SnowflakeConnection
File "/usr/src/app/.tox/integration-snowflake-py36/lib/python3.6/site-packages/snowflake/connector/connection.py", line 14, in <module>
from . import network
File "/usr/src/app/.tox/integration-snowflake-py36/lib/python3.6/site-packages/snowflake/connector/network.py", line 28, in <module>
from . import ssl_wrap_socket
File "/usr/src/app/.tox/integration-snowflake-py36/lib/python3.6/site-packages/snowflake/connector/ssl_wrap_socket.py", line 103, in <module>
from .ocsp_pyopenssl import SnowflakeOCSP
File "/usr/src/app/.tox/integration-snowflake-py36/lib/python3.6/site-packages/snowflake/connector/ocsp_pyopenssl.py", line 37, in <module>
from pyasn1_modules import (rfc2459, rfc2437, rfc2560)
File "/usr/src/app/.tox/integration-snowflake-py36/lib/python3.6/site-packages/pyasn1_modules/rfc2560.py", line 137, in <module>
class ResponseData(univ.Sequence):
File "/usr/src/app/.tox/integration-snowflake-py36/lib/python3.6/site-packages/pyasn1_modules/rfc2560.py", line 143, in ResponseData
namedtype.NamedType('responses', univ.SequenceOf(SingleResponse())),
TypeError: __init__() takes 1 positional argument but 2 were given
The Snowflake connector for python seems to be implemented essentially as API calls over HTTP. Using aiohttp, companion subclasses to SnowflakeConnector
, SnowflakeCursor
, SnowflakeRestful
etc, could be created that implement the key methods as asynchronous coroutines. Then asyncio tools could be used to run Snowflake connection routines alongside other I/O-centric or API-driven tasks.
Has this been considered? Is it a viable addition to the Snowflake Connector? If so I'm happy to contribute, I'd love to hear any requirements you folks might have in mind. Or on the other hand, is it more appropriate as a fork, or as a separate project altogether in the style of aiobotocore?
The current install_requires
in setup.py
look to be inconsistent because of the boto3/botocore requirements.
I bring this up because it makes this package incompatible with pipenv which we use to lock our python requirements for production. The lock fails because it can't find a botocore version that satisfies all listed requirements.
I tested with a forked version of this repo and just removed the botocore requirement entirely, which results in it using botocore==1.9.0
. All of our snowflake tests passed, although I'm not sure if I'm missing a rare case that this requirement was supposed to address.
For reference here's the full dependency graph I'm getting from pipenv, where you can see the conflict:
> pipenv install snowflake-connector-python --skip-lock
> pipenv graph
snowflake-connector-python==1.5.1
- azure-common [required: <2, installed: 1.1.8]
- azure-nspkg [required: >=2.0.0, installed: 2.0.0]
- azure-nspkg [required: <3, installed: 2.0.0]
- azure-storage [required: <=0.34.2, installed: 0.34.2]
- azure-common [required: >=1.1.5, installed: 1.1.8]
- azure-nspkg [required: >=2.0.0, installed: 2.0.0]
- azure-nspkg [required: >=2.0.0, installed: 2.0.0]
- cryptography [required: Any, installed: 2.1.4]
- asn1crypto [required: >=0.21.0, installed: 0.24.0]
- cffi [required: >=1.7, installed: 1.11.5]
- pycparser [required: Any, installed: 2.18]
- enum34 [required: Any, installed: 1.1.6]
- idna [required: >=2.1, installed: 2.6]
- ipaddress [required: Any, installed: 1.0.19]
- six [required: >=1.4.1, installed: 1.11.0]
- python-dateutil [required: Any, installed: 2.6.1]
- six [required: >=1.5, installed: 1.11.0]
- requests [required: Any, installed: 2.18.4]
- certifi [required: >=2017.4.17, installed: 2018.1.18]
- chardet [required: >=3.0.2,<3.1.0, installed: 3.0.4]
- idna [required: >=2.5,<2.7, installed: 2.6]
- urllib3 [required: <1.23,>=1.21.1, installed: 1.22]
- boto3 [required: <=1.6.0,>=1.4.4, installed: 1.6.0]
- botocore [required: <1.10.0,>=1.9.0, installed: 1.8.50]
- docutils [required: >=0.10, installed: 0.14]
- jmespath [required: <1.0.0,>=0.7.1, installed: 0.9.3]
- python-dateutil [required: <3.0.0,>=2.1, installed: 2.6.1]
- six [required: >=1.5, installed: 1.11.0]
- jmespath [required: <1.0.0,>=0.7.1, installed: 0.9.3]
- s3transfer [required: <0.2.0,>=0.1.10, installed: 0.1.13]
- botocore [required: <2.0.0,>=1.3.0, installed: 1.8.50]
- docutils [required: >=0.10, installed: 0.14]
- jmespath [required: <1.0.0,>=0.7.1, installed: 0.9.3]
- python-dateutil [required: <3.0.0,>=2.1, installed: 2.6.1]
- six [required: >=1.5, installed: 1.11.0]
- futures [required: <4.0.0,>=2.2.0, installed: 3.2.0]
- botocore [required: <1.9.0,>=1.5.0, installed: 1.8.50]
- docutils [required: >=0.10, installed: 0.14]
- jmespath [required: <1.0.0,>=0.7.1, installed: 0.9.3]
- python-dateutil [required: <3.0.0,>=2.1, installed: 2.6.1]
- six [required: >=1.5, installed: 1.11.0]
- certifi [required: Any, installed: 2018.1.18]
- cffi [required: >=1.9, installed: 1.11.5]
- pycparser [required: Any, installed: 2.18]
- cryptography [required: >=1.8.2,<2.2, installed: 2.1.4]
- asn1crypto [required: >=0.21.0, installed: 0.24.0]
- cffi [required: >=1.7, installed: 1.11.5]
- pycparser [required: Any, installed: 2.18]
- enum34 [required: Any, installed: 1.1.6]
- idna [required: >=2.1, installed: 2.6]
- ipaddress [required: Any, installed: 1.0.19]
- six [required: >=1.4.1, installed: 1.11.0]
- future [required: Any, installed: 0.16.0]
- ijson [required: Any, installed: 2.3]
- pycryptodome [required: >=3.2, installed: 3.4.11]
- pyjwt [required: Any, installed: 1.5.3]
- pyOpenSSL [required: >=16.2.0,<18.0.0, installed: 17.5.0]
- cryptography [required: >=2.1.4, installed: 2.1.4]
- asn1crypto [required: >=0.21.0, installed: 0.24.0]
- cffi [required: >=1.7, installed: 1.11.5]
- pycparser [required: Any, installed: 2.18]
- enum34 [required: Any, installed: 1.1.6]
- idna [required: >=2.1, installed: 2.6]
- ipaddress [required: Any, installed: 1.0.19]
- six [required: >=1.4.1, installed: 1.11.0]
- six [required: >=1.5.2, installed: 1.11.0]
- pytz [required: Any, installed: 2018.3]
- six [required: Any, installed: 1.11.0]
The packages is installed to snowflake/connector
directory. However, the snowflake
directory does not contain the __init__.py
file.
Is there a reason why the library depends on cryptography < 1.8? I'm having problems installing an earlier version of cryptography with new versions of openssl, and therefore can't install this library.
Hi,
I'm using airflow to run some sql scripts. When I try to use query:
select *
from table
where column like '%abc":"def%'
(% is wildcard for string match)
I got an error: File "/usr/local/lib/python3.6/site-packages/snowflake/connector/cursor.py", line 454, in execute query = command % processed_params TypeError: not enough arguments for format string
I guess this was caused by some inner config in cursor.py since I could run it on snowflake.
I finally fixed the problem by replacing the first '%' with the exact string. However, that's not a global solution when you do need a wildcard at the beginning of the string.
Could you take a look at this problem and fix it for the future convenience?
Thanks!
Hi smtakeda
i made a script for loading the data from sql server to snowflake we have tried to load 1.5millions to record into snowflake and that worked fine for me but it took more than half hour to perform it and load the data in chunks i.e chunksize= 15000 i believe it snowflake limitation is 16000 .
so i want to load 50 millions of data into snowflake with python script
my question is how we can increase the chunksize of snowflake and and how can increase the performance .
please suggest?
below is the code
import pandas as pd
from sqlalchemy import *
import snowflake.connector
try:
# ms Sql Connnection
mssql_engine = create_engi@localhost:1433/testdb')
conn = mssql_engine.connect()
print "Connected To Ms SQl using Sql Alchemy..."
# Snowflake connector
snowflake_conn =
snowflake.connector.connect(user='account=',)
snowflake_cursor = snowflake_conn.cursor()
print "Connected to Snowflake..."
if snowflake_cursor:
snowflake_cursor.execute("CREATE DATABASE IF NOT EXISTS ms_to_sql_testdb")
snowflake_cursor.execute("USE DATABASE ms_to_sql_testdb")
snowflake_cursor.execute("CREATE SCHEMA IF NOT EXISTS ms_to_sql_testdb_schema")
snowflake_cursor.execute("USE SCHEMA ms_to_sql_testdb_schema")
# Snow Flake SQL Alchemy connection
snw_flk_engine = create_engine(
'snowflake://{user}:{password}@{account}/{db}'.format(user,account='decisionminds', db='ms_to_sql_testdb'))
snw_flk_conn = snw_flk_engine.connect()
print "Connected to SQL Alchemy..."
# for loop to fetch dataframe
if snw_flk_conn:
count = 1
for data in pd.read_sql_query('SELECT * FROM onelakhrecords', conn,chunksize=16000):
results = data.to_sql("onelakhrecords", snw_flk_conn, if_exists='append',index=False,index_label=None, schema='ms_to_sql_testdb_schema')
print str(count) + ' Chunk Uploaded...'
count += 1
print 'data uploaded successfully'
else:
print "Snow Flake Connection error"
else:
print "An error occured while inserting a table"
except Exception as error:
print error
print "Connection error"
finally:
snowflake_cursor.close()
conn.close()
snw_flk_conn.close()
@smtakeda
Collaborator
smtakeda commented 10 minutes ago
@RiteshM1993 the limitation came from the size of SQL, e.g., INSERT INTO followed by all data in a single statement implemented in executemany
can you try the connection parameter paramstyle=qmark? It is undocumented, but I believe it is worth to try. Binding data happens in the server end instead of composing a sql in the clients
If any issue is found, please file a new issue. This is issue was already closed, so it would be easier for us to track the open issue separately.
@RiteshM1993
Write
Will keep eyes on the Python community and decide when we'll do.
Any comment is welcome here.
Not sure if this is a documentation issue or something that could be handled in code (or maybe it's just my issue), but I was having issues with a connection object not being closed. Version 1.4.9 of the connector:
import snowflake.connector
conn = snowflake.connector.connect(
user='some_user',
password='some_pwd',
account='some_account'
)
cursor = conn.cursor()
cursor.execute('USE WAREHOUSE some_warehouse;')
cursor.execute('USE DATABASE some_db;')
cursor.execute(some_sql)
for row in cursor:
#do something
cursor.close()
If I don't close the connection at the end (conn.close()
) then my code will never finish - it just sits there with an idle connection. If users are expected to manually close the connection I think it could be a bit clearer in the docs. Or else maybe there's a way to automatically close the connection or switch to a with
structure?
As we bumped up the upper bound of boto3 and botocore, it appeared PUT and GET command won't be able to renew tokens if it takes more than 2 hours.
Workaround is downgrade boto3 and botocore
pip install -U botocore==1.8.50 boto3==1.5.36
snowflake-connector-python/converter.py
Line 155 in 6c3e666
Hello,
those dynamically-defined classes are not pickleable because they are not attached to the module:
PicklingError: Can't pickle <class 'snowflake.connector.converter.GMTN0400'>: attribute lookup snowflake.connector.converter.GMTN0400 failed
I wonder if we can cache them in the module's namespace so that they can be accessed and pickled.
Python Version: 2.7
Snowflake connector version: 1.6.3
Recently we've been getting this error when running snowflake connector from freshly built docker image:
File "/usr/local/lib/python2.7/site-packages/snowflake/connector/__init__.py", line 21, in <module>
from .connection import SnowflakeConnection
File "/usr/local/lib/python2.7/site-packages/snowflake/connector/connection.py", line 19, in <module>
from .cursor import SnowflakeCursor
File "/usr/local/lib/python2.7/site-packages/snowflake/connector/cursor.py", line 29, in <module>
from .file_transfer_agent import (SnowflakeFileTransferAgent)
File "/usr/local/lib/python2.7/site-packages/snowflake/connector/file_transfer_agent.py", line 31, in <module>
from .s3_util import (SnowflakeS3FileEncryptionMaterial, SnowflakeS3Util,
File "/usr/local/lib/python2.7/site-packages/snowflake/connector/s3_util.py", line 25, in <module>
from Crypto.Cipher import AES
File "/usr/local/lib/python2.7/site-packages/Crypto/Cipher/__init__.py", line 36, in <module>
from Crypto.Cipher._mode_gcm import _create_gcm_cipher
File "/usr/local/lib/python2.7/site-packages/Crypto/Cipher/_mode_gcm.py", line 47, in <module>
from Crypto.Util import _cpuid
ImportError: dynamic module does not define init function (init_cpuid)
This seems to be caused by a changes to pycryptodome as pinning the installed version to 3.6.2 fixes the issue.
code.py
import snowflake.connector as sf_lib
with sf_lib.connect(**sf_details) as conn:
with conn.cursor(sf_lib.DictCursor) as cursor:
***do db stuff***
pylint error
No name 'connector' in module 'snowflake'
Metadata-Version: 2.0
Name: snowflake-connector-python
Version: 1.4.9
Summary: Snowflake Connector for Python
Home-page: https://www.snowflake.net/
It is quite common to use awscli, especially if sync between buckets is needed, but if you install the latest version of awscli, botocore version is upgraded to botocore 1.8.42, and when trying to query snowflake using sqlalchemy, the following error rises:
File "/some/test/virtualenv/path/python2.7/site-packages/sqlalchemy/engine/__init__.py", line 387, in create_engine return strategy.create(*args, **kwargs) File "/some/test/virtualenv/path/python2.7/site-packages/sqlalchemy/engine/strategies.py", line 56, in create entrypoint = u._get_entrypoint() File "/some/test/virtualenv/path/python2.7/site-packages/sqlalchemy/engine/url.py", line 139, in _get_entrypoint cls = registry.load(name) File "/some/test/virtualenv/path/python2.7/site-packages/sqlalchemy/util/langhelpers.py", line 214, in load return impl.load() File "/some/test/virtualenv/path/python2.7/site-packages/pkg_resources/__init__.py", line 2407, in load self.require(*args, **kwargs) File "/some/test/virtualenv/path/python2.7/site-packages/pkg_resources/__init__.py", line 2430, in require items = working_set.resolve(reqs, env, installer, extras=self.extras) File "/some/test/virtualenv/path/python2.7/site-packages/pkg_resources/__init__.py", line 875, in resolve raise VersionConflict(dist, req).with_context(dependent_req) pkg_resources.ContextualVersionConflict: (botocore 1.8.42 (/some/test/virtualenv/path/python2.7/site-packages), Requirement.parse('botocore<1.8.0,>=1.7.0'), set(['boto3']))
Is it really necessary to set an upper limit for bocore versions?
Could it be possible to allow the usage of the latest botocore version?
The snowflake documentation on the PUT command claims to accept an argument of SOURCE_COMPRESSION = AUTO
. This is not the case. The value the python connector expects is auto_detect
. Using source_compression = auto_detect
in a PUT command, works. Using source_compression = AUTO
returns an error. Somewhere in the cursor object, the request is getting sent to snowflake (cursor._connection.cmd_query
), where it sets the ret
return value.
As both the python client, and the java client have always expected AUTO_DETECT
, something must have changed on snowflake's backend.
https://github.com/snowflakedb/snowflake-connector-python/blame/master/file_transfer_agent.py#L801
On OS X, running fine for the last week with the version v1.6.3
of snowflake-connector-python
in python 3.6 until this morning I started getting this error:
File "bin/repl-scraping-tables.py", line 31, in <module>
replicate()
File "bin/repl-scraping-tables.py", line 26, in replicate
_export_mappings()
File "bin/repl-scraping-tables.py", line 20, in _export_mappings
exec_store_wait(sql, output_location)
File "/Users/samuelhopkins/datamart/etl/dbs.py", line 103, in exec_store_wait
'OutputLocation': output_location,
File "/Users/samuelhopkins/.virtualenvs/datamart/lib/python3.6/site-packages/botocore/client.py", line 314, in _api_call
return self._make_api_call(operation_name, kwargs)
File "/Users/samuelhopkins/.virtualenvs/datamart/lib/python3.6/site-packages/botocore/client.py", line 599, in _make_api_call
operation_model, request_dict)
File "/Users/samuelhopkins/.virtualenvs/datamart/lib/python3.6/site-packages/botocore/endpoint.py", line 148, in make_request
return self._send_request(request_dict, operation_model)
File "/Users/samuelhopkins/.virtualenvs/datamart/lib/python3.6/site-packages/botocore/endpoint.py", line 177, in _send_request
success_response, exception):
File "/Users/samuelhopkins/.virtualenvs/datamart/lib/python3.6/site-packages/botocore/endpoint.py", line 273, in _needs_retry
caught_exception=caught_exception, request_dict=request_dict)
File "/Users/samuelhopkins/.virtualenvs/datamart/lib/python3.6/site-packages/botocore/hooks.py", line 227, in emit
return self._emit(event_name, kwargs)
File "/Users/samuelhopkins/.virtualenvs/datamart/lib/python3.6/site-packages/botocore/hooks.py", line 210, in _emit
response = handler(**kwargs)
File "/Users/samuelhopkins/.virtualenvs/datamart/lib/python3.6/site-packages/botocore/retryhandler.py", line 183, in __call__
if self._checker(attempts, response, caught_exception):
File "/Users/samuelhopkins/.virtualenvs/datamart/lib/python3.6/site-packages/botocore/retryhandler.py", line 251, in __call__
caught_exception)
File "/Users/samuelhopkins/.virtualenvs/datamart/lib/python3.6/site-packages/botocore/retryhandler.py", line 269, in _should_retry
return self._checker(attempt_number, response, caught_exception)
File "/Users/samuelhopkins/.virtualenvs/datamart/lib/python3.6/site-packages/botocore/retryhandler.py", line 317, in __call__
caught_exception)
File "/Users/samuelhopkins/.virtualenvs/datamart/lib/python3.6/site-packages/botocore/retryhandler.py", line 223, in __call__
attempt_number, caught_exception)
File "/Users/samuelhopkins/.virtualenvs/datamart/lib/python3.6/site-packages/botocore/retryhandler.py", line 359, in _check_caught_exception
raise caught_exception
File "/Users/samuelhopkins/.virtualenvs/datamart/lib/python3.6/site-packages/botocore/endpoint.py", line 222, in _get_response
proxies=self.proxies, timeout=self.timeout)
File "/Users/samuelhopkins/.virtualenvs/datamart/lib/python3.6/site-packages/botocore/vendored/requests/sessions.py", line 573, in send
r = adapter.send(request, **kwargs)
File "/Users/samuelhopkins/.virtualenvs/datamart/lib/python3.6/site-packages/botocore/vendored/requests/adapters.py", line 370, in send
timeout=timeout
File "/Users/samuelhopkins/.virtualenvs/datamart/lib/python3.6/site-packages/botocore/vendored/requests/packages/urllib3/connectionpool.py", line 544, in urlopen
body=body, headers=headers)
File "/Users/samuelhopkins/.virtualenvs/datamart/lib/python3.6/site-packages/botocore/vendored/requests/packages/urllib3/connectionpool.py", line 341, in _make_request
self._validate_conn(conn)
File "/Users/samuelhopkins/.virtualenvs/datamart/lib/python3.6/site-packages/botocore/vendored/requests/packages/urllib3/connectionpool.py", line 761, in _validate_conn
conn.connect()
File "/Users/samuelhopkins/.virtualenvs/datamart/lib/python3.6/site-packages/botocore/vendored/requests/packages/urllib3/connection.py", line 238, in connect
ssl_version=resolved_ssl_version)
File "/Users/samuelhopkins/.virtualenvs/datamart/lib/python3.6/site-packages/snowflake/connector/ssl_wrap_socket.py", line 397, in ssl_wrap_socket_with_ocsp
).validate(server_hostname, ret.connection)
File "/Users/samuelhopkins/.virtualenvs/datamart/lib/python3.6/site-packages/snowflake/connector/ocsp_asn1crypto.py", line 997, in validate
return self._validate(hostname, cert_data)
File "/Users/samuelhopkins/.virtualenvs/datamart/lib/python3.6/site-packages/snowflake/connector/ocsp_asn1crypto.py", line 1006, in _validate
results = _validate_certificates_sequential(cert_data, do_retry)
File "/Users/samuelhopkins/.virtualenvs/datamart/lib/python3.6/site-packages/snowflake/connector/ocsp_asn1crypto.py", line 800, in _validate_certificates_sequential
r = validate_by_direct_connection(issuer, subject, do_retry)
File "/Users/samuelhopkins/.virtualenvs/datamart/lib/python3.6/site-packages/snowflake/connector/ocsp_asn1crypto.py", line 724, in validate_by_direct_connection
raise err
File "/Users/samuelhopkins/.virtualenvs/datamart/lib/python3.6/site-packages/snowflake/connector/ocsp_asn1crypto.py", line 714, in validate_by_direct_connection
_process_ocsp_response(issuer, cert_id, ocsp_response)
File "/Users/samuelhopkins/.virtualenvs/datamart/lib/python3.6/site-packages/snowflake/connector/ocsp_asn1crypto.py", line 554, in _process_ocsp_response
_process_good_status(single_response, cert_id, ocsp_response)
File "/Users/samuelhopkins/.virtualenvs/datamart/lib/python3.6/site-packages/snowflake/connector/ocsp_asn1crypto.py", line 467, in _process_good_status
errno=ER_INVALID_OCSP_RESPONSE)
snowflake.connector.errors.OperationalError: 254007: Response is unreliable. Its validity date is out of range: current_time=2018-06-21 08:50:33Z, this_update=2018-06-14 02:25:39Z, next_update=2018-06-21 01:40:39Z, tolerable next_update=2018-06-21 03:21:00Z. A potential cause is client clock is skewed, CA fails to update OCSP response in time.
It continues to occur for versions > 1.6.0, downgrading to 1.6.0 and everything runs swimmingly now.
pip freeze is as follows
asn1crypto==0.24.0
atomicwrites==1.1.5
attrs==18.1.0
azure-common==1.1.12
azure-nspkg==2.0.0
azure-storage==0.36.0
boto3==1.7.35
botocore==1.10.35
certifi==2018.4.16
cffi==1.11.5
chardet==3.0.4
cryptography==2.2.2
docutils==0.14
expects==0.8.0
future==0.16.0
idna==2.7
ijson==2.3
jmespath==0.9.3
more-itertools==4.2.0
pluggy==0.6.0
py==1.5.3
pyasn1==0.4.3
pyasn1-modules==0.2.1
PyAthena==1.2.5
pycparser==2.18
pycryptodome==3.6.1
PyJWT==1.6.4
pyOpenSSL==17.5.0
pytest==3.6.1
python-dateutil==2.7.3
pytz==2018.4
requests==2.19.1
s3fs==0.1.5
s3transfer==0.1.13
six==1.11.0
snowflake-connector-python==1.6.0
tenacity==4.12.0
urllib3==1.23
Initially, in this commit boto3 was pinned below 1.5.0
. Then in a subsequent commit it was reduced to <=1.4.7
. This threw some errors in Pipenv and I was curious what the limitation is.
See Legrandin/pycryptodome#142
/usr/bin/clang -fno-strict-aliasing -Wsign-compare -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -arch i386 -arch x86_64 -g -DHAVE_X86INTRIN_H -DPY_LITTLE_ENDIAN -DLTC_NO_ASM -DHAVE_CPUID_H -Isrc/ -I/private/tmp/a/include -I/Library/Frameworks/Python.framework/Versions/3.5/include/python3.5m -c src/multiply_64.c -o build/temp.macosx-10.6-intel-3.5/src/multiply_64.o -msse2
src/multiply_64.c:57:10: error: conflicting types for 'addmul128'
uint64_t addmul128(uint64_t * RESTRICT t, const uint64_t * RESTRICT a, uint64_t b0, uint64_t b1, size_t words)
^
src/multiply.h:62:8: note: previous declaration is here
size_t addmul128(uint64_t * RESTRICT t, const uint64_t * RESTRICT a, uint64_t b0, uint64_t b1, size_t words)
^
1 error generated.
error: command '/usr/bin/clang' failed with exit status 1
----------------------------------------
Command "/private/tmp/a/bin/python3.5 -u -c "import setuptools, tokenize;__file__='/private/var/folders/60/7z__dz2n0s710cbl21vxh6q00000gn/T/pip-build-a1dso6du/pycryptodome/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /var/folders/60/7z__dz2n0s710cbl21vxh6q00000gn/T/pip-x6gz6wi9-record/install-record.txt --single-version-externally-managed --compile --install-headers /private/tmp/a/include/site/python3.5/pycryptodome" failed with error code 1 in /private/var/folders/60/7z__dz2n0s710cbl21vxh6q00000gn/T/pip-build-a1dso6du/pycryptodome/
Hi,
The required botocore version is incompatible with Python 3.6. When attempting to use the snowflake connector, I get:
... trimmed ...
File "/Users/connor/.virtualenvs/hosted-test3/lib/python3.6/site-packages/snowflake_connector_python-1.3.7-py3.6.egg/snowflake/connector/__init__.py", line 32, in Connect
return SnowflakeConnection(**kwargs)
File "/Users/connor/.virtualenvs/hosted-test3/lib/python3.6/site-packages/snowflake_connector_python-1.3.7-py3.6.egg/snowflake/connector/connection.py", line 107, in __init__
self.connect(**kwargs)
File "/Users/connor/.virtualenvs/hosted-test3/lib/python3.6/site-packages/snowflake_connector_python-1.3.7-py3.6.egg/snowflake/connector/connection.py", line 235, in connect
'password_callback'))
File "/Users/connor/.virtualenvs/hosted-test3/lib/python3.6/site-packages/snowflake_connector_python-1.3.7-py3.6.egg/snowflake/connector/connection.py", line 440, in __open_connection
session_parameters=self._session_parameters,
File "/Users/connor/.virtualenvs/hosted-test3/lib/python3.6/site-packages/snowflake_connector_python-1.3.7-py3.6.egg/snowflake/connector/network.py", line 300, in authenticate
ret = self._post_request(url, headers, json.dumps(body))
File "/Users/connor/.virtualenvs/hosted-test3/lib/python3.6/site-packages/snowflake_connector_python-1.3.7-py3.6.egg/snowflake/connector/network.py", line 541, in _post_request
max_connection_pool=self._max_connection_pool)
File "/Users/connor/.virtualenvs/hosted-test3/lib/python3.6/site-packages/snowflake_connector_python-1.3.7-py3.6.egg/snowflake/connector/network.py", line 780, in access_url
ret = pool_result.get(timeout=request_timeout)
File "/usr/local/Cellar/python3/3.6.0/Frameworks/Python.framework/Versions/3.6/lib/python3.6/multiprocessing/pool.py", line 608, in get
raise self._value
File "/usr/local/Cellar/python3/3.6.0/Frameworks/Python.framework/Versions/3.6/lib/python3.6/multiprocessing/pool.py", line 119, in worker
result = (True, func(*args, **kwds))
File "/usr/local/Cellar/python3/3.6.0/Frameworks/Python.framework/Versions/3.6/lib/python3.6/multiprocessing/pool.py", line 44, in mapstar
return list(map(*args))
File "/Users/connor/.virtualenvs/hosted-test3/lib/python3.6/site-packages/snowflake_connector_python-1.3.7-py3.6.egg/snowflake/connector/network.py", line 659, in request_thread
auth=SnowflakeAuth(token),
File "/Users/connor/.virtualenvs/hosted-test3/lib/python3.6/site-packages/botocore-1.4.37-py3.6.egg/botocore/vendored/requests/sessions.py", line 465, in request
resp = self.send(prep, **send_kwargs)
File "/Users/connor/.virtualenvs/hosted-test3/lib/python3.6/site-packages/botocore-1.4.37-py3.6.egg/botocore/vendored/requests/sessions.py", line 573, in send
r = adapter.send(request, **kwargs)
File "/Users/connor/.virtualenvs/hosted-test3/lib/python3.6/site-packages/botocore-1.4.37-py3.6.egg/botocore/vendored/requests/adapters.py", line 370, in send
timeout=timeout
File "/Users/connor/.virtualenvs/hosted-test3/lib/python3.6/site-packages/botocore-1.4.37-py3.6.egg/botocore/vendored/requests/packages/urllib3/connectionpool.py", line 544, in urlopen
body=body, headers=headers)
File "/Users/connor/.virtualenvs/hosted-test3/lib/python3.6/site-packages/botocore-1.4.37-py3.6.egg/botocore/vendored/requests/packages/urllib3/connectionpool.py", line 341, in _make_request
self._validate_conn(conn)
File "/Users/connor/.virtualenvs/hosted-test3/lib/python3.6/site-packages/botocore-1.4.37-py3.6.egg/botocore/vendored/requests/packages/urllib3/connectionpool.py", line 761, in _validate_conn
conn.connect()
File "/Users/connor/.virtualenvs/hosted-test3/lib/python3.6/site-packages/botocore-1.4.37-py3.6.egg/botocore/vendored/requests/packages/urllib3/connection.py", line 238, in connect
ssl_version=resolved_ssl_version)
File "/Users/connor/.virtualenvs/hosted-test3/lib/python3.6/site-packages/snowflake_connector_python-1.3.7-py3.6.egg/snowflake/connector/ssl_wrap_socket.py", line 750, in ssl_wrap_socket_with_ocsp
).validate(server_hostname, ret.connection)
File "/Users/connor/.virtualenvs/hosted-test3/lib/python3.6/site-packages/snowflake_connector_python-1.3.7-py3.6.egg/snowflake/connector/ocsp_pyopenssl.py", line 1097, in validate
u'hostname={0}, err={1}', hostname, r.get()))
File "/usr/local/Cellar/python3/3.6.0/Frameworks/Python.framework/Versions/3.6/lib/python3.6/multiprocessing/pool.py", line 608, in get
raise self._value
File "/usr/local/Cellar/python3/3.6.0/Frameworks/Python.framework/Versions/3.6/lib/python3.6/multiprocessing/pool.py", line 119, in worker
result = (True, func(*args, **kwds))
File "/Users/connor/.virtualenvs/hosted-test3/lib/python3.6/site-packages/snowflake_connector_python-1.3.7-py3.6.egg/snowflake/connector/ocsp_pyopenssl.py", line 1113, in validate_by_direct_connection_sim
ple
ocsp_uri, ocsp_issuer, ocsp_subject)
File "/Users/connor/.virtualenvs/hosted-test3/lib/python3.6/site-packages/snowflake_connector_python-1.3.7-py3.6.egg/snowflake/connector/ocsp_pyopenssl.py", line 1155, in validate_by_direct_connection
raise err
File "/Users/connor/.virtualenvs/hosted-test3/lib/python3.6/site-packages/snowflake_connector_python-1.3.7-py3.6.egg/snowflake/connector/ocsp_pyopenssl.py", line 1135, in validate_by_direct_connection
ocsp_response = execute_ocsp_request(ocsp_uri, cert_id)
File "/Users/connor/.virtualenvs/hosted-test3/lib/python3.6/site-packages/snowflake_connector_python-1.3.7-py3.6.egg/snowflake/connector/ocsp_pyopenssl.py", line 558, in execute_ocsp_request
data=data)
File "/Users/connor/.virtualenvs/hosted-test3/lib/python3.6/site-packages/botocore-1.4.37-py3.6.egg/botocore/vendored/requests/sessions.py", line 508, in post
return self.request('POST', url, data=data, json=json, **kwargs)
File "/Users/connor/.virtualenvs/hosted-test3/lib/python3.6/site-packages/botocore-1.4.37-py3.6.egg/botocore/vendored/requests/sessions.py", line 465, in request
resp = self.send(prep, **send_kwargs)
File "/Users/connor/.virtualenvs/hosted-test3/lib/python3.6/site-packages/botocore-1.4.37-py3.6.egg/botocore/vendored/requests/sessions.py", line 573, in send
r = adapter.send(request, **kwargs)
File "/Users/connor/.virtualenvs/hosted-test3/lib/python3.6/site-packages/botocore-1.4.37-py3.6.egg/botocore/vendored/requests/adapters.py", line 370, in send
timeout=timeout
File "/Users/connor/.virtualenvs/hosted-test3/lib/python3.6/site-packages/botocore-1.4.37-py3.6.egg/botocore/vendored/requests/packages/urllib3/connectionpool.py", line 544, in urlopen
body=body, headers=headers)
File "/Users/connor/.virtualenvs/hosted-test3/lib/python3.6/site-packages/botocore-1.4.37-py3.6.egg/botocore/vendored/requests/packages/urllib3/connectionpool.py", line 349, in _make_request
conn.request(method, url, **httplib_request_kw)
File "/usr/local/Cellar/python3/3.6.0/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1239, in request
self._send_request(method, url, body, headers, encode_chunked)
TypeError: _send_request() takes 5 positional arguments but 6 were given
Per this issue in botocore this is fixed in 1.4.87, the latest botocore is 1.4.93 at the time of writing. Is there specific functionality in 1.4.37 that this needs? Would the correct fix simply be to update the requirements in this repository? If so, I would be happy to send a pull request for that.
I appreciate the help.
When connecting to snowflake, the database
and schema
parameters to the connect
method (https://docs.snowflake.net/manuals/user-guide/python-connector-api.html#connect) can be silently ignored if they don't exist or the user does not have the required permissions on them. It seems to be possible to double-check with something like this:
conn = snowflake.connector.connect(
...,
database='dbdoesnotexist',
schema='schemadoesnotexist'
)
if conn.schema != 'schemadoesnoteixst' or conn._database != 'dbdoesnotexist':
raise something
...but it feels like the connector should detect this case itself (and also, the schema
and _database
attributes are undocumented).
Then again, this is also how snowsql
behaves, so maybe this is expected behavior?
Snowflake connector seems to have limitation of accepting large sets at once (> 16,384 items). Would be nice to substantially increase this limit. My code is simply trying to upload a large (190k) csv file via use of Agate and SQLAlchemy. This code works fine when targeting against RDS MySQL 5.6.
Creating transformer_loader for /Users/jdfagan/tmp/property_ledger/LIONSTONEGRP/transform/GLSummary.csv ...
Loading source: /Users/jdfagan/tmp/property_ledger/LIONSTONEGRP/transform/transform/GLSummary.csv ...
database table: LIONSTONEGRP_GLSummary
| column | data_type |
| ---------- | --------- |
| AccountID | Number |
| AcctNum | Text |
| EntityID | Text |
| Department | Text |
| Basis | Text |
| Period | Text |
| PeriodDate | Date |
| Balfor | Text |
| Activity | Number |
FAILED
etl/mri/test_property_ledger_etl.py:109 (test_MriPropertyLedgerETL_load[LIONSTONEGRP])
self = <sqlalchemy.engine.base.Connection object at 0x10fe42438>
dialect = <snowflake.sqlalchemy.base.SnowflakeDialect object at 0x10fe42080>
constructor = <bound method DefaultExecutionContext._init_compiled of <class 'snowflake.sqlalchemy.base.SnowflakeExecutionContext'>>
statement = 'INSERT INTO "LIONSTONEGRP_GLSummary" ("AccountID", "AcctNum", "EntityID", "Department", "Basis", "Period", "PeriodDat...countID)s, %(AcctNum)s, %(EntityID)s, %(Department)s, %(Basis)s, %(Period)s, %(PeriodDate)s, %(Balfor)s, %(Activity)s)'
parameters = ({'AccountID': Decimal('1001000'), 'AcctNum': '1001-000', 'Activity': Decimal('-99.37'), 'Balfor': 'B', ...}, {'Accoun......}, {'AccountID': Decimal('1001000'), 'AcctNum': '1001-000', 'Activity': Decimal('300.00'), 'Balfor': 'B', ...}, ...)
args = (<snowflake.sqlalchemy.base.SnowflakeCompiler object at 0x10fe425c0>, [{'AccountID': Decimal('1001000'), 'AcctNum': '1.....}, {'AccountID': Decimal('1001000'), 'AcctNum': '1001-000', 'Activity': Decimal('300.00'), 'Balfor': 'B', ...}, ...])
conn = <sqlalchemy.pool._ConnectionFairy object at 0x10fe541d0>
context = <snowflake.sqlalchemy.base.SnowflakeExecutionContext object at 0x10fe429b0>
def _execute_context(self, dialect, constructor,
statement, parameters,
*args):
"""Create an :class:`.ExecutionContext` and execute, returning
a :class:`.ResultProxy`."""
try:
try:
conn = self.__connection
except AttributeError:
conn = self._revalidate_connection()
context = constructor(dialect, self, conn, *args)
except BaseException as e:
self._handle_dbapi_exception(
e,
util.text_type(statement), parameters,
None, None)
if context.compiled:
context.pre_exec()
cursor, statement, parameters = context.cursor, \
context.statement, \
context.parameters
if not context.executemany:
parameters = parameters[0]
if self._has_events or self.engine._has_events:
for fn in self.dispatch.before_cursor_execute:
statement, parameters = \
fn(self, cursor, statement, parameters,
context, context.executemany)
if self._echo:
self.engine.logger.info(statement)
self.engine.logger.info(
"%r",
sql_util._repr_params(parameters, batches=10)
)
evt_handled = False
try:
if context.executemany:
if self.dialect._has_events:
for fn in self.dialect.dispatch.do_executemany:
if fn(cursor, statement, parameters, context):
evt_handled = True
break
if not evt_handled:
self.dialect.do_executemany(
cursor,
statement,
parameters,
> context)
/Users/jdfagan/miniconda3/envs/waypoint_extraction/lib/python3.6/site-packages/sqlalchemy/engine/base.py:1159:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <snowflake.sqlalchemy.base.SnowflakeDialect object at 0x10fe42080>
cursor = <snowflake.connector.cursor.SnowflakeCursor object at 0x10fe42eb8>
statement = 'INSERT INTO "LIONSTONEGRP_GLSummary" ("AccountID", "AcctNum", "EntityID", "Department", "Basis", "Period", "PeriodDat...countID)s, %(AcctNum)s, %(EntityID)s, %(Department)s, %(Basis)s, %(Period)s, %(PeriodDate)s, %(Balfor)s, %(Activity)s)'
parameters = ({'AccountID': Decimal('1001000'), 'AcctNum': '1001-000', 'Activity': Decimal('-99.37'), 'Balfor': 'B', ...}, {'Accoun......}, {'AccountID': Decimal('1001000'), 'AcctNum': '1001-000', 'Activity': Decimal('300.00'), 'Balfor': 'B', ...}, ...)
context = <snowflake.sqlalchemy.base.SnowflakeExecutionContext object at 0x10fe429b0>
def do_executemany(self, cursor, statement, parameters, context=None):
> cursor.executemany(statement, parameters)
/Users/jdfagan/miniconda3/envs/waypoint_extraction/lib/python3.6/site-packages/sqlalchemy/engine/default.py:467:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <snowflake.connector.cursor.SnowflakeCursor object at 0x10fe42eb8>
command = 'INSERT INTO "LIONSTONEGRP_GLSummary" ("AccountID", "AcctNum", "EntityID", "Department", "Basis", "Period", "PeriodDat...B\', \'3252.70\'),(\'9999000\', \'9999-000\', \'SDTCRS\', \'@\', \'C\', \'01/16\', \'2016-01-01\', \'B\', \'3252.70\')'
seqparams = ({'AccountID': Decimal('1001000'), 'AcctNum': '1001-000', 'Activity': Decimal('-99.37'), 'Balfor': 'B', ...}, {'Accoun......}, {'AccountID': Decimal('1001000'), 'AcctNum': '1001-000', 'Activity': Decimal('300.00'), 'Balfor': 'B', ...}, ...)
def executemany(self, command, seqparams):
u"""
Executes a command/query with the given set of parameters sequentially.
"""
logger.info(u'executing many SQLs/commands')
command = command.strip(u' \t\n\r') if command else None
if self.INSERT_SQL_RE.match(command):
logger.debug(u'rewriting INSERT query')
command_wo_comments = re.sub(self.COMMENT_SQL_RE, u'', command)
m = self.INSERT_SQL_VALUES_RE.match(command_wo_comments)
if not m:
errorvalue = {
u'msg': u"Failed to rewrite multi-row insert",
u'errno': ER_FAILED_TO_REWRITE_MULTI_ROW_INSERT
}
Error.errorhandler_wrapper(
self.connection, self, InterfaceError, errorvalue
)
fmt = m.group(1)
values = []
for param in seqparams:
logger.debug(u'parameter: %s', param)
values.append(fmt % self.__process_params(param))
command = command.replace(fmt, u','.join(values), 1)
> self.execute(command)
/Users/jdfagan/miniconda3/envs/waypoint_extraction/lib/python3.6/site-packages/snowflake/connector/cursor.py:666:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <snowflake.connector.cursor.SnowflakeCursor object at 0x10fe42eb8>
command = 'INSERT INTO "LIONSTONEGRP_GLSummary" ("AccountID", "AcctNum", "EntityID", "Department", "Basis", "Period", "PeriodDat...B\', \'3252.70\'),(\'9999000\', \'9999-000\', \'SDTCRS\', \'@\', \'C\', \'01/16\', \'2016-01-01\', \'B\', \'3252.70\')'
params = None, timeout = None, _do_reset = True, _put_callback = None
_put_callback_output_stream = <_io.TextIOWrapper name='<stdout>' mode='w' encoding='UTF-8'>
_get_callback = None
_get_callback_output_stream = <_io.TextIOWrapper name='<stdout>' mode='w' encoding='UTF-8'>
_statement_params = None, _is_internal = False, _no_results = False
_use_ijson = False, _is_put_get = None
def execute(self, command, params=None, timeout=None,
_do_reset=True,
_put_callback=None,
_put_callback_output_stream=sys.stdout,
_get_callback=None,
_get_callback_output_stream=sys.stdout,
_statement_params=None,
_is_internal=False,
_no_results=False,
_use_ijson=False,
_is_put_get=None):
u"""
Executes a command/query
"""
logger.debug(u'executing SQL/command')
if self.is_closed():
Error.errorhandler_wrapper(
self.connection, self,
DatabaseError,
{u'msg': u"Cursor is closed in execute.",
u'errno': ER_CURSOR_IS_CLOSED})
if _do_reset:
self.reset()
command = command.strip(u' \t\n\r') if command else None
if not command:
logger.warning(u'execute: no query is given to execute')
return
processed_params = self.__process_params(params)
logger.debug(u'binding: %s with input=%s, processed=%s',
command,
params, processed_params)
if len(processed_params) > 0:
query = command % processed_params
else:
query = command
if logger.getEffectiveLevel() <= logging.DEBUG:
logger.debug(
u'query: [%s]',
u' '.join(line.strip() for line in query.split(u'\n')))
m = DESC_TABLE_RE.match(query)
if m:
query1 = u'describe table {0}'.format(m.group(1))
if logger.getEffectiveLevel() <= logging.WARNING:
logger.warning(
u'query was rewritten: org=%s, new=%s',
u' '.join(line.strip() for line in query.split(u'\n')),
query1
)
query = query1
ret = self._execute_helper(query, timeout=timeout,
statement_params=_statement_params,
is_internal=_is_internal,
_no_results=_no_results,
_is_put_get=_is_put_get)
self._sfqid = ret[u'data'][
u'queryId'] if u'data' in ret and u'queryId' in ret[
u'data'] else None
self._sqlstate = ret[u'data'][
u'sqlState'] if u'data' in ret and u'sqlState' in ret[
u'data'] else None
logger.debug('sfqid: %s', self.sfqid)
if ret[u'success']:
logger.debug(u'SUCCESS')
data = ret[u'data']
if u'finalDatabaseName' in data:
self._connection._database = data[u'finalDatabaseName']
if u'finalSchemaName' in data:
self._connection._schema = data[u'finalSchemaName']
if u'finalWarehouseName' in data:
self._connection._warehouse = data[u'finalWarehouseName']
if u'finalRoleName' in data:
self._connection._role = data[u'finalRoleName']
# logger.debug(ret)
logger.debug(u"PUT OR GET: %s", self.is_file_transfer)
if self.is_file_transfer:
sf_file_transfer_agent = SnowflakeFileTransferAgent(
self, query, ret,
put_callback=_put_callback,
put_callback_output_stream=_put_callback_output_stream,
get_callback=_get_callback,
get_callback_output_stream=_get_callback_output_stream)
sf_file_transfer_agent.execute()
data = sf_file_transfer_agent.result()
self._total_rowcount = len(data[u'rowset']) if \
u'rowset' in data else -1
m = self.ALTER_SESSION_RE.match(query)
if m:
# session parameters
param = m.group(1).upper()
value = m.group(2)
self._connection.converter.set_parameter(param, value)
if _no_results:
self._total_rowcount = ret[u'data'][
u'total'] if u'data' in ret and u'total' in ret[
u'data'] else -1
return data
self.chunk_info(data, use_ijson=_use_ijson)
else:
self._total_rowcount = ret[u'data'][
u'total'] if u'data' in ret and u'total' in ret[u'data'] else -1
logger.debug(ret)
err = ret[u'message']
code = ret.get(u'code', -1)
if u'data' in ret and u'errorMessage' in ret[u'data']:
err += ret[u'data'][u'errorMessage']
errvalue = {
u'msg': err,
u'errno': int(code),
u'sqlstate': self._sqlstate,
u'sfqid': self._sfqid
}
Error.errorhandler_wrapper(self.connection, self,
ProgrammingError,
> errvalue)
/Users/jdfagan/miniconda3/envs/waypoint_extraction/lib/python3.6/site-packages/snowflake/connector/cursor.py:526:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
connection = <snowflake.connector.connection.SnowflakeConnection object at 0x10fe42c50>
cursor = <snowflake.connector.cursor.SnowflakeCursor object at 0x10fe42eb8>
errorclass = <class 'snowflake.connector.errors.ProgrammingError'>
errorvalue = {'done_format_msg': False, 'errno': 1795, 'msg': 'SQL compilation error: error line 1 at position 143\nmaximum number ...ssions in a list exceeded, expected at most 16,384, got 190,800', 'sfqid': 'c56de412-761c-44d6-b202-5867134658cc', ...}
@staticmethod
def errorhandler_wrapper(connection, cursor, errorclass, errorvalue=None):
u"""
Error handler wrapper that calls the errorhandler method
"""
if errorvalue is None:
# no value indicates errorclass is errorobject
errorobject = errorclass
errorclass = type(errorobject)
errorvalue = {
u'msg': errorobject.msg,
u'errno': errorobject.errno,
u'sqlstate': errorobject.sqlstate,
u'done_format_msg': True
}
else:
errorvalue[u'done_format_msg'] = False
if connection is not None:
connection.messages.append((errorclass, errorvalue))
if cursor is not None:
cursor.messages.append((errorclass, errorvalue))
> cursor.errorhandler(connection, cursor, errorclass, errorvalue)
/Users/jdfagan/miniconda3/envs/waypoint_extraction/lib/python3.6/site-packages/snowflake/connector/errors.py:97:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
connection = <snowflake.connector.connection.SnowflakeConnection object at 0x10fe42c50>
cursor = <snowflake.connector.cursor.SnowflakeCursor object at 0x10fe42eb8>
errorclass = <class 'snowflake.connector.errors.ProgrammingError'>
errorvalue = {'done_format_msg': False, 'errno': 1795, 'msg': 'SQL compilation error: error line 1 at position 143\nmaximum number ...ssions in a list exceeded, expected at most 16,384, got 190,800', 'sfqid': 'c56de412-761c-44d6-b202-5867134658cc', ...}
@staticmethod
def default_errorhandler(connection, cursor, errorclass, errorvalue):
u"""
Default error handler that raises an error
"""
raise errorclass(
msg=errorvalue.get(u'msg'),
errno=errorvalue.get(u'errno'),
sqlstate=errorvalue.get(u'sqlstate'),
sfqid=errorvalue.get(u'sfqid'),
> done_format_msg=errorvalue.get(u'done_format_msg'))
E snowflake.connector.errors.ProgrammingError: 001795 (42601): SQL compilation error: error line 1 at position 143
E maximum number of expressions in a list exceeded, expected at most 16,384, got 190,800
/Users/jdfagan/miniconda3/envs/waypoint_extraction/lib/python3.6/site-packages/snowflake/connector/errors.py:73: ProgrammingError
The above exception was the direct cause of the following exception:
client = 'LIONSTONEGRP'
@pytest.mark.parametrize("client", clients)
def test_MriPropertyLedgerETL_load(client: str):
etl = MriPropertyLedgerETL(client_database=client, etl_dir="{}{}".format(ETL_DIR_BASE, client))
> etl.load()
etl/mri/test_property_ledger_etl.py:113:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
../../waypoint/etl/property_ledger_etl.py:80: in load
loader.from_csv().to_sql(database_url, self.client_code)
../../waypoint/etl/mri/transformer_loader.py:89: in to_sql
self.table.to_sql(database_url, table_name, overwrite=True)
/Users/jdfagan/miniconda3/envs/waypoint_extraction/lib/python3.6/site-packages/agatesql/table.py:219: in to_sql
connection.execute(insert, [dict(zip(self.column_names, row)) for row in self.rows])
/Users/jdfagan/miniconda3/envs/waypoint_extraction/lib/python3.6/site-packages/sqlalchemy/engine/base.py:945: in execute
return meth(self, multiparams, params)
/Users/jdfagan/miniconda3/envs/waypoint_extraction/lib/python3.6/site-packages/sqlalchemy/sql/elements.py:263: in _execute_on_connection
return connection._execute_clauseelement(self, multiparams, params)
/Users/jdfagan/miniconda3/envs/waypoint_extraction/lib/python3.6/site-packages/sqlalchemy/engine/base.py:1053: in _execute_clauseelement
compiled_sql, distilled_params
/Users/jdfagan/miniconda3/envs/waypoint_extraction/lib/python3.6/site-packages/sqlalchemy/engine/base.py:1189: in _execute_context
context)
/Users/jdfagan/miniconda3/envs/waypoint_extraction/lib/python3.6/site-packages/sqlalchemy/engine/base.py:1402: in _handle_dbapi_exception
exc_info
/Users/jdfagan/miniconda3/envs/waypoint_extraction/lib/python3.6/site-packages/sqlalchemy/util/compat.py:203: in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
/Users/jdfagan/miniconda3/envs/waypoint_extraction/lib/python3.6/site-packages/sqlalchemy/util/compat.py:186: in reraise
raise value.with_traceback(tb)
/Users/jdfagan/miniconda3/envs/waypoint_extraction/lib/python3.6/site-packages/sqlalchemy/engine/base.py:1159: in _execute_context
context)
/Users/jdfagan/miniconda3/envs/waypoint_extraction/lib/python3.6/site-packages/sqlalchemy/engine/default.py:467: in do_executemany
cursor.executemany(statement, parameters)
/Users/jdfagan/miniconda3/envs/waypoint_extraction/lib/python3.6/site-packages/snowflake/connector/cursor.py:666: in executemany
self.execute(command)
/Users/jdfagan/miniconda3/envs/waypoint_extraction/lib/python3.6/site-packages/snowflake/connector/cursor.py:526: in execute
errvalue)
/Users/jdfagan/miniconda3/envs/waypoint_extraction/lib/python3.6/site-packages/snowflake/connector/errors.py:97: in errorhandler_wrapper
cursor.errorhandler(connection, cursor, errorclass, errorvalue)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
connection = <snowflake.connector.connection.SnowflakeConnection object at 0x10fe42c50>
cursor = <snowflake.connector.cursor.SnowflakeCursor object at 0x10fe42eb8>
errorclass = <class 'snowflake.connector.errors.ProgrammingError'>
errorvalue = {'done_format_msg': False, 'errno': 1795, 'msg': 'SQL compilation error: error line 1 at position 143\nmaximum number ...ssions in a list exceeded, expected at most 16,384, got 190,800', 'sfqid': 'c56de412-761c-44d6-b202-5867134658cc', ...}
@staticmethod
def default_errorhandler(connection, cursor, errorclass, errorvalue):
u"""
Default error handler that raises an error
"""
raise errorclass(
msg=errorvalue.get(u'msg'),
errno=errorvalue.get(u'errno'),
sqlstate=errorvalue.get(u'sqlstate'),
sfqid=errorvalue.get(u'sfqid'),
> done_format_msg=errorvalue.get(u'done_format_msg'))
E sqlalchemy.exc.ProgrammingError: (snowflake.connector.errors.ProgrammingError) 001795 (42601): SQL compilation error: error line 1 at position 143
E maximum number of expressions in a list exceeded, expected at most 16,384, got 190,800 [SQL: 'INSERT INTO "LIONSTONEGRP_GLSummary" ("AccountID", "AcctNum", "EntityID", "Department", "Basis", "Period", "PeriodDate", "Balfor", "Activity") VALUES (%(AccountID)s, %(AcctNum)s, %(EntityID)s, %(Department)s, %(Basis)s, %(Period)s, %(PeriodDate)s, %(Balfor)s, %(Activity)s)'] [parameters: ({'AccountID': Decimal('1001000'), 'AcctNum': '1001-000', 'EntityID': '2525OF', 'Department': '@', 'Basis': 'A', 'Period': '01/16', 'PeriodDate': datetime.date(2016, 1, 1), 'Balfor': 'B', 'Activity': Decimal('-99.37')}, {'AccountID': Decimal('1001000'), 'AcctNum': '1001-000', 'EntityID': '2525OF', 'Department': '@', 'Basis': 'C', 'Period': '01/16', 'PeriodDate': datetime.date(2016, 1, 1), 'Balfor': 'B', 'Activity': Decimal('-99.37')}, {'AccountID': Decimal('1001000'), 'AcctNum': '1001-000', 'EntityID': '2525WE', 'Department': '@', 'Basis': 'A', 'Period': '01/16', 'PeriodDate': datetime.date(2016, 1, 1), 'Balfor': 'B', 'Activity': Decimal('99.37')}, {'AccountID': Decimal('1001000'), 'AcctNum': '1001-000', 'EntityID': '712MAI', 'Department': '@', 'Basis': 'A', 'Period': '07/16', 'PeriodDate': datetime.date(2016, 7, 1), 'Balfor': 'B', 'Activity': Decimal('300.00')}, {'AccountID': Decimal('1001000'), 'AcctNum': '1001-000', 'EntityID': 'CTYCTR', 'Department': '@', 'Basis': 'A', 'Period': '01/16', 'PeriodDate': datetime.date(2016, 1, 1), 'Balfor': 'B', 'Activity': Decimal('300.00')}, {'AccountID': Decimal('1001000'), 'AcctNum': '1001-000', 'EntityID': 'GSLTDP', 'Department': '@', 'Basis': 'A', 'Period': '01/16', 'PeriodDate': datetime.date(2016, 1, 1), 'Balfor': 'B', 'Activity': Decimal('300.00')}, {'AccountID': Decimal('1002001'), 'AcctNum': '1002-001', 'EntityID': '110ATR', 'Department': '@', 'Basis': 'A', 'Period': '11/16', 'PeriodDate': datetime.date(2016, 11, 1), 'Balfor': 'N', 'Activity': Decimal('440667.35')}, {'AccountID': Decimal('1002001'), 'AcctNum': '1002-001', 'EntityID': '110ATR', 'Department': '@', 'Basis': 'A', 'Period': '12/16', 'PeriodDate': datetime.date(2016, 12, 1), 'Balfor': 'N', 'Activity': Decimal('339664.73')} ... displaying 10 of 190800 total bound parameter sets ... {'AccountID': Decimal('9999000'), 'AcctNum': '9999-000', 'EntityID': 'SDTCRS', 'Department': '@', 'Basis': 'A', 'Period': '01/16', 'PeriodDate': datetime.date(2016, 1, 1), 'Balfor': 'B', 'Activity': Decimal('3252.70')}, {'AccountID': Decimal('9999000'), 'AcctNum': '9999-000', 'EntityID': 'SDTCRS', 'Department': '@', 'Basis': 'C', 'Period': '01/16', 'PeriodDate': datetime.date(2016, 1, 1), 'Balfor': 'B', 'Activity': Decimal('3252.70')})]
/Users/jdfagan/miniconda3/envs/waypoint_extraction/lib/python3.6/site-packages/snowflake/connector/errors.py:73: ProgrammingError
Below is my environment:
name: waypoint_extraction
channels:
- conda-forge
- defaults
- asmeurer
- snowflakedb
dependencies:
- agate=1.6.0=py36_2
- agate-dbf=0.2.0=py36_0
- agate-excel=0.2.1=py36_0
- agate-sql=0.5.2=py36_0
- asn1crypto=0.22.0=py36_0
- babel=2.4.0=py36_0
- ca-certificates=2017.7.27.1=0
- certifi=2017.7.27.1=py36_0
- cffi=1.10.0=py36_0
- chardet=3.0.4=py36_0
- coverage=4.4.1=py36_0
- cryptography=2.0.3=py36_0
- csvkit=1.0.2=py36_0
- dbfread=2.0.7=py36_0
- et_xmlfile=1.0.1=py36_0
- isodate=0.5.4=py36_0
- jdcal=1.3=py36_0
- leather=0.3.3=py36_0
- libffi=3.2.1=3
- mysql-connector-python=2.2.2=py36_0
- ncurses=5.9=10
- openpyxl=2.5.0a3=py36_0
- openssl=1.0.2l=0
- parsedatetime=2.4=py36_0
- pip=9.0.1=py36_0
- py=1.4.34=py36_0
- pycparser=2.18=py36_0
- pyopenssl=17.2.0=py36_0
- pysocks=1.6.7=py36_0
- pytest=3.2.2=py_0
- pytest-cov=2.5.1=py36_0
- python=3.6.2=0
- python-slugify=1.2.4=py36_1
- pytimeparse=1.1.7=py36_0
- pytz=2017.2=py36_0
- readline=6.2=0
- requests=2.18.4=py36_1
- requests-toolbelt=0.8.0=py36_0
- setuptools=36.3.0=py36_0
- six=1.10.0=py36_1
- sqlalchemy=1.1.13=py36_0
- sqlite=3.13.0=1
- tk=8.5.19=2
- unidecode=0.04.21=py36_0
- urllib3=1.22=py36_0
- wheel=0.29.0=py36_0
- xlrd=1.1.0=py36_1
- xz=5.2.3=0
- zlib=1.2.11=0
- idna=2.6=py36_0
- pymysql=0.7.9=py36_0
- pip:
- boto3==1.4.6
- botocore==1.6.8
- docutils==0.14
- et-xmlfile==1.0.1
- future==0.16.0
- hvac==0.2.17
- ijson==2.3
- jmespath==0.9.3
- pyasn1==0.2.3
- pyasn1-modules==0.0.9
- pycryptodome==3.4.7
- python-dateutil==2.6.1
- s3transfer==0.1.11
- snowflake-connector-python==1.4.6
- snowflake-sqlalchemy==1.0.7
prefix: /Users/jdfagan/miniconda3/envs/waypoint_extraction
setup.py
: snowflake-connector-python==1.4.10
Followed the installation instructions here: https://docs.snowflake.net/manuals/user-guide/python-connector-install.html
I can install snowflake-connector-python
successfully, but I get this error when import
ing it:
>>> import snowflake.connector
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/vagrant/.virtualenvs/example/lib/python3.4/site-packages/snowflake_connector_python-1.4.10-py3.4.egg/snowflake/connector/__init__.py", line 21, in <module>
from .connection import SnowflakeConnection
File "/home/vagrant/.virtualenvs/example/lib/python3.4/site-packages/snowflake_connector_python-1.4.10-py3.4.egg/snowflake/connector/connection.py", line 24, in <module>
from .cursor import SnowflakeCursor
File "/home/vagrant/.virtualenvs/example/lib/python3.4/site-packages/snowflake_connector_python-1.4.10-py3.4.egg/snowflake/connector/cursor.py", line 29, in <module>
from .file_transfer_agent import (SnowflakeFileTransferAgent)
File "/home/vagrant/.virtualenvs/example/lib/python3.4/site-packages/snowflake_connector_python-1.4.10-py3.4.egg/snowflake/connector/file_transfer_agent.py", line 31, in <module>
from .remote_storage_util import (SnowflakeFileEncryptionMaterial, SnowflakeRemoteStorageUtil,
File "/home/vagrant/.virtualenvs/example/lib/python3.4/site-packages/snowflake_connector_python-1.4.10-py3.4.egg/snowflake/connector/remote_storage_util.py", line 16, in <module>
from .azure_util import SnowflakeAzureUtil
File "/home/vagrant/.virtualenvs/example/lib/python3.4/site-packages/snowflake_connector_python-1.4.10-py3.4.egg/snowflake/connector/azure_util.py", line 8, in <module>
from azure.storage.blob import BlockBlobService
ImportError: No module named 'azure.storage'
I don't get this error with version 1.4.9.
My team works with conda to manage our python environments and we are looking into using snowflake. We've opened up a PR to distribute this connector via conda-forge: conda-forge/staged-recipes#7008
It would be nice if you could mention this way of installing the package in your docs (after the package becomes available), in addition to pypi/pip. Thanks!
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.