clickhouse插件连接阿里云上的clickhouse就会连接超时 about addax

[wgzhao]

请使用 bin/datax.py --loglevel debug xxx.json 运行后，贴出详细的报错信息，看看是不是驱动不兼容的问题

[ProjectMoss]

这是详细日志

2021-01-12 16:21:05.948 [main] INFO PerfTrace - PerfTrace traceId=job_-1, isEnable=false, priority=0
2021-01-12 16:21:05.948 [main] INFO JobContainer - DataX jobContainer starts job.
2021-01-12 16:21:05.951 [main] DEBUG JobContainer - jobContainer starts to do preHandle ...
2021-01-12 16:21:05.952 [main] DEBUG JobContainer - jobContainer starts to do init ...
2021-01-12 16:21:05.952 [main] INFO JobContainer - Set jobId = 0
2021-01-12 16:21:06.182 [job-0] INFO ClickHouseDriver - Driver registered
2021-01-12 16:21:06.193 [job-0] DEBUG ClickHouseDriver - Creating connection
2021-01-12 16:21:06.204 [job-0] DEBUG ClickHouseConnectionImpl - Create a new connection to jdbc:clickhouse://xxxxxxxxxxxxxx:8123
2021-01-12 16:21:06.361 [job-0] DEBUG ClickHouseStatementImpl - Executing SQL: select timezone()
FORMAT TabSeparatedWithNamesAndTypes;
2021-01-12 16:21:06.389 [job-0] DEBUG ClickHouseStatementImpl - Request url: xxxxxxxxxxxxxx:8123/?extremes=0&compress=1&database=default&user=yt_dev_etl&password=4jVtR2V5nTLnK4&query_id=153f3037-146d-422e-9883-27fa78695895
2021-01-12 16:21:16.564 [job-0] INFO ClickHouseStatementImpl - Error during connection to ru.yandex.clickhouse.settings.ClickHouseProperties@49139829, reporting failure to data source, message: Connect to xxxxxxxxxxxxxx:8123 [xxxxxxxxxxxxxx/xx.xx.xx.xx] failed: connect timed out
2021-01-12 16:21:16.565 [job-0] INFO ClickHouseStatementImpl - Error sql: select timezone()
FORMAT TabSeparatedWithNamesAndTypes;
2021-01-12 16:21:16.599 [job-0] WARN DBUtil - test connection of [jdbc:clickhouse://xxxxxxxxxxxxxx:8123] failed, for Code:[DBUtilErrorCode-10], Description:[连接数据库失败. 请检查您的 账号、密码、数据库名称、IP、Port或者向 DBA 寻求帮助(注意网络环境).]. - 具体错误信息为：java.lang.RuntimeException: ru.yandex.clickhouse.except.ClickHouseException: ClickHouse exception, code: 210, host: xxxxxxxxxxxxxx, port: 8123; Connect to xxxxxxxxxxxxxx:8123 [xxxxxxxxxxxxxx/xx.xx.xx.xx] failed: connect timed out.
2021-01-12 16:21:16.602 [job-0] ERROR RetryUtil - Exception when calling callable, 异常Msg:DataX无法连接对应的数据库，可能原因是：1) 配置的ip/port/database/jdbc错误，无法连接。2) 配置的username/password错误，鉴权失败。请和DBA确认该数据库的连接信息是否正确。
java.lang.Exception: DataX无法连接对应的数据库，可能原因是：1) 配置的ip/port/database/jdbc错误，无法连接。2) 配置的username/password错误，鉴权失败。请和DBA确认该数据库的连接信息是否正确。
at com.alibaba.datax.plugin.rdbms.util.DBUtil.lambda$chooseJdbcUrl$1(DBUtil.java:78)
at com.alibaba.datax.common.util.RetryUtil$Retry.call(RetryUtil.java:180)
at com.alibaba.datax.common.util.RetryUtil$Retry.doRetry(RetryUtil.java:126)
at com.alibaba.datax.common.util.RetryUtil.executeWithRetry(RetryUtil.java:39)
at com.alibaba.datax.plugin.rdbms.util.DBUtil.chooseJdbcUrl(DBUtil.java:60)
at com.alibaba.datax.plugin.rdbms.reader.util.OriginalConfPretreatmentUtil.dealJdbcAndTable(OriginalConfPretreatmentUtil.java:104)
at com.alibaba.datax.plugin.rdbms.reader.util.OriginalConfPretreatmentUtil.simplifyConf(OriginalConfPretreatmentUtil.java:69)
at com.alibaba.datax.plugin.rdbms.reader.util.OriginalConfPretreatmentUtil.doPretreatment(OriginalConfPretreatmentUtil.java:41)
at com.alibaba.datax.plugin.rdbms.reader.CommonRdbmsReader$Job.init(CommonRdbmsReader.java:58)
at com.alibaba.datax.plugin.reader.clickhousereader.ClickHouseReader$Job.init(ClickHouseReader.java:32)
at com.alibaba.datax.core.job.JobContainer.initJobReader(JobContainer.java:661)
at com.alibaba.datax.core.job.JobContainer.init(JobContainer.java:306)
at com.alibaba.datax.core.job.JobContainer.start(JobContainer.java:113)
at com.alibaba.datax.core.Engine.start(Engine.java:86)
at com.alibaba.datax.core.Engine.entry(Engine.java:147)
at com.alibaba.datax.core.Engine.main(Engine.java:165)
2021-01-12 16:21:17.603 [job-0] ERROR RetryUtil - Exception when calling callable, 即将尝试执行第1次重试.本次重试计划等待[1000]ms,实际等待[1000]ms, 异常Msg:[DataX无法连接对应的数据库，可能原因是：1) 配置的ip/port/database/jdbc错误，无法连接。2) 配置的username/password错误，鉴权失败。请和DBA确认该数据库的连接信息是否正确。]
2021-01-12 16:21:17.604 [job-0] DEBUG ClickHouseDriver - Creating connection
2021-01-12 16:21:17.605 [job-0] DEBUG ClickHouseConnectionImpl - Create a new connection to jdbc:clickhouse://xxxxxxxxxxxxxx:8123
2021-01-12 16:21:17.607 [job-0] DEBUG ClickHouseStatementImpl - Executing SQL: select timezone()
FORMAT TabSeparatedWithNamesAndTypes;
2021-01-12 16:21:17.608 [job-0] DEBUG ClickHouseStatementImpl - Request url: xxxxxxxxxxxxxx:8123/?extremes=0&compress=1&database=default&user=yt_dev_etl&password=4jVtR2V5nTLnK4&query_id=58dd02dd-0706-4c71-bdf3-93958bf6109c
2021-01-12 16:21:27.621 [job-0] INFO ClickHouseStatementImpl - Error during connection to ru.yandex.clickhouse.settings.ClickHouseProperties@130d63be, reporting failure to data source, message: Connect to xxxxxxxxxxxxxx:8123 [xxxxxxxxxxxxxx/xx.xx.xx.xx] failed: connect timed out
2021-01-12 16:21:27.622 [job-0] INFO ClickHouseStatementImpl - Error sql: select timezone()
FORMAT TabSeparatedWithNamesAndTypes;
2021-01-12 16:21:27.622 [job-0] WARN DBUtil - test connection of [jdbc:clickhouse://xxxxxxxxxxxxxx:8123] failed, for Code:[DBUtilErrorCode-10], Description:[连接数据库失败. 请检查您的 账号、密码、数据库名称、IP、Port或者向 DBA 寻求帮助(注意网络环境).]. - 具体错误信息为：java.lang.RuntimeException: ru.yandex.clickhouse.except.ClickHouseException: ClickHouse exception, code: 210, host: xxxxxxxxxxxxxx, port: 8123; Connect to xxxxxxxxxxxxxx:8123 [xxxxxxxxxxxxxx/xx.xx.xx.xx] failed: connect timed out.
2021-01-12 16:21:29.623 [job-0] ERROR RetryUtil - Exception when calling callable, 即将尝试执行第2次重试.本次重试计划等待[2000]ms,实际等待[2000]ms, 异常Msg:[DataX无法连接对应的数据库，可能原因是：1) 配置的ip/port/database/jdbc错误，无法连接。2) 配置的username/password错误，鉴权失败。请和DBA确认该数据库的连接信息是否正确。]
2021-01-12 16:21:29.624 [job-0] DEBUG ClickHouseDriver - Creating connection
2021-01-12 16:21:29.625 [job-0] DEBUG ClickHouseConnectionImpl - Create a new connection to jdbc:clickhouse://xxxxxxxxxxxxxx:8123
2021-01-12 16:21:29.627 [job-0] DEBUG ClickHouseStatementImpl - Executing SQL: select timezone()
FORMAT TabSeparatedWithNamesAndTypes;
2021-01-12 16:21:29.628 [job-0] DEBUG ClickHouseStatementImpl - Request url: xxxxxxxxxxxxxx:8123/?extremes=0&compress=1&database=default&user=yt_dev_etl&password=4jVtR2V5nTLnK4&query_id=b4198242-da01-4f62-8ad5-c468e18de313
2021-01-12 16:21:39.635 [job-0] INFO ClickHouseStatementImpl - Error during connection to ru.yandex.clickhouse.settings.ClickHouseProperties@1283bb96, reporting failure to data source, message: Connect to xxxxxxxxxxxxxx:8123 [xxxxxxxxxxxxxx/xx.xx.xx.xx] failed: connect timed out
2021-01-12 16:21:39.636 [job-0] INFO ClickHouseStatementImpl - Error sql: select timezone()
FORMAT TabSeparatedWithNamesAndTypes;
2021-01-12 16:21:39.636 [job-0] WARN DBUtil - test connection of [jdbc:clickhouse://xxxxxxxxxxxxxx:8123] failed, for Code:[DBUtilErrorCode-10], Description:[连接数据库失败. 请检查您的 账号、密码、数据库名称、IP、Port或者向 DBA 寻求帮助(注意网络环境).]. - 具体错误信息为：java.lang.RuntimeException: ru.yandex.clickhouse.except.ClickHouseException: ClickHouse exception, code: 210, host: xxxxxxxxxxxxxx, port: 8123; Connect to xxxxxxxxxxxxxx:8123 [xxxxxxxxxxxxxx/xx.xx.xx.xx] failed: connect timed out.
2021-01-12 16:21:39.646 [job-0] INFO StandAloneJobContainerCommunicator - Total 0 records, 0 bytes | Speed 0B/s, 0 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 0.000s | Percentage 0.00%
2021-01-12 16:21:39.648 [job-0] ERROR Engine - Code:[DBUtilErrorCode-10], Description:[连接数据库失败. 请检查您的 账号、密码、数据库名称、IP、Port或者向 DBA 寻求帮助(注意网络环境).]. - Code:[DBUtilErrorCode-10], Description:[连接数据库失败. 请检查您的 账号、密码、数据库名称、IP、Port或者向 DBA 寻求帮助(注意网络环境).]. - 数据库连接失败. 因为根据您配置的连接信息,无法从:jdbc:clickhouse://xxxxxxxxxxxxxx:8123 中找到可连接的jdbcUrl. 请检查您的配置并作出修改. - java.lang.Exception: DataX无法连接对应的数据库，可能原因是：1) 配置的ip/port/database/jdbc错误，无法连接。2) 配置的username/password错误，鉴权失败。请和DBA确认该数据库的连接信息是否正确。
at com.alibaba.datax.plugin.rdbms.util.DBUtil.lambda$chooseJdbcUrl$1(DBUtil.java:78)
at com.alibaba.datax.common.util.RetryUtil$Retry.call(RetryUtil.java:180)
at com.alibaba.datax.common.util.RetryUtil$Retry.doRetry(RetryUtil.java:126)
at com.alibaba.datax.common.util.RetryUtil.executeWithRetry(RetryUtil.java:39)
at com.alibaba.datax.plugin.rdbms.util.DBUtil.chooseJdbcUrl(DBUtil.java:60)
at com.alibaba.datax.plugin.rdbms.reader.util.OriginalConfPretreatmentUtil.dealJdbcAndTable(OriginalConfPretreatmentUtil.java:104)
at com.alibaba.datax.plugin.rdbms.reader.util.OriginalConfPretreatmentUtil.simplifyConf(OriginalConfPretreatmentUtil.java:69)
at com.alibaba.datax.plugin.rdbms.reader.util.OriginalConfPretreatmentUtil.doPretreatment(OriginalConfPretreatmentUtil.java:41)
at com.alibaba.datax.plugin.rdbms.reader.CommonRdbmsReader$Job.init(CommonRdbmsReader.java:58)
at com.alibaba.datax.plugin.reader.clickhousereader.ClickHouseReader$Job.init(ClickHouseReader.java:32)
at com.alibaba.datax.core.job.JobContainer.initJobReader(JobContainer.java:661)
at com.alibaba.datax.core.job.JobContainer.init(JobContainer.java:306)
at com.alibaba.datax.core.job.JobContainer.start(JobContainer.java:113)
at com.alibaba.datax.core.Engine.start(Engine.java:86)
at com.alibaba.datax.core.Engine.entry(Engine.java:147)
at com.alibaba.datax.core.Engine.main(Engine.java:165)

[ProjectMoss]

这是驱动的问题吗

[wgzhao]

从日志暂时看不出原因是啥，你尝试调高connect timeout 值 看看，或者看看clickhouse 端的日志能否找到一些端倪。

[ProjectMoss]

找到原因了，是阿里云的ck需要添加ip白名单，不然默认都是黑名单。所以是端口不通导致的。不好意思，浪费时间了。