Comments (6)
wgzhao
commented on August 19, 2024
请粘贴完整的配置脚本执行日志，请勿采用截图方式，谢谢。
from addax.
sxfang32
commented on August 19, 2024
==================== DEPRECATED WARNING ========================
addax.py is deprecated, It's going to be removed in future release.
As a replacement, you can use addax.sh to run job
==================== DEPRECATED WARNING ========================
___ _ _
/ _ \ | | | |
/ /_\ \ __| | __| | __ ___ __
| _ |/ _` |/ _` |/ _` \ \/ /
| | | | (_| | (_| | (_| |> <
\_| |_/\__,_|\__,_|\__,_/_/\_\
:: Addax version :: (v4.1.0)
2023-07-10 20:56:53.641 [ main] INFO VMInfo - VMInfo# operatingSystem class => sun.management.OperatingSystemImpl
2023-07-10 20:56:53.653 [ main] INFO Engine -
{
"setting":{
"speed":{
"channel":3
},
"errorLimit":{
"record":0,
"percentage":0.02
}
},
"content":{
"reader":{
"name":"clickhousereader",
"parameter":{
"username":"*****",
"password":"******",
"column":[
"*"
],
"connection":[
{
"jdbcUrl":[
"jdbc:clickhouse://*****:8123/Marketing_DWM_DT"
],
"table":[
"***"
]
}
]
}
},
"writer":{
"name":"clickhousewriter",
"parameter":{
"username":"*****",
"password":"****",
"column":[
"*"
],
"connection":[
{
"table":[
"*****"
],
"jdbcUrl":"jdbc:clickhouse://*****:8123/tmp"
}
]
}
}
}
}
2023-07-10 20:56:53.665 [ main] INFO JobContainer - The jobContainer begins to process the job.
2023-07-10 20:56:54.843 [ job-0] INFO OriginalConfPretreatmentUtil - Available jdbcUrl:jdbc:clickhouse://172.31.163.36:8123/Marketing_DWM_DT.
2023-07-10 20:56:54.845 [ job-0] WARN OriginalConfPretreatmentUtil - 您的配置文件中的列配置存在一定的风险. 因为您未配置读取数据库表的列，当您的表字段个数、类型有变动时，可能影响任务正确性甚至会运行出错。请检查您的配置并作出修改.
2023-07-10 20:56:54.951 [ job-0] INFO OriginalConfPretreatmentUtil - table:[dwd_ads] all columns:[idate,campaign_id,campaign_name,adset_id,adset_name,ad_id,ad_name,media_source,account_id,country,product,is_new,data_type,terminal,book_id,ad_code,creator_code,language,branch_type,cost_type,impressions,clicks,reach,frequency,link_clicks,install_users,attr_users,engagement_users,cost,r0,r1,r2,r3,r4,r5,r6,r7,r8,r9,r10,r11,r12,r13,r14,r15,r16,r17,r18,r19,r20,r21,r22,r23,r24,r25,r26,r27,r28,r29,r30,weekly_cumulative_revenue,data_source,monthly_cumulative_revenue].
2023-07-10 20:56:54.951 [ job-0] WARN OriginalConfPretreatmentUtil - 您的配置文件中的列配置信息存在风险. 因为您配置的写入数据库表的列为*，当您的表字段个数、类型有变动时，可能影响任务正确性甚至会运行出错。请检查您的配置并作出修改.
2023-07-10 20:56:54.952 [ job-0] INFO OriginalConfPretreatmentUtil - Write data [INSERT INTO %s ( idate,campaign_id,campaign_name,adset_id,adset_name,ad_id,ad_name,media_source,account_id,country,product,is_new,data_type,terminal,book_id,ad_code,creator_code,language,branch_type,cost_type,impressions,clicks,reach,frequency,link_clicks,install_users,attr_users,engagement_users,cost,r0,r1,r2,r3,r4,r5,r6,r7,r8,r9,r10,r11,r12,r13,r14,r15,r16,r17,r18,r19,r20,r21,r22,r23,r24,r25,r26,r27,r28,r29,r30,weekly_cumulative_revenue,data_source,monthly_cumulative_revenue) VALUES ( ?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,? )], which jdbcUrl [jdbc:clickhouse://172.26.0.237:8123/tmp]
2023-07-10 20:56:54.952 [ job-0] INFO JobContainer - The Reader.Job [clickhousereader] perform prepare work .
2023-07-10 20:56:54.953 [ job-0] INFO JobContainer - The Writer.Job [clickhousewriter] perform prepare work .
2023-07-10 20:56:54.953 [ job-0] INFO JobContainer - Job set Channel-Number to 3 channel(s).
2023-07-10 20:56:54.954 [ job-0] INFO JobContainer - The Reader.Job [clickhousereader] is divided into [1] task(s).
2023-07-10 20:56:54.976 [ job-0] INFO JobContainer - The Writer.Job [clickhousewriter] is divided into [1] task(s).
2023-07-10 20:56:54.992 [ job-0] INFO JobContainer - The Scheduler launches [1] taskGroup(s).
2023-07-10 20:56:54.999 [ taskGroup-0] INFO TaskGroupContainer - The taskGroupId=[0] started [1] channels for [1] tasks.
2023-07-10 20:56:55.000 [ taskGroup-0] INFO Channel - The Channel set byte_speed_limit to 10000000.
2023-07-10 20:56:55.001 [ taskGroup-0] INFO Channel - The Channel set record_speed_limit to -1, No tps activated.
2023-07-10 20:56:55.008 [ reader-0-0] INFO CommonRdbmsReader$Task - Begin reading records by executing SQL query: [select * from dwd_ads ].
2023-07-10 20:56:55.813 [ reader-0-0] WARN ClickHouseConnectionImpl - [JDBC Compliant Mode] Transaction is not supported. Change jdbcCompliant to false to throw SQLException instead.
2023-07-10 20:56:56.906 [ writer-0-0] INFO CommonRdbmsWriter$Task - insert sql: insert into dwd_ads select idate,campaign_id,campaign_name,adset_id,adset_name,ad_id,ad_name,media_source,account_id,country,product,is_new,data_type,terminal,book_id,ad_code,creator_code,language,branch_type,cost_type,impressions,clicks,reach,frequency,link_clicks,install_users,attr_users,engagement_users,cost,r0,r1,r2,r3,r4,r5,r6,r7,r8,r9,r10,r11,r12,r13,r14,r15,r16,r17,r18,r19,r20,r21,r22,r23,r24,r25,r26,r27,r28,r29,r30,weekly_cumulative_revenue,data_source,monthly_cumulative_revenue from input('idate Date32,campaign_id String,campaign_name String,adset_id String,adset_name String,ad_id String,ad_name String,media_source String,account_id String,country String,product String,is_new String,data_type String,terminal String,book_id String,ad_code String,creator_code String,language String,branch_type String,cost_type String,impressions Int32,clicks Int32,reach Int32,frequency Decimal(32, 10),link_clicks Int32,install_users Int32,attr_users Int32,engagement_users Int32,cost Decimal(32, 10),r0 String,r1 String,r2 String,r3 String,r4 String,r5 String,r6 String,r7 String,r8 String,r9 String,r10 String,r11 String,r12 String,r13 String,r14 String,r15 String,r16 String,r17 String,r18 String,r19 String,r20 String,r21 String,r22 String,r23 String,r24 String,r25 String,r26 String,r27 String,r28 String,r29 String,r30 String,weekly_cumulative_revenue String,data_source Int32,monthly_cumulative_revenue String')
2023-07-10 20:56:56.991 [ writer-0-0] ERROR WriterRunner - Writer Runner Received Exceptions:
com.wgzhao.addax.common.exception.AddaxException: java.sql.SQLException: Code: 49. DB::Exception: Too large size (9223372036854775808) passed to allocator. It indicates an error.: While executing BinaryRowInputFormat. (LOGICAL_ERROR) (version 22.8.4.7 (official build))
, server ClickHouseNode(addr=http:172.26.0.237:8123, db=tmp)@-946000314
at com.clickhouse.jdbc.SqlExceptionUtils.handle(SqlExceptionUtils.java:54)
at com.clickhouse.jdbc.SqlExceptionUtils.handle(SqlExceptionUtils.java:69)
at com.clickhouse.jdbc.internal.ClickHouseStatementImpl.executeInsert(ClickHouseStatementImpl.java:161)
at com.clickhouse.jdbc.internal.InputBasedPreparedStatement.executeBatch(InputBasedPreparedStatement.java:261)
at org.apache.commons.dbcp2.DelegatingStatement.executeBatch(DelegatingStatement.java:241)
at org.apache.commons.dbcp2.DelegatingStatement.executeBatch(DelegatingStatement.java:241)
at com.wgzhao.addax.plugin.writer.clickhousewriter.ClickHouseWriter$Task$1.doBatchInsert(ClickHouseWriter.java:146)
at com.wgzhao.addax.rdbms.writer.CommonRdbmsWriter$Task.startWriteWithConnection(CommonRdbmsWriter.java:316)
at com.wgzhao.addax.rdbms.writer.CommonRdbmsWriter$Task.startWrite(CommonRdbmsWriter.java:340)
at com.wgzhao.addax.plugin.writer.clickhousewriter.ClickHouseWriter$Task.startWrite(ClickHouseWriter.java:163)
at com.wgzhao.addax.core.taskgroup.runner.WriterRunner.run(WriterRunner.java:81)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.IOException: Code: 49. DB::Exception: Too large size (9223372036854775808) passed to allocator. It indicates an error.: While executing BinaryRowInputFormat. (LOGICAL_ERROR) (version 22.8.4.7 (official build))
at com.clickhouse.client.http.HttpUrlConnectionImpl.checkResponse(HttpUrlConnectionImpl.java:130)
at com.clickhouse.client.http.HttpUrlConnectionImpl.post(HttpUrlConnectionImpl.java:201)
at com.clickhouse.client.http.ClickHouseHttpClient.postRequest(ClickHouseHttpClient.java:88)
at com.clickhouse.client.http.ClickHouseHttpClient.execute(ClickHouseHttpClient.java:115)
at com.clickhouse.client.ClickHouseRequest.execute(ClickHouseRequest.java:1375)
at com.clickhouse.jdbc.internal.ClickHouseStatementImpl.executeInsert(ClickHouseStatementImpl.java:152)
... 9 more
at com.wgzhao.addax.common.exception.AddaxException.asAddaxException(AddaxException.java:66)
at com.wgzhao.addax.rdbms.writer.CommonRdbmsWriter$Task.startWriteWithConnection(CommonRdbmsWriter.java:327)
at com.wgzhao.addax.rdbms.writer.CommonRdbmsWriter$Task.startWrite(CommonRdbmsWriter.java:340)
at com.wgzhao.addax.plugin.writer.clickhousewriter.ClickHouseWriter$Task.startWrite(ClickHouseWriter.java:163)
at com.wgzhao.addax.core.taskgroup.runner.WriterRunner.run(WriterRunner.java:81)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.sql.SQLException: Code: 49. DB::Exception: Too large size (9223372036854775808) passed to allocator. It indicates an error.: While executing BinaryRowInputFormat. (LOGICAL_ERROR) (version 22.8.4.7 (official build))
, server ClickHouseNode(addr=http:172.26.0.237:8123, db=tmp)@-946000314
at com.clickhouse.jdbc.SqlExceptionUtils.handle(SqlExceptionUtils.java:54)
at com.clickhouse.jdbc.SqlExceptionUtils.handle(SqlExceptionUtils.java:69)
at com.clickhouse.jdbc.internal.ClickHouseStatementImpl.executeInsert(ClickHouseStatementImpl.java:161)
at com.clickhouse.jdbc.internal.InputBasedPreparedStatement.executeBatch(InputBasedPreparedStatement.java:261)
at org.apache.commons.dbcp2.DelegatingStatement.executeBatch(DelegatingStatement.java:241)
at org.apache.commons.dbcp2.DelegatingStatement.executeBatch(DelegatingStatement.java:241)
at com.wgzhao.addax.plugin.writer.clickhousewriter.ClickHouseWriter$Task$1.doBatchInsert(ClickHouseWriter.java:146)
at com.wgzhao.addax.rdbms.writer.CommonRdbmsWriter$Task.startWriteWithConnection(CommonRdbmsWriter.java:316)
... 4 common frames omitted
Caused by: java.io.IOException: Code: 49. DB::Exception: Too large size (9223372036854775808) passed to allocator. It indicates an error.: While executing BinaryRowInputFormat. (LOGICAL_ERROR) (version 22.8.4.7 (official build))
at com.clickhouse.client.http.HttpUrlConnectionImpl.checkResponse(HttpUrlConnectionImpl.java:130)
at com.clickhouse.client.http.HttpUrlConnectionImpl.post(HttpUrlConnectionImpl.java:201)
at com.clickhouse.client.http.ClickHouseHttpClient.postRequest(ClickHouseHttpClient.java:88)
at com.clickhouse.client.http.ClickHouseHttpClient.execute(ClickHouseHttpClient.java:115)
at com.clickhouse.client.ClickHouseRequest.execute(ClickHouseRequest.java:1375)
at com.clickhouse.jdbc.internal.ClickHouseStatementImpl.executeInsert(ClickHouseStatementImpl.java:152)
... 9 common frames omitted
Exception in thread "taskGroup-0" com.wgzhao.addax.common.exception.AddaxException: java.sql.SQLException: Code: 49. DB::Exception: Too large size (9223372036854775808) passed to allocator. It indicates an error.: While executing BinaryRowInputFormat. (LOGICAL_ERROR) (version 22.8.4.7 (official build))
, server ClickHouseNode(addr=http:172.26.0.237:8123, db=tmp)@-946000314
at com.clickhouse.jdbc.SqlExceptionUtils.handle(SqlExceptionUtils.java:54)
at com.clickhouse.jdbc.SqlExceptionUtils.handle(SqlExceptionUtils.java:69)
at com.clickhouse.jdbc.internal.ClickHouseStatementImpl.executeInsert(ClickHouseStatementImpl.java:161)
at com.clickhouse.jdbc.internal.InputBasedPreparedStatement.executeBatch(InputBasedPreparedStatement.java:261)
at org.apache.commons.dbcp2.DelegatingStatement.executeBatch(DelegatingStatement.java:241)
at org.apache.commons.dbcp2.DelegatingStatement.executeBatch(DelegatingStatement.java:241)
at com.wgzhao.addax.plugin.writer.clickhousewriter.ClickHouseWriter$Task$1.doBatchInsert(ClickHouseWriter.java:146)
at com.wgzhao.addax.rdbms.writer.CommonRdbmsWriter$Task.startWriteWithConnection(CommonRdbmsWriter.java:316)
at com.wgzhao.addax.rdbms.writer.CommonRdbmsWriter$Task.startWrite(CommonRdbmsWriter.java:340)
at com.wgzhao.addax.plugin.writer.clickhousewriter.ClickHouseWriter$Task.startWrite(ClickHouseWriter.java:163)
at com.wgzhao.addax.core.taskgroup.runner.WriterRunner.run(WriterRunner.java:81)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.IOException: Code: 49. DB::Exception: Too large size (9223372036854775808) passed to allocator. It indicates an error.: While executing BinaryRowInputFormat. (LOGICAL_ERROR) (version 22.8.4.7 (official build))
at com.clickhouse.client.http.HttpUrlConnectionImpl.checkResponse(HttpUrlConnectionImpl.java:130)
at com.clickhouse.client.http.HttpUrlConnectionImpl.post(HttpUrlConnectionImpl.java:201)
at com.clickhouse.client.http.ClickHouseHttpClient.postRequest(ClickHouseHttpClient.java:88)
at com.clickhouse.client.http.ClickHouseHttpClient.execute(ClickHouseHttpClient.java:115)
at com.clickhouse.client.ClickHouseRequest.execute(ClickHouseRequest.java:1375)
at com.clickhouse.jdbc.internal.ClickHouseStatementImpl.executeInsert(ClickHouseStatementImpl.java:152)
... 9 more
at com.wgzhao.addax.common.exception.AddaxException.asAddaxException(AddaxException.java:66)
at com.wgzhao.addax.rdbms.writer.CommonRdbmsWriter$Task.startWriteWithConnection(CommonRdbmsWriter.java:327)
at com.wgzhao.addax.rdbms.writer.CommonRdbmsWriter$Task.startWrite(CommonRdbmsWriter.java:340)
at com.wgzhao.addax.plugin.writer.clickhousewriter.ClickHouseWriter$Task.startWrite(ClickHouseWriter.java:163)
at com.wgzhao.addax.core.taskgroup.runner.WriterRunner.run(WriterRunner.java:81)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.sql.SQLException: Code: 49. DB::Exception: Too large size (9223372036854775808) passed to allocator. It indicates an error.: While executing BinaryRowInputFormat. (LOGICAL_ERROR) (version 22.8.4.7 (official build))
, server ClickHouseNode(addr=http:172.26.0.237:8123, db=tmp)@-946000314
at com.clickhouse.jdbc.SqlExceptionUtils.handle(SqlExceptionUtils.java:54)
at com.clickhouse.jdbc.SqlExceptionUtils.handle(SqlExceptionUtils.java:69)
at com.clickhouse.jdbc.internal.ClickHouseStatementImpl.executeInsert(ClickHouseStatementImpl.java:161)
at com.clickhouse.jdbc.internal.InputBasedPreparedStatement.executeBatch(InputBasedPreparedStatement.java:261)
at org.apache.commons.dbcp2.DelegatingStatement.executeBatch(DelegatingStatement.java:241)
at org.apache.commons.dbcp2.DelegatingStatement.executeBatch(DelegatingStatement.java:241)
at com.wgzhao.addax.plugin.writer.clickhousewriter.ClickHouseWriter$Task$1.doBatchInsert(ClickHouseWriter.java:146)
at com.wgzhao.addax.rdbms.writer.CommonRdbmsWriter$Task.startWriteWithConnection(CommonRdbmsWriter.java:316)
... 4 more
Caused by: java.io.IOException: Code: 49. DB::Exception: Too large size (9223372036854775808) passed to allocator. It indicates an error.: While executing BinaryRowInputFormat. (LOGICAL_ERROR) (version 22.8.4.7 (official build))
at com.clickhouse.client.http.HttpUrlConnectionImpl.checkResponse(HttpUrlConnectionImpl.java:130)
at com.clickhouse.client.http.HttpUrlConnectionImpl.post(HttpUrlConnectionImpl.java:201)
at com.clickhouse.client.http.ClickHouseHttpClient.postRequest(ClickHouseHttpClient.java:88)
at com.clickhouse.client.http.ClickHouseHttpClient.execute(ClickHouseHttpClient.java:115)
at com.clickhouse.client.ClickHouseRequest.execute(ClickHouseRequest.java:1375)
at com.clickhouse.jdbc.internal.ClickHouseStatementImpl.executeInsert(ClickHouseStatementImpl.java:152)
... 9 more
2023-07-10 20:56:58.006 [ job-0] ERROR JobContainer - The scheduler failed to run.
2023-07-10 20:56:58.010 [ job-0] INFO StandAloneJobContainerCommunicator - Total 2560 records, 1191082 bytes | Speed 1.14MB/s, 2560 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.001s | All Task WaitReaderTime 1.076s | Percentage 0.00%
com.wgzhao.addax.common.exception.AddaxException: java.sql.SQLException: Code: 49. DB::Exception: Too large size (9223372036854775808) passed to allocator. It indicates an error.: While executing BinaryRowInputFormat. (LOGICAL_ERROR) (version 22.8.4.7 (official build))
, server ClickHouseNode(addr=http:172.26.0.237:8123, db=tmp)@-946000314
at com.clickhouse.jdbc.SqlExceptionUtils.handle(SqlExceptionUtils.java:54)
at com.clickhouse.jdbc.SqlExceptionUtils.handle(SqlExceptionUtils.java:69)
at com.clickhouse.jdbc.internal.ClickHouseStatementImpl.executeInsert(ClickHouseStatementImpl.java:161)
at com.clickhouse.jdbc.internal.InputBasedPreparedStatement.executeBatch(InputBasedPreparedStatement.java:261)
at org.apache.commons.dbcp2.DelegatingStatement.executeBatch(DelegatingStatement.java:241)
at org.apache.commons.dbcp2.DelegatingStatement.executeBatch(DelegatingStatement.java:241)
at com.wgzhao.addax.plugin.writer.clickhousewriter.ClickHouseWriter$Task$1.doBatchInsert(ClickHouseWriter.java:146)
at com.wgzhao.addax.rdbms.writer.CommonRdbmsWriter$Task.startWriteWithConnection(CommonRdbmsWriter.java:316)
at com.wgzhao.addax.rdbms.writer.CommonRdbmsWriter$Task.startWrite(CommonRdbmsWriter.java:340)
at com.wgzhao.addax.plugin.writer.clickhousewriter.ClickHouseWriter$Task.startWrite(ClickHouseWriter.java:163)
at com.wgzhao.addax.core.taskgroup.runner.WriterRunner.run(WriterRunner.java:81)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.IOException: Code: 49. DB::Exception: Too large size (9223372036854775808) passed to allocator. It indicates an error.: While executing BinaryRowInputFormat. (LOGICAL_ERROR) (version 22.8.4.7 (official build))
at com.clickhouse.client.http.HttpUrlConnectionImpl.checkResponse(HttpUrlConnectionImpl.java:130)
at com.clickhouse.client.http.HttpUrlConnectionImpl.post(HttpUrlConnectionImpl.java:201)
at com.clickhouse.client.http.ClickHouseHttpClient.postRequest(ClickHouseHttpClient.java:88)
at com.clickhouse.client.http.ClickHouseHttpClient.execute(ClickHouseHttpClient.java:115)
at com.clickhouse.client.ClickHouseRequest.execute(ClickHouseRequest.java:1375)
at com.clickhouse.jdbc.internal.ClickHouseStatementImpl.executeInsert(ClickHouseStatementImpl.java:152)
... 9 more
at com.wgzhao.addax.common.exception.AddaxException.asAddaxException(AddaxException.java:66)
at com.wgzhao.addax.rdbms.writer.CommonRdbmsWriter$Task.startWriteWithConnection(CommonRdbmsWriter.java:327)
at com.wgzhao.addax.rdbms.writer.CommonRdbmsWriter$Task.startWrite(CommonRdbmsWriter.java:340)
at com.wgzhao.addax.plugin.writer.clickhousewriter.ClickHouseWriter$Task.startWrite(ClickHouseWriter.java:163)
at com.wgzhao.addax.core.taskgroup.runner.WriterRunner.run(WriterRunner.java:81)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.sql.SQLException: Code: 49. DB::Exception: Too large size (9223372036854775808) passed to allocator. It indicates an error.: While executing BinaryRowInputFormat. (LOGICAL_ERROR) (version 22.8.4.7 (official build))
, server ClickHouseNode(addr=http:172.26.0.237:8123, db=tmp)@-946000314
at com.clickhouse.jdbc.SqlExceptionUtils.handle(SqlExceptionUtils.java:54)
at com.clickhouse.jdbc.SqlExceptionUtils.handle(SqlExceptionUtils.java:69)
at com.clickhouse.jdbc.internal.ClickHouseStatementImpl.executeInsert(ClickHouseStatementImpl.java:161)
at com.clickhouse.jdbc.internal.InputBasedPreparedStatement.executeBatch(InputBasedPreparedStatement.java:261)
at org.apache.commons.dbcp2.DelegatingStatement.executeBatch(DelegatingStatement.java:241)
at org.apache.commons.dbcp2.DelegatingStatement.executeBatch(DelegatingStatement.java:241)
at com.wgzhao.addax.plugin.writer.clickhousewriter.ClickHouseWriter$Task$1.doBatchInsert(ClickHouseWriter.java:146)
at com.wgzhao.addax.rdbms.writer.CommonRdbmsWriter$Task.startWriteWithConnection(CommonRdbmsWriter.java:316)
... 4 more
Caused by: java.io.IOException: Code: 49. DB::Exception: Too large size (9223372036854775808) passed to allocator. It indicates an error.: While executing BinaryRowInputFormat. (LOGICAL_ERROR) (version 22.8.4.7 (official build))
at com.clickhouse.client.http.HttpUrlConnectionImpl.checkResponse(HttpUrlConnectionImpl.java:130)
at com.clickhouse.client.http.HttpUrlConnectionImpl.post(HttpUrlConnectionImpl.java:201)
at com.clickhouse.client.http.ClickHouseHttpClient.postRequest(ClickHouseHttpClient.java:88)
at com.clickhouse.client.http.ClickHouseHttpClient.execute(ClickHouseHttpClient.java:115)
at com.clickhouse.client.ClickHouseRequest.execute(ClickHouseRequest.java:1375)
at com.clickhouse.jdbc.internal.ClickHouseStatementImpl.executeInsert(ClickHouseStatementImpl.java:152)
... 9 morefrom addax.
wgzhao
commented on August 19, 2024
从报错信息来看
DB::Exception: Too large size (9223372036854775808) passed to allocator.
这是查询的数据过大导致内存不够，比较好奇怎么会有 9223372036854775808 这么大的一个数据，换算下来是 8192EB 了
from addax.
sxfang32
commented on August 19, 2024
感谢回复！
对了一下表结构，只有可能是广告id这个字段和上述的值比较接近。但是我比较好奇的是，广告id我是设置成了String类型，应该不会变成一个Int32或者Decimal？
另外我试了一张更小的表，尝试做数据抽取时同样也报错了，感觉可能是同一类问题？报错信息如下：
==================== DEPRECATED WARNING ========================
addax.py is deprecated, It's going to be removed in future release.
As a replacement, you can use addax.sh to run job
==================== DEPRECATED WARNING ========================
___ _ _
/ _ \ | | | |
/ /_\ \ __| | __| | __ ___ __
| _ |/ _` |/ _` |/ _` \ \/ /
| | | | (_| | (_| | (_| |> <
\_| |_/\__,_|\__,_|\__,_/_/\_\
:: Addax version :: (v4.1.0)
2023-07-11 13:45:16.779 [ main] INFO VMInfo - VMInfo# operatingSystem class => sun.management.OperatingSystemImpl
2023-07-11 13:45:16.791 [ main] INFO Engine -
{
"setting":{
"speed":{
"channel":3
},
"errorLimit":{
"record":0,
"percentage":0.02
}
},
"content":{
"reader":{
"name":"clickhousereader",
"parameter":{
"username":"***",
"password":"*****",
"column":[
"*"
],
"connection":[
{
"jdbcUrl":[
"jdbc:clickhouse://172.31.163.36:8123/Marketing_DWM_DT"
],
"table":[
"dwd_ads_novel"
]
}
]
}
},
"writer":{
"name":"clickhousewriter",
"parameter":{
"username":"***",
"password":"*****",
"column":[
"*"
],
"connection":[
{
"table":[
"dwd_ads_novel"
],
"jdbcUrl":"jdbc:clickhouse://172.26.0.237:8123/tmp"
}
]
}
}
}
}
2023-07-11 13:45:16.832 [ main] INFO JobContainer - The jobContainer begins to process the job.
2023-07-11 13:45:19.079 [ job-0] INFO OriginalConfPretreatmentUtil - Available jdbcUrl:jdbc:clickhouse://172.31.163.36:8123/Marketing_DWM_DT.
2023-07-11 13:45:19.081 [ job-0] WARN OriginalConfPretreatmentUtil - 您的配置文件中的列配置存在一定的风险. 因为您未配置读取数据库表的列，当您的表字段个数、类型有变动时，可能影响任务正确性甚至会运行出错。请检查您的配置并作出修改.
2023-07-11 13:45:19.433 [ job-0] INFO OriginalConfPretreatmentUtil - table:[dwd_ads_novel] all columns:[idate,campaign_id,campaign_name,adset_id,adset_name,ad_id,ad_name,media_source,account_id,country,product,is_new,data_type,terminal,book_id,ad_code,creator_code,language,branch_type,cost_type,is_test,department,impressions,clicks,reach,frequency,link_clicks,install_users,attr_users,engagement_users,cost,r0,r1,r2,r3,r4,r5,r6,r7,r8,r9,r10,r11,r12,r13,r14,r15,r16,r17,r18,r19,r20,r21,r22,r23,r24,r25,r26,r27,r28,r29,r30,weekly_cumulative_revenue,data_source,monthly_cumulative_revenue].
2023-07-11 13:45:19.434 [ job-0] WARN OriginalConfPretreatmentUtil - 您的配置文件中的列配置信息存在风险. 因为您配置的写入数据库表的列为*，当您的表字段个数、类型有变动时，可能影响任务正确性甚至会运行出错。请检查您的配置并作出修改.
2023-07-11 13:45:19.444 [ job-0] INFO OriginalConfPretreatmentUtil - Write data [INSERT INTO %s ( idate,campaign_id,campaign_name,adset_id,adset_name,ad_id,ad_name,media_source,account_id,country,product,is_new,data_type,terminal,book_id,ad_code,creator_code,language,branch_type,cost_type,is_test,department,impressions,clicks,reach,frequency,link_clicks,install_users,attr_users,engagement_users,cost,r0,r1,r2,r3,r4,r5,r6,r7,r8,r9,r10,r11,r12,r13,r14,r15,r16,r17,r18,r19,r20,r21,r22,r23,r24,r25,r26,r27,r28,r29,r30,weekly_cumulative_revenue,data_source,monthly_cumulative_revenue) VALUES ( ?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,? )], which jdbcUrl [jdbc:clickhouse://172.26.0.237:8123/tmp]
2023-07-11 13:45:19.444 [ job-0] INFO JobContainer - The Reader.Job [clickhousereader] perform prepare work .
2023-07-11 13:45:19.444 [ job-0] INFO JobContainer - The Writer.Job [clickhousewriter] perform prepare work .
2023-07-11 13:45:19.445 [ job-0] INFO JobContainer - Job set Channel-Number to 3 channel(s).
2023-07-11 13:45:19.484 [ job-0] INFO JobContainer - The Reader.Job [clickhousereader] is divided into [1] task(s).
2023-07-11 13:45:19.485 [ job-0] INFO JobContainer - The Writer.Job [clickhousewriter] is divided into [1] task(s).
2023-07-11 13:45:19.501 [ job-0] INFO JobContainer - The Scheduler launches [1] taskGroup(s).
2023-07-11 13:45:19.508 [ taskGroup-0] INFO TaskGroupContainer - The taskGroupId=[0] started [1] channels for [1] tasks.
2023-07-11 13:45:19.510 [ taskGroup-0] INFO Channel - The Channel set byte_speed_limit to 10000000.
2023-07-11 13:45:19.510 [ taskGroup-0] INFO Channel - The Channel set record_speed_limit to -1, No tps activated.
2023-07-11 13:45:19.520 [ reader-0-0] INFO CommonRdbmsReader$Task - Begin reading records by executing SQL query: [select * from dwd_ads_novel ].
2023-07-11 13:45:20.319 [ reader-0-0] WARN ClickHouseConnectionImpl - [JDBC Compliant Mode] Transaction is not supported. Change jdbcCompliant to false to throw SQLException instead.
2023-07-11 13:45:21.562 [ writer-0-0] INFO CommonRdbmsWriter$Task - insert sql: insert into dwd_ads_novel select idate,campaign_id,campaign_name,adset_id,adset_name,ad_id,ad_name,media_source,account_id,country,product,is_new,data_type,terminal,book_id,ad_code,creator_code,language,branch_type,cost_type,is_test,department,impressions,clicks,reach,frequency,link_clicks,install_users,attr_users,engagement_users,cost,r0,r1,r2,r3,r4,r5,r6,r7,r8,r9,r10,r11,r12,r13,r14,r15,r16,r17,r18,r19,r20,r21,r22,r23,r24,r25,r26,r27,r28,r29,r30,weekly_cumulative_revenue,data_source,monthly_cumulative_revenue from input('idate Date32,campaign_id String,campaign_name String,adset_id String,adset_name String,ad_id String,ad_name String,media_source String,account_id String,country String,product String,is_new String,data_type String,terminal String,book_id String,ad_code String,creator_code String,language String,branch_type String,cost_type String,is_test String,department String,impressions Int32,clicks Int32,reach Int32,frequency Decimal(10, 6),link_clicks Int32,install_users Int32,attr_users Int32,engagement_users Int32,cost Decimal(10, 6),r0 Map(String, Decimal(10, 6)),r1 Map(String, Decimal(10, 6)),r2 Map(String, Decimal(10, 6)),r3 Map(String, Decimal(10, 2)),r4 Map(String, Decimal(10, 2)),r5 Map(String, Decimal(10, 2)),r6 Map(String, Decimal(10, 2)),r7 Map(String, Decimal(10, 2)),r8 Map(String, Decimal(10, 2)),r9 Map(String, Decimal(10, 2)),r10 Map(String, Decimal(10, 2)),r11 Map(String, Decimal(10, 2)),r12 Map(String, Decimal(10, 2)),r13 Map(String, Decimal(10, 2)),r14 Map(String, Decimal(10, 2)),r15 Map(String, Decimal(10, 2)),r16 Map(String, Decimal(10, 2)),r17 Map(String, Decimal(10, 2)),r18 Map(String, Decimal(10, 2)),r19 Map(String, Decimal(10, 2)),r20 Map(String, Decimal(10, 2)),r21 Map(String, Decimal(10, 2)),r22 Map(String, Decimal(10, 2)),r23 Map(String, Decimal(10, 2)),r24 Map(String, Decimal(10, 2)),r25 Map(String, Decimal(10, 2)),r26 Map(String, Decimal(10, 2)),r27 Map(String, Decimal(10, 2)),r28 Map(String, Decimal(10, 2)),r29 Map(String, Decimal(10, 2)),r30 Map(String, Decimal(10, 2)),weekly_cumulative_revenue Map(String, Decimal(10, 2)),data_source Int32,monthly_cumulative_revenue Map(String, Decimal(10, 6))')
2023-07-11 13:45:21.575 [ writer-0-0] ERROR WriterRunner - Writer Runner Received Exceptions:
com.wgzhao.addax.common.exception.AddaxException: java.lang.ClassCastException: Cannot cast java.lang.String to java.math.BigDecimal
at java.lang.Class.cast(Class.java:3369)
at com.clickhouse.client.data.ClickHouseMapValue.update(ClickHouseMapValue.java:251)
at com.clickhouse.client.data.ClickHouseMapValue.update(ClickHouseMapValue.java:22)
at com.clickhouse.client.ClickHouseValue.update(ClickHouseValue.java:1202)
at com.clickhouse.client.data.ClickHouseObjectValue.update(ClickHouseObjectValue.java:144)
at com.clickhouse.client.data.ClickHouseMapValue.update(ClickHouseMapValue.java:364)
at com.clickhouse.jdbc.internal.InputBasedPreparedStatement.setObject(InputBasedPreparedStatement.java:392)
at com.clickhouse.jdbc.ClickHousePreparedStatement.setObject(ClickHousePreparedStatement.java:68)
at org.apache.commons.dbcp2.DelegatingPreparedStatement.setObject(DelegatingPreparedStatement.java:529)
at org.apache.commons.dbcp2.DelegatingPreparedStatement.setObject(DelegatingPreparedStatement.java:529)
at com.wgzhao.addax.rdbms.writer.CommonRdbmsWriter$Task.fillPreparedStatementColumnType(CommonRdbmsWriter.java:560)
at com.wgzhao.addax.plugin.writer.clickhousewriter.ClickHouseWriter$Task$1.fillPreparedStatementColumnType(ClickHouseWriter.java:123)
at com.wgzhao.addax.rdbms.writer.CommonRdbmsWriter$Task.fillPreparedStatement(CommonRdbmsWriter.java:457)
at com.wgzhao.addax.plugin.writer.clickhousewriter.ClickHouseWriter$Task$1.doBatchInsert(ClickHouseWriter.java:143)
at com.wgzhao.addax.rdbms.writer.CommonRdbmsWriter$Task.startWriteWithConnection(CommonRdbmsWriter.java:316)
at com.wgzhao.addax.rdbms.writer.CommonRdbmsWriter$Task.startWrite(CommonRdbmsWriter.java:340)
at com.wgzhao.addax.plugin.writer.clickhousewriter.ClickHouseWriter$Task.startWrite(ClickHouseWriter.java:163)
at com.wgzhao.addax.core.taskgroup.runner.WriterRunner.run(WriterRunner.java:81)
at java.lang.Thread.run(Thread.java:748)
at com.wgzhao.addax.common.exception.AddaxException.asAddaxException(AddaxException.java:66)
at com.wgzhao.addax.rdbms.writer.CommonRdbmsWriter$Task.startWriteWithConnection(CommonRdbmsWriter.java:327)
at com.wgzhao.addax.rdbms.writer.CommonRdbmsWriter$Task.startWrite(CommonRdbmsWriter.java:340)
at com.wgzhao.addax.plugin.writer.clickhousewriter.ClickHouseWriter$Task.startWrite(ClickHouseWriter.java:163)
at com.wgzhao.addax.core.taskgroup.runner.WriterRunner.run(WriterRunner.java:81)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.ClassCastException: Cannot cast java.lang.String to java.math.BigDecimal
at java.lang.Class.cast(Class.java:3369)
at com.clickhouse.client.data.ClickHouseMapValue.update(ClickHouseMapValue.java:251)
at com.clickhouse.client.data.ClickHouseMapValue.update(ClickHouseMapValue.java:22)
at com.clickhouse.client.ClickHouseValue.update(ClickHouseValue.java:1202)
at com.clickhouse.client.data.ClickHouseObjectValue.update(ClickHouseObjectValue.java:144)
at com.clickhouse.client.data.ClickHouseMapValue.update(ClickHouseMapValue.java:364)
at com.clickhouse.jdbc.internal.InputBasedPreparedStatement.setObject(InputBasedPreparedStatement.java:392)
at com.clickhouse.jdbc.ClickHousePreparedStatement.setObject(ClickHousePreparedStatement.java:68)
at org.apache.commons.dbcp2.DelegatingPreparedStatement.setObject(DelegatingPreparedStatement.java:529)
at org.apache.commons.dbcp2.DelegatingPreparedStatement.setObject(DelegatingPreparedStatement.java:529)
at com.wgzhao.addax.rdbms.writer.CommonRdbmsWriter$Task.fillPreparedStatementColumnType(CommonRdbmsWriter.java:560)
at com.wgzhao.addax.plugin.writer.clickhousewriter.ClickHouseWriter$Task$1.fillPreparedStatementColumnType(ClickHouseWriter.java:123)
at com.wgzhao.addax.rdbms.writer.CommonRdbmsWriter$Task.fillPreparedStatement(CommonRdbmsWriter.java:457)
at com.wgzhao.addax.plugin.writer.clickhousewriter.ClickHouseWriter$Task$1.doBatchInsert(ClickHouseWriter.java:143)
at com.wgzhao.addax.rdbms.writer.CommonRdbmsWriter$Task.startWriteWithConnection(CommonRdbmsWriter.java:316)
... 4 common frames omitted
Exception in thread "taskGroup-0" com.wgzhao.addax.common.exception.AddaxException: java.lang.ClassCastException: Cannot cast java.lang.String to java.math.BigDecimal
at java.lang.Class.cast(Class.java:3369)
at com.clickhouse.client.data.ClickHouseMapValue.update(ClickHouseMapValue.java:251)
at com.clickhouse.client.data.ClickHouseMapValue.update(ClickHouseMapValue.java:22)
at com.clickhouse.client.ClickHouseValue.update(ClickHouseValue.java:1202)
at com.clickhouse.client.data.ClickHouseObjectValue.update(ClickHouseObjectValue.java:144)
at com.clickhouse.client.data.ClickHouseMapValue.update(ClickHouseMapValue.java:364)
at com.clickhouse.jdbc.internal.InputBasedPreparedStatement.setObject(InputBasedPreparedStatement.java:392)
at com.clickhouse.jdbc.ClickHousePreparedStatement.setObject(ClickHousePreparedStatement.java:68)
at org.apache.commons.dbcp2.DelegatingPreparedStatement.setObject(DelegatingPreparedStatement.java:529)
at org.apache.commons.dbcp2.DelegatingPreparedStatement.setObject(DelegatingPreparedStatement.java:529)
at com.wgzhao.addax.rdbms.writer.CommonRdbmsWriter$Task.fillPreparedStatementColumnType(CommonRdbmsWriter.java:560)
at com.wgzhao.addax.plugin.writer.clickhousewriter.ClickHouseWriter$Task$1.fillPreparedStatementColumnType(ClickHouseWriter.java:123)
at com.wgzhao.addax.rdbms.writer.CommonRdbmsWriter$Task.fillPreparedStatement(CommonRdbmsWriter.java:457)
at com.wgzhao.addax.plugin.writer.clickhousewriter.ClickHouseWriter$Task$1.doBatchInsert(ClickHouseWriter.java:143)
at com.wgzhao.addax.rdbms.writer.CommonRdbmsWriter$Task.startWriteWithConnection(CommonRdbmsWriter.java:316)
at com.wgzhao.addax.rdbms.writer.CommonRdbmsWriter$Task.startWrite(CommonRdbmsWriter.java:340)
at com.wgzhao.addax.plugin.writer.clickhousewriter.ClickHouseWriter$Task.startWrite(ClickHouseWriter.java:163)
at com.wgzhao.addax.core.taskgroup.runner.WriterRunner.run(WriterRunner.java:81)
at java.lang.Thread.run(Thread.java:748)
at com.wgzhao.addax.common.exception.AddaxException.asAddaxException(AddaxException.java:66)
at com.wgzhao.addax.rdbms.writer.CommonRdbmsWriter$Task.startWriteWithConnection(CommonRdbmsWriter.java:327)
at com.wgzhao.addax.rdbms.writer.CommonRdbmsWriter$Task.startWrite(CommonRdbmsWriter.java:340)
at com.wgzhao.addax.plugin.writer.clickhousewriter.ClickHouseWriter$Task.startWrite(ClickHouseWriter.java:163)
at com.wgzhao.addax.core.taskgroup.runner.WriterRunner.run(WriterRunner.java:81)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.ClassCastException: Cannot cast java.lang.String to java.math.BigDecimal
at java.lang.Class.cast(Class.java:3369)
at com.clickhouse.client.data.ClickHouseMapValue.update(ClickHouseMapValue.java:251)
at com.clickhouse.client.data.ClickHouseMapValue.update(ClickHouseMapValue.java:22)
at com.clickhouse.client.ClickHouseValue.update(ClickHouseValue.java:1202)
at com.clickhouse.client.data.ClickHouseObjectValue.update(ClickHouseObjectValue.java:144)
at com.clickhouse.client.data.ClickHouseMapValue.update(ClickHouseMapValue.java:364)
at com.clickhouse.jdbc.internal.InputBasedPreparedStatement.setObject(InputBasedPreparedStatement.java:392)
at com.clickhouse.jdbc.ClickHousePreparedStatement.setObject(ClickHousePreparedStatement.java:68)
at org.apache.commons.dbcp2.DelegatingPreparedStatement.setObject(DelegatingPreparedStatement.java:529)
at org.apache.commons.dbcp2.DelegatingPreparedStatement.setObject(DelegatingPreparedStatement.java:529)
at com.wgzhao.addax.rdbms.writer.CommonRdbmsWriter$Task.fillPreparedStatementColumnType(CommonRdbmsWriter.java:560)
at com.wgzhao.addax.plugin.writer.clickhousewriter.ClickHouseWriter$Task$1.fillPreparedStatementColumnType(ClickHouseWriter.java:123)
at com.wgzhao.addax.rdbms.writer.CommonRdbmsWriter$Task.fillPreparedStatement(CommonRdbmsWriter.java:457)
at com.wgzhao.addax.plugin.writer.clickhousewriter.ClickHouseWriter$Task$1.doBatchInsert(ClickHouseWriter.java:143)
at com.wgzhao.addax.rdbms.writer.CommonRdbmsWriter$Task.startWriteWithConnection(CommonRdbmsWriter.java:316)
... 4 more
2023-07-11 13:45:22.516 [ job-0] ERROR JobContainer - The scheduler failed to run.
2023-07-11 13:45:22.538 [ job-0] INFO StandAloneJobContainerCommunicator - Total 2560 records, 974555 bytes | Speed 951.71KB/s, 2560 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.001s | All Task WaitReaderTime 1.215s | Percentage 0.00%
com.wgzhao.addax.common.exception.AddaxException: java.lang.ClassCastException: Cannot cast java.lang.String to java.math.BigDecimal
at java.lang.Class.cast(Class.java:3369)
at com.clickhouse.client.data.ClickHouseMapValue.update(ClickHouseMapValue.java:251)
at com.clickhouse.client.data.ClickHouseMapValue.update(ClickHouseMapValue.java:22)
at com.clickhouse.client.ClickHouseValue.update(ClickHouseValue.java:1202)
at com.clickhouse.client.data.ClickHouseObjectValue.update(ClickHouseObjectValue.java:144)
at com.clickhouse.client.data.ClickHouseMapValue.update(ClickHouseMapValue.java:364)
at com.clickhouse.jdbc.internal.InputBasedPreparedStatement.setObject(InputBasedPreparedStatement.java:392)
at com.clickhouse.jdbc.ClickHousePreparedStatement.setObject(ClickHousePreparedStatement.java:68)
at org.apache.commons.dbcp2.DelegatingPreparedStatement.setObject(DelegatingPreparedStatement.java:529)
at org.apache.commons.dbcp2.DelegatingPreparedStatement.setObject(DelegatingPreparedStatement.java:529)
at com.wgzhao.addax.rdbms.writer.CommonRdbmsWriter$Task.fillPreparedStatementColumnType(CommonRdbmsWriter.java:560)
at com.wgzhao.addax.plugin.writer.clickhousewriter.ClickHouseWriter$Task$1.fillPreparedStatementColumnType(ClickHouseWriter.java:123)
at com.wgzhao.addax.rdbms.writer.CommonRdbmsWriter$Task.fillPreparedStatement(CommonRdbmsWriter.java:457)
at com.wgzhao.addax.plugin.writer.clickhousewriter.ClickHouseWriter$Task$1.doBatchInsert(ClickHouseWriter.java:143)
at com.wgzhao.addax.rdbms.writer.CommonRdbmsWriter$Task.startWriteWithConnection(CommonRdbmsWriter.java:316)
at com.wgzhao.addax.rdbms.writer.CommonRdbmsWriter$Task.startWrite(CommonRdbmsWriter.java:340)
at com.wgzhao.addax.plugin.writer.clickhousewriter.ClickHouseWriter$Task.startWrite(ClickHouseWriter.java:163)
at com.wgzhao.addax.core.taskgroup.runner.WriterRunner.run(WriterRunner.java:81)
at java.lang.Thread.run(Thread.java:748)
at com.wgzhao.addax.common.exception.AddaxException.asAddaxException(AddaxException.java:66)
at com.wgzhao.addax.rdbms.writer.CommonRdbmsWriter$Task.startWriteWithConnection(CommonRdbmsWriter.java:327)
at com.wgzhao.addax.rdbms.writer.CommonRdbmsWriter$Task.startWrite(CommonRdbmsWriter.java:340)
at com.wgzhao.addax.plugin.writer.clickhousewriter.ClickHouseWriter$Task.startWrite(ClickHouseWriter.java:163)
at com.wgzhao.addax.core.taskgroup.runner.WriterRunner.run(WriterRunner.java:81)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.ClassCastException: Cannot cast java.lang.String to java.math.BigDecimal
at java.lang.Class.cast(Class.java:3369)
at com.clickhouse.client.data.ClickHouseMapValue.update(ClickHouseMapValue.java:251)
at com.clickhouse.client.data.ClickHouseMapValue.update(ClickHouseMapValue.java:22)
at com.clickhouse.client.ClickHouseValue.update(ClickHouseValue.java:1202)
at com.clickhouse.client.data.ClickHouseObjectValue.update(ClickHouseObjectValue.java:144)
at com.clickhouse.client.data.ClickHouseMapValue.update(ClickHouseMapValue.java:364)
at com.clickhouse.jdbc.internal.InputBasedPreparedStatement.setObject(InputBasedPreparedStatement.java:392)
at com.clickhouse.jdbc.ClickHousePreparedStatement.setObject(ClickHousePreparedStatement.java:68)
at org.apache.commons.dbcp2.DelegatingPreparedStatement.setObject(DelegatingPreparedStatement.java:529)
at org.apache.commons.dbcp2.DelegatingPreparedStatement.setObject(DelegatingPreparedStatement.java:529)
at com.wgzhao.addax.rdbms.writer.CommonRdbmsWriter$Task.fillPreparedStatementColumnType(CommonRdbmsWriter.java:560)
at com.wgzhao.addax.plugin.writer.clickhousewriter.ClickHouseWriter$Task$1.fillPreparedStatementColumnType(ClickHouseWriter.java:123)
at com.wgzhao.addax.rdbms.writer.CommonRdbmsWriter$Task.fillPreparedStatement(CommonRdbmsWriter.java:457)
at com.wgzhao.addax.plugin.writer.clickhousewriter.ClickHouseWriter$Task$1.doBatchInsert(ClickHouseWriter.java:143)
at com.wgzhao.addax.rdbms.writer.CommonRdbmsWriter$Task.startWriteWithConnection(CommonRdbmsWriter.java:316)
... 4 morefrom addax.
wgzhao
commented on August 19, 2024
Cannot cast java.lang.String to java.math.BigDecimal
你源表的字段类型是字符类型，但是目标表的类型是数值类型，但是因为字符串数字太大导致类型转换失败了。
可以尝试把目标表的类型也改成字符串
from addax.
sxfang32
commented on August 19, 2024
好的，感谢作者！
from addax.
Related Issues (20)
- [Bug]: kafka writer 无法指定key HOT 3
- 关于influxDB2写入的疑惑 HOT 3
- [Bug]: 连接postgrsql失败 HOT 4
- [Bug]: windows调试源码包方法抛出'java.lang.NoClassDefFoundError' HOT 1
- 请问一下postgresqlreader支持querySql吗？ HOT 1
- [Bug]: 各个组件中的libs目录下的jar包与根目录lib包下的jar包冲突
- [Bug]: ftp文件过多程序崩溃 HOT 3
- [Bug]: java.lang.NoClassDefFoundError: org/apache/logging/log4j/spi/LoggerAdapter HOT 5
- [Bug]: 4.1.4版本中是否把Record中的removeColumn方法移除
- [Bug]: 数据从rdbms 写入到Doris 报错 HOT 5
- [Bug]: log4j-core-2.17.1.jar HOT 3
- [Bug]: MongoDB写入starrocks报错 HOT 2
- [Bug]: java往数据库中直接存为clob字段时，oracle会报ORA-01704问题：字符串文字过长。 HOT 9
- [Bug]: mongoDB 写入 mongoDB 报错：Command failed with error 18 (AuthenticationFailed) HOT 2
- [help wanted]: 任务结果上报使用详细描述 HOT 1
- 关于influxdb2同步influxdb2问题
- 关于influxdb2同步到influxdb2问题 HOT 1
- [Bug]: MongoDBReader连接复制集类型mongo报错连接超时 HOT 2
- [question]: Support for deployment via k8s HOT 1
- [Bug]: KafkaReader无法读取 HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from addax.