Comments (7)
是支持yarn-cluster 模式部署的
from chunjun.
提交任务的命令是?
from chunjun.
eg: sh flinkx.sh -mode yarnPer -job /opt/dtstack/tmp/sqlserver_sqlserver.txt -plugin /opt/dtstack/flinkx_171plugin -flinkconf /opt/dtstack/flink-1.7.1/conf -yarnconf /opt/dtstack/hadoop/etc/hadoop -confProp {"jobmanager.memory.mb":1276,"taskmanager.memory.mb":1276,"taskmanager.num":2,"taskmanager.slots":2} -flinkLibJar /opt/dtstack/flink-1.7.1/lib
from chunjun.
支持吗? 我看到ClusterClientFactory 里面并没有支持啊?
public static ClusterClient createClusterClient(LauncherOptions launcherOptions) throws Exception { String mode = launcherOptions.getMode(); if(mode.equals(ClusterMode.standalone.name())) { return createStandaloneClient(launcherOptions); } else if(mode.equals(ClusterMode.yarn.name())) { return createYarnClient(launcherOptions); } throw new IllegalArgumentException("Unsupported cluster client type: "); }
ClusterMode 里面只是定义了yarnPer 但是并没有用到?
from chunjun.
支持yarn-perjob这种模式提交任务的呀 不过我用的1.7这个分支
from chunjun.
支持yarn-perjob这种模式提交任务的呀 不过我用的1.7这个分支
切换到1.7可以了, 但是还是报以下错误
Exception in thread "main" org.apache.flink.shaded.jackson2.com.fasterxml.jackson.core.JsonParseException: Unrecognized token 'jobmanager': was expecting ('true', 'false' or 'null') at [Source: jobmanager\.memory\.mb:1276; line: 1, column: 11] at org.apache.flink.shaded.jackson2.com.fasterxml.jackson.core.JsonParser._constructError(JsonParser.java:1586) at org.apache.flink.shaded.jackson2.com.fasterxml.jackson.core.base.ParserMinimalBase._reportError(ParserMinimalBase.java:521) at org.apache.flink.shaded.jackson2.com.fasterxml.jackson.core.json.ReaderBasedJsonParser._reportInvalidToken(ReaderBasedJsonParser.java:2753) at org.apache.flink.shaded.jackson2.com.fasterxml.jackson.core.json.ReaderBasedJsonParser._handleOddValue(ReaderBasedJsonParser.java:1824) at org.apache.flink.shaded.jackson2.com.fasterxml.jackson.core.json.ReaderBasedJsonParser.nextToken(ReaderBasedJsonParser.java:708) at org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.ObjectMapper._initForReading(ObjectMapper.java:3847) at org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:3792) at org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2797) at com.dtstack.flinkx.util.JsonUtils.jsonStrToObject(JsonUtils.java:38) at com.dtstack.flinkx.launcher.Launcher.main(Launcher.java:132)
有遇到过吗?
from chunjun.
支持yarn-perjob这种模式提交任务的呀 不过我用的1.7这个分支
切换到1.7可以了, 但是还是报以下错误
Exception in thread "main" org.apache.flink.shaded.jackson2.com.fasterxml.jackson.core.JsonParseException: Unrecognized token 'jobmanager': was expecting ('true', 'false' or 'null') at [Source: jobmanager\.memory\.mb:1276; line: 1, column: 11] at org.apache.flink.shaded.jackson2.com.fasterxml.jackson.core.JsonParser._constructError(JsonParser.java:1586) at org.apache.flink.shaded.jackson2.com.fasterxml.jackson.core.base.ParserMinimalBase._reportError(ParserMinimalBase.java:521) at org.apache.flink.shaded.jackson2.com.fasterxml.jackson.core.json.ReaderBasedJsonParser._reportInvalidToken(ReaderBasedJsonParser.java:2753) at org.apache.flink.shaded.jackson2.com.fasterxml.jackson.core.json.ReaderBasedJsonParser._handleOddValue(ReaderBasedJsonParser.java:1824) at org.apache.flink.shaded.jackson2.com.fasterxml.jackson.core.json.ReaderBasedJsonParser.nextToken(ReaderBasedJsonParser.java:708) at org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.ObjectMapper._initForReading(ObjectMapper.java:3847) at org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:3792) at org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2797) at com.dtstack.flinkx.util.JsonUtils.jsonStrToObject(JsonUtils.java:38) at com.dtstack.flinkx.launcher.Launcher.main(Launcher.java:132)
有遇到过吗?
已解决, 是提交时 需要对{} 和 "" 都要加 \
from chunjun.
Related Issues (20)
- [Feature][s3] 增加支持读取压缩格式文件
- [Bug] [connector jdbc] jdbc data is consumed during dimensional table join, resulting in location dislocation
- 请问一下,Oracle全量+增量同步问题 是否会丢失数据
- [Question]chunjun 启动yarn-session报错 HOT 1
- [Bug] [jdbc base] 使用Collections.singletonList返回不可修改list,但代码中进行了修改 HOT 1
- StarRocksOutputFormat.writeMultipleRecordsInternal()中if判断导致数据积压问题bug
- [Bug][chunjun-connectors] jdbc's properties not work where jdbc's reader has set properties
- 纯钧如何使用Kubernetes Operator提交
- [Bug] [chunjun-client] 本地local模式提交执行任务情况下connector 的加载错误
- [Feature][ftp] 支持读取中文路径、优化读取execl
- [Bug] [ddl] maven package failed
- Release v1.12.8-cross join unnest使用报错
- [mongodb-x]-mongo维表连接失败
- [Feature][hbase] sql模式下新增特性
- 本地执行mysqlcdcreader到streamwriter,GenericRowData类型无法支持rowSize! HOT 1
- [Feature][ftp] sql模式下,支持指定字段的列索引,支持指定sheetNo工作表
- [Feature][Module Name] support hbase incremental read
- [Question][chunjun-oracle] oracle的clob类型,使用transformer报错
- [Bug] [chunjun-connector-binlog] Skipping unrecognized binlog event Unknown type:41 from: mysql-bin.xxx
- [Bug] [Module Name] Bug title mysql-mysql中加入transformer操作,sql可能有点长报错
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from chunjun.