Comments (8)
yangsishu
commented on July 19, 2024
就是直接写hdfs的
from chunjun.
yangsishu
commented on July 19, 2024
你看的是那个分支的代码
from chunjun.
x-otto-x
commented on July 19, 2024
那为何要字段映射？
from chunjun.
x-otto-x
commented on July 19, 2024
看的是现在最新的版本，hdfswriter 里的columns 参数是必选的。
from chunjun.
lijiangbo
commented on July 19, 2024
看的是现在最新的版本，hdfswriter 里的columns 参数是必选的。
hdfs里的文件有orc,parquet等格式，这文件里是包含元数据信息的，必须填写字段名称或者字段索引
from chunjun.
x-otto-x
commented on July 19, 2024
原来是这样，谢谢大佬
from chunjun.
x-otto-x
commented on July 19, 2024
写text文件是否可以不指定字段映射？
from chunjun.
lijiangbo
commented on July 19, 2024
写text文件是否可以不指定字段映射？
需要指定的，虽然是写字符串，但是我们也会对写入的数据先做类型处理，比如时间类型的数据，会指定特定格式写入的
from chunjun.
Related Issues (20)
- Job failed during initialization of JobManage HOT 1
- [ChunJun连接器][Mysql Source] connection jdbcUrl的样例和文字描述不符
- [Bug] [hdfs] orc file size calculation method is wrong
- [Bug] [connector-oracle] oracle消费date数据类型出现时间错乱的情况 HOT 1
- [Bug] [hdfs] Synchronizing text and parquet table data, the speed is too slow when the table field contains date type
- [Bug] [kudu] kudu write mode parameters do not take effect
- [Bug] [chunjun] flink web ui 查看 checkpoints 报错
- [Bug] [sqlservercdc-x] sql脚本配置的lsn无法生效
- [Bug] [dirty] 配置了脏数据处理，Prometheus nErrors指标不能同步最后一次失败的次数
- [Bug] [connector][kafka] 缺少依赖包 HOT 1
- [Bug] [connector]][rocketmq] rocketmq如果offset模式中，offset传入-1报错 HOT 1
- [Bug] [doris-x] sink-遇到源端update时，会出现重复统计 HOT 7
- [Bug] [clickhouse-x] sink-启动报错，无法正常工作：No value present
- [Question][Launcher]Yarn Session模式提交任务之后无响应
- [Feature] [ftp] 写入时增加支持压缩、写入模式以及修复获取sftp config bug
- 使用yarn-session模式运行官方实例stream.json的时候prometheus出现问题
- 纯钧执行任务时，校验数据表时使用大小写敏感的SQL语句问题
- [Question][KafkaSource] kafkasource sync模式编码解码器有没有计划支持canal-json
- [Bug] [chunjun-connector-kafka] DynamicKafkaSerializationSchema durationCounter have two different type
- [Bug] [jdbc] mysql 维表使用lru方式加载的数据为空
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from chunjun.