alldatacenter / alldata Goto Github PK

🔥🔥 AllData大数据产品是可定义数据中台，以数据平台为底座，以数据中台为桥梁，以机器学习平台为中层框架，以大模型应用为上游产品，提供全链路数字化解决方案。全新会员商业版 X 微信群：https://docs.qq.com/doc/DVHlkSEtvVXVCdEFo

Home Page: http://122.51.43.143:5173/ui_moat

License: GNU General Public License v3.0

Shell 0.37% Java 69.05% SCSS 0.50% Makefile 0.01% Batchfile 0.01% Python 0.05% FreeMarker 0.38% ANTLR 0.72% Vue 28.91%

flink griffin hudi iceberg kong paimon ranger tis datart dinky

alldata's People

Contributors

Stargazers

Watchers

Forkers

sxyseo lyc88 1574359445 jzxyouok gaoxiong523 1105748319 iweisi bossding majiajue wenzhenxing liangjw zeng08 cyofeiyue bit-star davegeneral hanhongyuan samchen1981 frankiegu lenmom jayfeihe benhe119 baipengyan bi0821 heibaifu dcmr zeem0 sonygrq2014 kangjunxiang hezhengl sangyuruo arraycto daniel-yee heshengwang ycd2003 binrowang zhengjiujiang kingofqueen bell-fintech chrisliu12345 605774644 donnieburgs tianhuage1997 guoliduo3 luodongfu synpore jsonbao amiliy westbridge chenyao1234 fucora gitdevep lovepli winimi lvpeng11 daniellhu elliottfdh daichangya lanwu114 silentchen joelll666 hwlsniper suanping qinchun2000 fammr 3972794 tom2jack cenbow abothli shutong12345 lukw00heck persevere1 zhang072 dong-guosheng jerry185 slyfalcon sumlxt123 kentoo-zbcast xccoder meunity seraphim-eve aixuedegege hotsmile chocoai muqzq tkn4401088 552301 debug001 fourth-fresh-man xuduofeng jasonhe78 bestjex xiejiajun huaapp edgeowner adslfup xzycr7 heloleo forhelostar tiantiansong savepong

alldata's Issues

CSRF Token has been associated to this client 登录接口提示跨域

部署文档错误

README.md中“部署方式”下相关sql脚本在相关位置无法找到
例如

source install/16gmaster/studio/studio_alldatadc.sql

大数据链路稽查SRE设计与规划

待认领，联系作者: https://docs.qq.com/doc/DVFVMYUp6cFhSRVJs

AllData实时计算平台规划与设计

待认领，联系作者: https://docs.qq.com/doc/DVFVMYUp6cFhSRVJs

BitSail POC调研，输出BitSail安装部署文档，梳理与alldata studio集成思路

alldata/factory/studio，与bitsail集成需要设计一套SPI的插件化的设计，引入bitsail相关功能，输出BitSail安装部署文档，梳理与alldata studio集成思路

Welcome update to OpenMMLab 2.0

I am Vansin, the technical operator of OpenMMLab. In September of last year, we announced the release of OpenMMLab 2.0 at the World Artificial Intelligence Conference in Shanghai. We invite you to upgrade your algorithm library to OpenMMLab 2.0 using MMEngine, which can be used for both research and commercial purposes. If you have any questions, please feel free to join us on the OpenMMLab Discord at https://discord.gg/amFNsyUBvm or add me on WeChat (van-sin) and I will invite you to the OpenMMLab WeChat group.

Here are the OpenMMLab 2.0 repos branches:

	OpenMMLab 1.0 branch	OpenMMLab 2.0 branch
MMEngine		0.x
MMCV	1.x	2.x
MMDetection	0.x 、1.x、2.x	3.x
MMAction2	0.x	1.x
MMClassification	0.x	1.x
MMSegmentation	0.x	1.x
MMDetection3D	0.x	1.x
MMEditing	0.x	1.x
MMPose	0.x	1.x
MMDeploy	0.x	1.x
MMTracking	0.x	1.x
MMOCR	0.x	1.x
MMRazor	0.x	1.x
MMSelfSup	0.x	1.x
MMRotate	1.x	1.x
MMYOLO		0.x

Attention: please create a new virtual environment for OpenMMLab 2.0.

Apache Drill POC调研，输出Apache Drill安装部署README.md

Apache Drill POC调研，输出Apache Drill安装部署README.md 以及studio集成文档Drill文档

初始化sql里，表名有重复的

体验版本部分菜单报请求接口失败

数据比对调研DataCompare调研

alldata项目数据比对调研，数据集成进行同步之后，定时或立即触发DataCompare Spark跑批任务做数据比对，数据比对功能,应用在源，目的端数据一致性校验，特别特别轻量级而且很方便验证一致性，尝试POC输出体验文档

AllData Studio集成Tis调研POC，输出TIS安装流程、Studio集成思路

[Studio]网关启动报redis 连接异常

体验版功能数据集成>资源管理添加点击无反应

AIOPS 规划与设计

待认领，联系作者: https://docs.qq.com/doc/DVFVMYUp6cFhSRVJs

DataX POC调研，输出DataX安装部署文档，梳理与alldata studio集成思路

微服务架构RoadMap设计与规划

数据平台的微服务引擎, 将JAVA应用拆分为粒度小的SpringBoot应用，使用微服务治理

1、AllData涉及很多Java后台的服务

2、引入Github已有的SpringCloud脚手架

3、统一通过OneData项目来纳管粒度足够小的SpringBoot应用

alldata/factory/studio【安装部署教程】,输出详细的README.md

Studio Flinkx code refactoring

Studio Flinkx代码重构

墨刀原型还有吗

StreamPark调研 && 输出安装+调试的README.md

AllData前端规划与技术选型

经内部考虑，后续讨论

AllData产品原型规划设计

待认领，联系作者: https://docs.qq.com/doc/DVFVMYUp6cFhSRVJs

Dependency org.yaml:snakeyaml, leading to CVE problem

Hi, In /shuffle/celeborn/common，there is a dependency org.yaml:snakeyaml:1.30 that calls the risk method.

CVE-2022-25857

The scope of this CVE affected version is ** [0,1.31)**

After further analysis, in this project, the main Api called is org.yaml.snakeyaml.composer.Composer: composeNode(org.yaml.snakeyaml.nodes.Node)Lorg.yaml.snakeyaml.nodes.Node;

Risk method repair link : GitHub

CVE Bug Invocation Path--

Path Length : 5

CVE Bug Invocation Path : 
org.apache.celeborn.common.network.client.TransportResponseHandler: failExpiredPushRequest()V .m2/repository/io/netty/netty-codec-redis/4.1.77.Final/netty-codec-redis-4.1.77.Final.jar
org.yaml.snakeyaml.Yaml$2: next()Ljava.lang.Object; .m2/repository/io/netty/netty-codec-redis/4.1.77.Final/netty-codec-redis-4.1.77.Final.jar
org.yaml.snakeyaml.Yaml$2: next()Lorg.yaml.snakeyaml.nodes.Node; .m2/repository/io/netty/netty-codec-redis/4.1.77.Final/netty-codec-redis-4.1.77.Final.jar
org.yaml.snakeyaml.composer.Composer: getNode()Lorg.yaml.snakeyaml.nodes.Node; .m2/repository/io/netty/netty-codec-redis/4.1.77.Final/netty-codec-redis-4.1.77.Final.jar
org.yaml.snakeyaml.composer.Composer: composeNode(org.yaml.snakeyaml.nodes.Node)Lorg.yaml.snakeyaml.nodes.Node;

Dependency tree--

[INFO] org.apache.celeborn:celeborn-common_2.12:jar:0.3.0-SNAPSHOT
[INFO] +- org.apache.ratis:ratis-common:jar:2.4.1:compile
[INFO] |  +- org.apache.ratis:ratis-thirdparty-misc:jar:1.0.3:compile
[INFO] |  \- org.apache.ratis:ratis-proto:jar:2.4.1:compile
[INFO] +- org.apache.ratis:ratis-client:jar:2.4.1:compile
[INFO] +- io.dropwizard.metrics:metrics-core:jar:3.2.6:compile
[INFO] +- io.dropwizard.metrics:metrics-graphite:jar:3.2.6:compile
[INFO] +- io.dropwizard.metrics:metrics-jvm:jar:3.2.6:compile
[INFO] +- org.yaml:snakeyaml:jar:1.30:compile
[INFO] +- org.slf4j:slf4j-api:jar:1.7.36:compile
[INFO] +- org.slf4j:jul-to-slf4j:jar:1.7.36:compile
[INFO] +- org.slf4j:jcl-over-slf4j:jar:1.7.36:compile
[INFO] +- commons-io:commons-io:jar:2.8.0:compile
[INFO] +- org.apache.commons:commons-crypto:jar:1.0.0:compile
[INFO] +- org.apache.commons:commons-lang3:jar:3.10:compile
[INFO] +- io.netty:netty-all:jar:4.1.77.Final:compile
[INFO] |  +- io.netty:netty-buffer:jar:4.1.77.Final:compile
[INFO] |  +- io.netty:netty-codec:jar:4.1.77.Final:compile
[INFO] |  +- io.netty:netty-codec-dns:jar:4.1.77.Final:compile
[INFO] |  +- io.netty:netty-codec-haproxy:jar:4.1.77.Final:compile
[INFO] |  +- io.netty:netty-codec-http:jar:4.1.77.Final:compile
[INFO] |  +- io.netty:netty-codec-http2:jar:4.1.77.Final:compile
[INFO] |  +- io.netty:netty-codec-memcache:jar:4.1.77.Final:compile
[INFO] |  +- io.netty:netty-codec-mqtt:jar:4.1.77.Final:compile
[INFO] |  +- io.netty:netty-codec-redis:jar:4.1.77.Final:compile
[INFO] |  +- io.netty:netty-codec-smtp:jar:4.1.77.Final:compile
[INFO] |  +- io.netty:netty-codec-socks:jar:4.1.77.Final:compile
[INFO] |  +- io.netty:netty-codec-stomp:jar:4.1.77.Final:compile
[INFO] |  +- io.netty:netty-codec-xml:jar:4.1.77.Final:compile
[INFO] |  +- io.netty:netty-common:jar:4.1.77.Final:compile
[INFO] |  +- io.netty:netty-handler:jar:4.1.77.Final:compile
[INFO] |  +- io.netty:netty-handler-proxy:jar:4.1.77.Final:compile
[INFO] |  +- io.netty:netty-resolver:jar:4.1.77.Final:compile
[INFO] |  +- io.netty:netty-resolver-dns:jar:4.1.77.Final:compile
[INFO] |  +- io.netty:netty-transport:jar:4.1.77.Final:compile
[INFO] |  +- io.netty:netty-transport-rxtx:jar:4.1.77.Final:compile
[INFO] |  +- io.netty:netty-transport-sctp:jar:4.1.77.Final:compile
[INFO] |  +- io.netty:netty-transport-udt:jar:4.1.77.Final:compile
[INFO] |  +- io.netty:netty-transport-classes-epoll:jar:4.1.77.Final:compile
[INFO] |  +- io.netty:netty-transport-native-unix-common:jar:4.1.77.Final:compile
[INFO] |  +- io.netty:netty-transport-classes-kqueue:jar:4.1.77.Final:compile
[INFO] |  +- io.netty:netty-resolver-dns-classes-macos:jar:4.1.77.Final:compile
[INFO] |  +- io.netty:netty-transport-native-epoll:jar:linux-x86_64:4.1.77.Final:runtime
[INFO] |  +- io.netty:netty-transport-native-epoll:jar:linux-aarch_64:4.1.77.Final:runtime
[INFO] |  +- io.netty:netty-transport-native-kqueue:jar:osx-x86_64:4.1.77.Final:runtime
[INFO] |  +- io.netty:netty-transport-native-kqueue:jar:osx-aarch_64:4.1.77.Final:runtime
[INFO] |  +- io.netty:netty-resolver-dns-native-macos:jar:osx-x86_64:4.1.77.Final:runtime
[INFO] |  \- io.netty:netty-resolver-dns-native-macos:jar:osx-aarch_64:4.1.77.Final:runtime
[INFO] +- org.fusesource.leveldbjni:leveldbjni-all:jar:1.8:compile
[INFO] +- com.google.code.findbugs:jsr305:jar:1.3.9:compile
[INFO] +- com.google.guava:guava:jar:14.0.1:compile
[INFO] +- com.google.protobuf:protobuf-java:jar:3.19.2:compile
[INFO] +- org.scala-lang:scala-library:jar:2.12.15:compile
[INFO] +- org.scala-lang:scala-reflect:jar:2.12.15:compile
[INFO] +- org.apache.hadoop:hadoop-client-api:jar:3.2.1:compile
[INFO] +- org.apache.hadoop:hadoop-client-runtime:jar:3.2.1:compile
[INFO] |  +- org.apache.htrace:htrace-core4:jar:4.1.0-incubating:runtime
[INFO] |  \- commons-logging:commons-logging:jar:1.1.3:runtime
[INFO] +- org.roaringbitmap:RoaringBitmap:jar:0.9.32:compile
[INFO] |  \- org.roaringbitmap:shims:jar:0.9.32:runtime
[INFO] +- org.mockito:mockito-core:jar:1.10.19:test
[INFO] |  +- org.hamcrest:hamcrest-core:jar:1.1:test
[INFO] |  \- org.objenesis:objenesis:jar:2.1:test
[INFO] +- org.apache.logging.log4j:log4j-slf4j-impl:jar:2.17.2:test
[INFO] |  +- org.apache.logging.log4j:log4j-api:jar:2.17.2:test
[INFO] |  \- org.apache.logging.log4j:log4j-core:jar:2.17.2:test
[INFO] +- org.apache.logging.log4j:log4j-1.2-api:jar:2.17.2:test
[INFO] +- junit:junit:jar:4.12:test
[INFO] \- org.scalatest:scalatest_2.12:jar:3.2.3:test
[INFO]    +- org.scalatest:scalatest-core_2.12:jar:3.2.3:test
[INFO]    |  +- org.scalatest:scalatest-compatible:jar:3.2.3:test
[INFO]    |  +- org.scalactic:scalactic_2.12:jar:3.2.3:test
[INFO]    |  \- org.scala-lang.modules:scala-xml_2.12:jar:1.2.0:test
[INFO]    +- org.scalatest:scalatest-featurespec_2.12:jar:3.2.3:test
[INFO]    +- org.scalatest:scalatest-flatspec_2.12:jar:3.2.3:test
[INFO]    +- org.scalatest:scalatest-freespec_2.12:jar:3.2.3:test
[INFO]    +- org.scalatest:scalatest-funsuite_2.12:jar:3.2.3:test
[INFO]    +- org.scalatest:scalatest-funspec_2.12:jar:3.2.3:test
[INFO]    +- org.scalatest:scalatest-propspec_2.12:jar:3.2.3:test
[INFO]    +- org.scalatest:scalatest-refspec_2.12:jar:3.2.3:test
[INFO]    +- org.scalatest:scalatest-wordspec_2.12:jar:3.2.3:test
[INFO]    +- org.scalatest:scalatest-diagrams_2.12:jar:3.2.3:test
[INFO]    +- org.scalatest:scalatest-matchers-core_2.12:jar:3.2.3:test
[INFO]    +- org.scalatest:scalatest-shouldmatchers_2.12:jar:3.2.3:test
[INFO]    \- org.scalatest:scalatest-mustmatchers_2.12:jar:3.2.3:test

Suggested solutions:

Update dependency version

Thank you very much.

接口请求失败

http://43.138.157.47:8013/login?redirect=%2Fdashboard

下面的接口报错了，帮忙看下

curl "http://43.138.157.47:8013/auth/code" ^
-H "Accept: application/json, text/plain, /" ^
-H "Accept-Language: zh-CN,zh;q=0.9" ^
-H "Connection: keep-alive" ^
-H "Referer: http://43.138.157.47:8013/login?redirect=^%^2Fdashboard" ^
-H "User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36" ^
--compressed ^
--insecure