Comments (13)
大型系统理论上应该可以结合 archguard/archguard#43 一起考虑
from scanner.
现有规则的设计
- Rule 作为基本的规则类。
- Rulesets 是一堆规则的合集
- RuleSetProvider 根据不同情况,如 Standard、Recommend 返回对应的规则。
- RuleVisitor 根据不同的规则类型(如 API、数据库、测试坏味道等)提供 Visitor 接口。
示例见:tbs
from scanner.
将scanner分为language/entry scanner和feature scanner
- 输入source(git,source code),通过language scanner,输出data structure
- 输入data structure,通过feature scanner,输出data structure(或db,api等特殊scheme)
- scanner以树状形式相连,按中缀遍历依次执行
scanner配置化,可重定向目标uri(file path or url)
- (后续)提供scanner project template,大家可以快速创建一个language or feature scanner
- 每个scanner即一个function,具有独立标识符并可以单独打包,提供一个配置类来声明scanner的树状结构
- 使用class for name or spi来动态加载指定的scanner,并保证链路正常
解决的问题
最大的scanner chapi目前包含全语言,40m,可以不着急按语言拆,主要是通过重构合理安排scanner的结构,防止无限制膨胀
- 单一scanner膨胀变大(可按data structure的分析阶段 任意拆分)
- 自定义扩展(通过overwrite配置 or append实现)
5.0?for 大型系统(类jenkins x)
- scanner进程化(类mapreduce),每发起一次scan请求,即创建一个pod
- pod内会包含相应的scanner环境
- 利用k8s的伸缩能力来安排scanner执行(or 自定义Operator)
- 将结果写到etcd或通过hooks写回archguard
from scanner.
archguard/archguard#30 (comment)
class DubboScanner:FeatureScanner
language:kotlin // kotlin sourcecode
features:
- api
- db
- dubbo:[email protected]
from scanner.
大型系统理论上应该可以结合 archguard/archguard#43 一起考虑
这个可以先展望一下 哈哈
from scanner.
现有规则的设计
- Rule 作为基本的规则类。
- Rulesets 是一堆规则的合集
- RuleSetProvider 根据不同情况,如 Standard、Recommend 返回对应的规则。
- RuleVisitor 根据不同的规则类型(如 API、数据库、测试坏味道等)提供 Visitor 接口。
示例见:tbs
看了一下,感觉实现会比较类似,但是scanner还是单独拆开好些(视角不一样);
scanner只做数据清洗的工作,即转换成标准的model(CodeDataStructure, ApiCodeCallMap, ContainerService...);
rule的input应该是scanner的output;
from scanner.
对,比较相似,但是其实是两个不同的东西。
from scanner.
这个图还是很清晰的
AST -> Chapi
Model Construction+Extraction -> Scanner
Analysis with Patterns -> Rule
from scanner.
Update latest design for scanner
from scanner.
https://github.com/archguard/scanner/tree/master/scanner_cli
https://github.com/archguard/scanner/tree/master/analyser_sourcecode/lang_kotlin
TODO:
- migrate all language analysers with new structure (e.g. lang_kotlin)
- migrate API/DB analysers
- test scanner cli to dispatch tasks to different analysers
- create scanner-client/http api in ArchGuard Backend
- test whole workflow
- create
analyser template
as a standard project - migrate other analysers (git, bytecode ...)
from scanner.
最后,估计还需要一个 scanner_output,比如支持直接插入数据库、生成 JSON、CSV
from scanner.
最后,估计还需要一个 scanner_output,比如支持直接插入数据库、生成 JSON、CSV
实现其他的ArchGuardClient
,里面拿到数据就打印JSON、CSV,在cli通过参数控制选Http的client还是JSON的client
from scanner.
design and implementation already done, close this issue
from scanner.
Related Issues (13)
- Data too long for column 'key' at row 。。。。 HOT 1
- 手动执行 scan_sourcecode-1.6.2-all.jar 直接异常 HOT 2
- Code Structure Style Identify
- Count code complexity
- Architecture Model Design
- Arch as Code
- Look me:ArchGuard 部署搭建 —— help !!! HOT 1
- refactor Rule with visitor patterns
- Document generate for all rules
- bytecode scaner无法识别@RequestMapping api HOT 2
- Multiple dataformat support for output HOT 1
- Tech debt from comments
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from scanner.