apache / servicecomb-toolkit

Apache servicecomb

Toolkit | 中文

Apache ServiceComb Toolkit is a contract-based microservice development toolkit

1 Introduction

1.1 Conceptions

• Provides the ability to convert and verify contracts, code, and documents, helping users quickly build microservice projects based on popular microservices frameworks and popular programming models, reducing the cost of microservices entry, enabling users to focus on business development, enhance refactoring and development efficiency.

1.2 Features

• Code extraction service contract

  In applications developed based on the SpringMVC/POJO/JAX-RS model, one-click generation of service contract files conforming to the OpenAPI specification.

• Service contract generation microservice project

  Enter a service contract that conforms to the OpenAPI specification, one-click generation of a microservice project with ServiceComb/SpringCloud/Swagger as the base microservice framework and SpringMVC/POJO/JAX-RS or SpringBoot as programming model.

• Service contract and code consistency check

  Verify that the actual implementation of the application (such as the data and service API) is consistent with the agreed service contract description.

• Service contract style checking and compatibility checking

  The style checking checks whether the contract conforms to [OAS 3.0.2 specification] [openapi-3.0.2] and custom rules; the compatibility checking checks whether new OAS spec compatible with old spec

• Service contract/code generation document

  Enter a service contract that conforms to the OpenAPI specification, one-click generation of a document in html format.

• Todo List

  • Support gradle plugin, eclipse plugin, intellij plugin.

  • Support for generating popular format documents such as word, pdf.

  • Support contract incremental generation code.

  • Make capability as a service.

  • Server-side automatic / semi-automatic test

  • Interface matching check

  • Support for generating microservice scaffolding containing code snippets that connect to common DBs such as Mysql / Redis

1.3 Applicable Scene

• For users who integrate multi-vendor applications

  Scene: the development languages, habits, and frameworks of different vendors are different, the entire system data and service standards are inconsistent, users are difficult to integrate, and it is difficult to manage and control the final delivery quality.

  Solution：Through a uniformly defined interface description standard (service contract), a toolkit is used to generate a microservice project based on a specified microservices framework, and the consistency of the whole system is coordinated through service contract verification. Coordinate multiple development teams to reduce communication costs and avoid post-chaos.

• For users who have evolved from legacy systems to microservices

  Scene: additional learning and understanding of the microservices-related framework details is required before the microservices project can be designed, built, and developed according to the selected microservices framework. For users, Need to be distracted to focus on things outside the business.

  Solution: Using the toolkit to analyze the legacy application extraction service contract, and then generate the microservices project based on the specified microservices framework, you can focus on business development and reduce the learning cost of the microservices framework.

2 Design

2.1 Architecture

2.2 Working Principle

3 Quick Start

3.1 Build tool and plugins from source

Build environment requirements

• Java 8
• Apache maven 3.5.0 or greater

# Get the latest source code for toolkit from github
$ git clone https://github.com/apache/servicecomb-toolkit.git
$ cd toolkit

# Build package
$ mvn clean install

3.2 Use the toolkit-maven-plugin plugin

3.2.1 configuration

Configured in the pom file of the maven project

<plugin>
    <groupId>org.apache.servicecomb.toolkit</groupId>
    <artifactId>toolkit-maven-plugin</artifactId>
    <version>0.3.0-SNAPSHOT</version>
    <configuration>
        <!-- Set to 'code' to resolve the current project. Set to 'contract' to resolve the contract file for the specified path.If not set, the default is 'code' -->
        <sourceType>code</sourceType>
        <!-- The type of the contract file is generated. If it is not set, the default is 'yaml' -->
        <contractFileType>yaml</contractFileType>
        <!-- The type of the generated document. If not set, the default is 'html' -->
        <documentType>html</documentType>
        <!-- The root directory to save microservice project,contract file and document. If it is not set, the default is the 'target' under the directory where the command is run -->
        <outputDirectory>./target</outputDirectory>
        <!-- Input contract file path. Valid when sourceType is set to 'contract', must be set -->
        <contractLocation>./contract</contractLocation>
        <!-- Checked contract file path. Valid when sourceType is set to 'contract', must be set -->
        <sourceContractPath>./target/contract</sourceContractPath>
        <!-- Sample contract file path, must be set -->
        <destinationContractPath>./contract</destinationContractPath>
        <!-- Generated microservice project configuration -->
        <service>
            <!-- Microservice type,can generated 'provider/consumer/all',the default is 'all' -->
            <serviceType>all</serviceType>
            <!-- Microservice project 'groupid',optional,the default is 'domain.orgnization.project' -->
            <groupId>domain.orgnization.project</groupId>
            <!-- Microservice project 'artifactId',optional,the default is 'sample' -->
            <artifactId>sample</artifactId>
            <!-- Microservice project 'artifactVersion',optional,the default is '0.1.0-SNAPSHOT' -->
            <artifactVersion>0.1.0-SNAPSHOT</artifactVersion>
            <!-- Microservice project 'packageName',optional,the default is 'domain.orgnization.project.sample' -->
            <packageName>domain.orgnization.project.sample</packageName>
        </service>
        <!-- Specify additional attribute values that can be referenced by the mustache template.Only used when plugin goal is generate -->
        <additionalProperties>
            <prop1>value</prop1>
            <prop2>value</prop2>
            ...
            <propN>value</propN>
        </additionalProperties>
    </configuration>
</plugin>

3.2.2 Command

# Generating contract, document and microservice project
mvn toolkit:generate

# Verify code and contract consistency
mvn toolkit:verify

3.2.2.1 Extract the microservice project, OpenAPI contract file and document from the code

Configuration(use default configuration if not set <configuration>)

example

<plugin>
    <groupId>org.apache.servicecomb.toolkit</groupId>
    <artifactId>toolkit-maven-plugin</artifactId>
    <version>0.3.0-SNAPSHOT</version>
    <configuration>
        <!-- Set to 'code' to resolve the current project. Set to 'contract' to resolve the contract file for the specified path.If not set, the default is 'code' -->
        <sourceType>code</sourceType>
        <!-- The root directory to save contract file and document. If it is not set, the default is the 'target' under the directory where the command is run -->
        <outputDirectory>./target</outputDirectory>
        <!-- Generated microservice project configuration -->
        <service>
            <!-- Microservice type,can generated 'provider/consumer/all',the default is 'all' -->
            <serviceType>all</serviceType>
        </service>
    </configuration>
</plugin>

Run in shell

mvn toolkit:generate

When generating contracts from code,support for identifying restful interfaces written by the following annotations (class level)

RestController, RestSchema, RpcSchema, RequestMapping

When generating contracts from code,the restful interface method access modifier must be specified as public

3.2.2.2 Generate the microservice project and document from contract

Configuration(use default configuration if not set <configuration>)

example

<plugin>
    <groupId>org.apache.servicecomb.toolkit</groupId>
    <artifactId>toolkit-maven-plugin</artifactId>
    <version>0.3.0-SNAPSHOT</version>
    <configuration>
        <!-- Set to 'code' to resolve the current project. Set to 'contract' to resolve the contract file for the specified path.If not set, the default is 'code' -->
        <sourceType>contract</sourceType>
        <!-- The root directory to save contract file and document. If it is not set, the default is the 'target' under the directory where the command is run -->
        <outputDirectory>./target</outputDirectory>
        <!-- Input contract file path. Valid when sourceType is set to 'contract', must be set -->
        <contractLocation>./contract</contractLocation>
        <!-- Generated microservice project configuration -->
        <service>
            <!-- Microservice type,can generated 'provider/consumer/all',the default is 'all' -->
            <serviceType>provider</serviceType>
        </service>
    </configuration>
</plugin>

Run in shell

mvn toolkit:generate

3.2.2.3 Contract verify

Configuration

example

<plugin>
    <groupId>org.apache.servicecomb.toolkit</groupId>
    <artifactId>toolkit-maven-plugin</artifactId>
    <version>0.3.0-SNAPSHOT</version>
    <configuration>
        <!-- Set to 'code' to resolve the current project. Set to 'contract' to resolve the contract file for the specified path.If not set, the default is 'code' -->
        <sourceType>code</sourceType>
        <!-- Sample contract file path, must be set -->
        <destinationContractPath>./contract</destinationContractPath>
    </configuration>
</plugin>

Run in shell

mvn toolkit:verify

3.3 Use the toolkit cli

• If you are using the official release version ( >=0.2.0 ), you can use the script files directly after decompressing the binary package
  • In Linux and Mac environment, please use cli.sh
  • In the Windows environment, please use cli.cmd
• If you are building from source, you can put cli/scripts/cli.* in the same directory ascli/target/bin/toolkit-cli-{version}.jar and then choose different scripts according to different environments

All the examples below are introduced through cli.sh for Linux environment

$ ./cli.sh help

3.3.1 Service contract generation microservice project

$ ./cli.sh codegenerate -m ServiceComb -i swagger.yaml -o ./project -p SpringMVC

codegenerate Command option

• -m, --microservice-framework. Specify microservices framework, now supports ServiceComb.
  e.g.：-m ServiceComb
• -p, --programming-model. Specify programming model, optional JAX-RS, POJO, SpringMVC, SpringBoot.
  e.g.：-p SpringMvc
• -i, --input. Specifies contract files that follow the OpenAPI specification, supports yaml and json formats, and supports specifying local and network files.
  e.g.：-i http://petstore.swagger.io/v2/swagger.json
• -o, --output. Generated project code output path.
  e.g.：-o ./project
• --group-id. Specify the group id of the generated project.
  e.g.：--group-id com.demo
• --artifact-id. Specify the artifact id of the generated project.
  e.g.：--artifact-id springmvc-example
• --artifact-version. Specify the artifact version of the generated project.
  e.g.：--artifact-version 1.0.0
• --api-package : Specify the api package of the generated project.
  e.g.：--api-package com.demo.api
• --model-package : Specify the model package of the generated project.
  e.g.：--model-package com.demo.model
• -t, --service-type : Specify microservice type of generated microservice project. optional value is provider,consumer,all
  e.g.：--service-type provider
• --properties : Specify additional attribute values that can be referenced by the mustache template e.g.：--properties applicationId=app,providerServiceId=provider

3.3.2 Service contract generation document

$ ./cli.sh docgenerate -i swagger.yaml -o ./document

docgenerate Command option

• -i, --input. Specifies contract files that follow the OpenAPI specification, supports yaml and json formats, and supports specifying local and network files. e.g：-i http://petstore.swagger.io/v2/swagger.json
• -o, --output. Document output path. e.g：-o ./document
• -f, --format. Specifies the output document format, now supports swagger-ui e.g：-f swagger-ui

3.3.3 Service contract style checking

$ ./cli.sh checkstyle -r style-check-rules.yaml -f oas.yaml
or
$ ./cli.sh cs -r style-check-rules.yaml -f oas.yaml

checkstyle Command argument

• -r, --rules-file. Rules properties file.
• -f, --file. OpenAPI v3 spec yaml.

See style check rules

3.3.4 Service contract compatibility checking

$ ./cli.sh checkcompatibility left-oas.yaml right-oas.yaml
or
$ ./cli.sh cc left-oas.yaml right-oas.yaml

checkcompatibility Command argument

• <files> Two OpenAPI v3 spec yaml file

See compatibilty check rules

3.4 Use case

some example of using plugin can be found here

4 Contact us

• issues
• gitter
• mailing list: subscribe view

5 Contribute

PR: Pull request

servicecomb-toolkit's Issues

windows上cli 处理文件编码问题

openapi yaml 文件有中文字符，文件编码格式是 utf-8 。当在windows cmd 下执行时会提示编码错误，需要加上 -Dfile.encoding=UTF-8。

错误信息如下：

`java -jar bin/cli-0.2.0-SNAPSHOT.jar checkstyle -r style-check-rules.properties -f openapi.yml
Exception in thread "main" java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
        at java.lang.reflect.Method.invoke(Unknown Source)
        at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48)
        at org.springframework.boot.loader.Launcher.launch(Launcher.java:87)
        at org.springframework.boot.loader.Launcher.launch(Launcher.java:50)
        at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:51)
Caused by: java.lang.RuntimeException: java.io.CharConversionException: Invalid UTF-8 start byte 0xb5 (at char #35, byte #-1)
 at [Source: (byte[])"openapi: '3.0.2'
info:
  title: ?????POS???
  version: '1.0.0'
  description: ???POS???????
servers:
  - url: 'https://xxxxxxxx'
    description: ???????????????
  - url: http://xxxxxxxxxx/{basePath}
    description: ?????????????
    variables:
      basePath:
        default: v2
        description: api version


paths:
  /pos/login.init:
    $ref: commonapi.yml#/components/login.init
  /pos/login:
    $ref: commonapi.yml#/compo"[truncated 1336 bytes]; line: 1, column: 1]
        at org.apache.servicecomb.toolkit.cli.CheckStyleBase.run(CheckStyleBase.java:84)
        at org.apache.servicecomb.toolkit.cli.ToolkitMain.main(ToolkitMain.java:48)
        ... 8 more
make: *** [Makefile:28: verify] Error 1`

检查不符合规范时退出处理

检查不符合规范时没有用 System.exit(1) 来退出，makefile 这样的工具就不能很好的处理。
例如这样的Makefile。如果 verify 不通过，generate 应该也不能执行。
`
verify:
java -jar cil.jar checkstyle -f openapi.yml

generate: verify
java -jar cil.jar codegenerate -i openapi.yml
`

建议增加一个参数 --use-strict 表示启用严格模式，当检查不通过时用System.exit(1) 来退出。

通过toolkit生成model对象中boolean属性时，生成的get方法为isgetxxxx()

通过toolkit生成model对象中boolean属性时，会生成的get方法为isgetxxxx()；
第一次启动服务正常运行，但是再重启服务（同一台服务器、不通服务器重启服务）之后显示sc上的契约文件与本地契约文件不一致，

遗留应用代码生成微服务工程

存量业务代码量大，传统的SOA开发模式，有rest接口，也有webservice接口，工具能否考虑支持下面的需求
1.工具支持webservice标记的接口转为POJO接口；
2.工具支持在遗留代码中，自动重构服务消费者的接口调用方式改为cse调用方式；
3.工具修改遗留代码，保留原项目的目录结构

[SCB-1493] Adjust the structure of the generated project

When choose to generate only the provider or consumer, it is enough to generate only one project, merge the provider and model projects into one project. I will try to adjust and submit my solution

[SCB-1586] support parse OpenAPI v3 yaml

Current project based on OpenAPI v2, But The OpenAPI specification has now been updated to the v3 version, so we need to follow it by update our project. I will do it and then submit my solution later

Question: how to expose the service contracts generated by the toolkit online along the Service API server URL

Question: how to expose the service contracts generated by the toolkit online along the Service API server URL.
So that it can be accessed by the developers from the direct API environment instead of static pages.?

支持通过网站页面生成工程模板

比如希望使用mybatis+servicecomb一起进行开发，或者apache shiro等项目进行集成，但是现在上手门槛较高，没有快速开始的方式