abhisheksr01 / spring-boot-microservice-best-practices

Best practices and integrations available for Spring Boot based Microservice in a single repository.

hey there 👋

👩‍💻 About Me

I am a DevOps and DevSecOps advocate with an appreciation of the benefits of software craft, emergent design, Agile and XP practices.

Loves to explore & Experiment with open source projects & continuously integrate them to achieve the best of Cloud Native practices.

Highly passionate about Learning, Practicing, Teaching DevOps & DevSecOps practices.

My mantra to a team’s success is “Experiment & break the barriers together”.

🛠 Programming Language

🛠 Tools & Technologies

🔥 My Stats :

spring-boot-microservice-best-practices's Issues

Fix/Replace Checkstyle

While bumping gradlew version checkstyle stopped working hence for time being we have commented on all its relevant references.
We should try fixing it or identify an alternative to it example snyk code test or sonarqube.

Performance Test

Given: The app is deployed
When: I run the performance test in the pipeline
Then: The app should pass the performance test

Done Criteria:
The pipeline should run the performance test after the app deployment successfully.

Notes:
Explore the performance test tooling available (eg jmeter, grinder etc).

CircleCI job fails with error "targets wrong architecture (found arm64 but need [amd64 i386 386])"

Problem:

The docker image "abhisheksr01/dind-utils" image was recently updated to use the latest version of dependencies.
The image published was an arm image and hence the CircleCI pipeline started failing with the below error:

WARNING: docker image abhisheksr01/dind-utils:latest targets wrong architecture (found arm64 but need [amd64 i386 386])

Job:
https://app.circleci.com/pipelines/github/abhisheksr01/spring-boot-microservice-best-practices/1073/workflows/51bd874a-7a2a-40cb-a232-8de29f3b03f6/jobs/4183

Docker Image:

https://hub.docker.com/repository/docker/abhisheksr01/dind-utils/general

Fix:

Use the arm resource class in circleci config as mentioned in the documentation https://circleci.com/docs/using-arm/

Bump Java 8 to Java 17

Since Springboot 3.0.X, Java 8 is not supported hence we need to migrate to Java 17.

ToDo:

1. Bump Java 8 to JDK 17 locally and test the application
2. Bump Springboot to 3.0.5
3. Update the documentation in README
4. Update the pipelines with JDK17
5. Update Docker Images to use JDK17

CircleCI: The k8s deploy job is unable to get the current docker image version

When the k8s deploy job is running it's unable to get the current docker image version from the version.txt file which is being mounted by the previous docker image build job.

Due to which we are always deploying the latest image which can be error-prone as the latest image can be cached & new changes won't be applied.

Possibly the issue could be with the way we are mounting the file as we have already replaced the previous eks-helm image to a new one but still the issue remains.

Bump Vulnerable Spring dependencies

Dependency check has identified below CRITICAL dependency vulnerabilities.

Dependencies:
spring-plugin-core-2.0.0.RELEASE: https://mvnrepository.com/artifact/org.springframework.plugin/spring-plugin-core
spring-plugin-metadata-2.0.0.RELEASE: https://mvnrepository.com/artifact/org.springframework.plugin/spring-plugin-metadata

Evidence:
https://github.com/abhisheksr01/spring-boot-microservice-best-practices/runs/6092145958?check_suite_focus=true

ToDo:

1. Currently they are latest available versions available hence until a new version is released suppress the vulnerabilities in the config.
2. And once a version with a fix is available bump the dependencies and remove the suppression.

Trivy vulnerability scanning is failing in CircleCI

Trivy vulnerability scanning is failing in CircleCI with an error message:

2021-10-12T13:17:22.094Z	FATAL	scan error: image scan failed: failed analysis: analyze error: timeout: context deadline exceeded

https://app.circleci.com/pipelines/github/abhisheksr01/spring-boot-microservice-best-practices/686/workflows/d94ba261-8e14-476c-b2d1-e3b28eae3d44/jobs/2651

jacocoTestCoverageVerification failing in circleci after JDK17, Gradle 7.6.1 upgrade

Error log:

!/bin/bash -eo pipefail
./gradlew jacocoTestCoverageVerification
Picked up _JAVA_OPTIONS: -Xmx3g

Welcome to Gradle 7.6.1!

Here are the highlights of this release:
 - Added support for Java 19.
 - Introduced `--rerun` flag for individual task rerun.
 - Improved dependency block for test suites to be strongly typed.
 - Added a pluggable system for Java toolchains provisioning.

For more details see https://docs.gradle.org/7.6.1/release-notes.html

> Task :compileJava UP-TO-DATE
> Task :processResources UP-TO-DATE
> Task :classes UP-TO-DATE
> Task :jacocoTestReport FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':jacocoTestReport'.
> Could not create service of type IsolatedAntBuilder using BuildScopeServices.createIsolatedAntBuilder().
   > Could not inspect the Groovy system for ClassLoader VisitableURLClassLoader(ant-and-gradle-loader)

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/7.6.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 6s
3 actionable tasks: 1 executed, 2 up-to-date

Exited with code exit status 1
CircleCI received exit code 1

Implement a Github Actions Pipeline for CICD

Auto merge Dependabot PR branch to master after all the tests pass

Description:

Dependabot bumps the new version of the dependencies. CircleCI runs all the tests for the new version of the dependency in the PR Branch. Then. I have to manually approve the PR to merge it to master.

Requirement:
Given A new dependabot PR is created
When All the tests in the PR branch are green
Then the PR branch should automatically merge to master
And The PR branch should be deleted

Fix Trivy Image scanning failure in circleci

Fix concourse pipeline

The pipeline is failing with an error "create resource config: base resource type not found: git".

Please find below error:

This might be due to the postgres & concourse old version in docker-compose.

Fix Swagger UI

Currently, the swagger page is not accessible hence we need to fix it.
We can refer to this document:
https://springdoc.org/v2/

ToDo:

1. Fix the dependency-related issues
2. Update the documentation in README

Use a tool to scan the k8s manifest and dockerfile

Identity a security tooling to scan the k8s manifest (standard and helm charts) locally and Dockerfile for security recommendations.

Identify the sections to be updated in README

Problem

Currently, the README is massively outdated and needs to be updated.

Solution

This Issue is responsible for identifying the areas which need to be updated and then creating smaller issues dedicated to updating the README as doing it in one go will be very difficult (Agile vs Waterfall)

Acceptance Criteria

• A curated list of the areas and what needs to be updated.
• Create specific issues.

Make Critical and High Vulnerability threshold configurable for docker image vulnerability scanning

Problem

Often trivy finds vulnerabilities in our docker images in the circleci pipeline which don't have a fix.
Hence causing the pipeline to fail until we bump the base image or dependency causing the issue.
But as mentioned above there might not be a fix available immediately hence in those scenarios we would like to proceed with the pipeline to allow the vulnerable images.

Solution

Create a variable in the CircleCI pipeline called al ALLOWED_VULNERABILITIES with comma-separated values for the allowed number of critical and high vulnerabilities.

example: ALLOWED_VULNERABILITIES=1,2

means 1 critical and 2 high vulnerabilities are allowed

Add test for CompaniesHouseController - RestController

Add a test for CompaniesHouseController which runs with SpringBootTest context and can test the annotations are working fine.

Bump dependencies and fix circleci pipeline

Add Concourse support for Apple M1 locally

Problem

The current docker-compose file for spinning up a local concourse pipeline doesn't work for M1 laptops.

Solution

tweak or implement functionality to work with Apple M1 laptops locally

Implement Snyk in CircleCI for SAST, LICENSE Scanning, depenendency vulnerability etc

Add a basic github action for PRs

Implement Cosign for signing docker images

bump gradlew version

Fix Dependency Vulnerability identified in com.fasterxml.jackson.core:jackson-databind by OWASP dependency check

Implement Semver Versioning

Problem

Currently, there is no consistent tagging strategy implemented. we use pipeline-based build tags for docker image tagging.

Suggestion:

Use coco-gito.

pitest seems not to be working

When ./gradlew pitest is executed some output is printed and looks like no mutation tests are running.

Output:

DEPRECATION WARNING. `testPlugin` is deprecated starting with GPP 1.7.4. It is also not used starting with PIT 1.6.7 (to be removed in 1.8.0).
> Task :clean

> Task :compileJava

> Task :processResources
> Task :classes

> Task :compileTestJava

> Task :processTestResources
> Task :testClasses

> Task :pitest
Option (* = required)                   Description      
---------------------                   -----------      
-?, -h                                  show help        
--avoidCallsTo <comma separated list                     
  of packages to consider as                             
  untouchable logging calls>                             
--classPath <coma separated list of                      
  additional classpath elements>                         
--classPathFile <File: File with a                       
  list of additional classpath                           
  elements (one per line)>                               
--coverageThreshold <Integer: Line      (default: 0)     
  coverage below which to throw an                       
  error>                                                 
--dependencyDistance <Integer: maximum  (default: -1)    
  distance to look from test for                         
  covered classes>                                       
--detectInlinedCode [Boolean: whether   (default: true)  
  or not to try and detect code                          
  inlined from finally blocks]                           
--excludedClasses <comma separated                       
  list of globs for classes to exclude                   
  when mutating>                                         
--excludedGroups <TestNG groups/JUnit                    
  categories to include>                                 
--excludedMethods <comma separated                       
  list of filters to match against                       
  methods to exclude from mutation                       
  analysis>                                              
--excludedTestClasses <comma separated                   
  list of globs of test classes to                       
  exclude>                                               
--exportLineCoverage [Boolean: whether  (default: true)  
  or not to dump per test line                           
  coverage data to disk]                                 
--failWhenNoMutations [Boolean:         (default: true)  
  whether to throw error if no                           
  mutations found]                                       
--features <comma separated list of                      
  features to enable/disable.>                           
--fullMutationMatrix <Boolean: Whether  (default: false) 
  to create a full mutation matrix>                      
--historyInputLocation <File: File to                    
  read history from for incremental                      
  analysis>                                              
--historyOutputLocation <File: File to                   
  write history to for incremental                       
  analysis>                                              
--includeLaunchClasspath [Boolean:      (default: true)  
  whether or not to analyse launch                       
  classpath]                                             
--includedGroups <TestNG groups/JUnit                    
  categories to include>                                 
--includedTestMethods <Test methods                      
  that should be included for                            
  challenging the mutants>                               
--jvmArgs <comma separated list of                       
  child JVM args>                                        
--jvmPath <path to java executable>                      
--maxMutationsPerClass <Integer: No     (default: 0)     
  longer supported. Use CLASSLIMIT                       
  (limit[42]) feature instead>                           
--maxSurviving <Integer: Maximum        (default: -1)    
  number of surviving mutants to allow                   
  without throwing an error>                             
--mutableCodePaths <Globs identifying                    
  classpath roots containing mutable                     
  code>                                                  
--mutationEngine <mutation engine to    (default: gregor)
  use>                                                   
--mutationThreshold <Integer: Mutation  (default: 0)     
  score below which to throw an error>                   
--mutationUnitSize <Integer: Maximum    (default: 0)     
  number of mutations to include                         
  within a single unit of analysis>                      
--mutators <comma separated list of                      
  mutation operators>                                    
--outputFormats <comma separated list   (default: HTML)  
  of listeners to receive mutation                       
  results>                                               
--pluginConfiguration <KeyValuePair:                     
  custom plugin properties>                              
* --reportDir <directory to create                       
  report folder in>                                      
--skipFailingTests [Boolean: whether    (default: false) 
  to ignore failing tests when                           
  computing coverage]                                    
* --sourceDirs <File: comma separated                    
  list of source directories>                            
* --targetClasses <comma separated                       
  list of filters to match against                       
  classes to test>                                       
--targetTests <comma separated list of                   
  filters to match against tests to                      
  run>                                                   
--testPlugin <test plugin to use>       (default: junit) 
--threads <Integer: number of threads   (default: 1)     
  to use for testing>                                    
--timeoutConst [Long: constant to       (default: 4000)  
  apply to calculate maximum test                        
  duration]                                              
--timeoutFactor [Float: factor to       (default: 1.25)  
  apply to calculate maximum test                        
  duration]                                              
--timestampedReports [Boolean: whether  (default: true)  
  or not to generated timestamped                        
  directories]                                           
--useClasspathJar [Boolean: support     (default: false) 
  large classpaths by creating a                         
  classpath jar]                                         
--verbose [Boolean: whether or not to   (default: true)  
  generate verbose output]                               
>>>> verbosity is not a recognized option

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/7.6.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD SUCCESSFUL in 3s
6 actionable tasks: 6 executed

Fix dependency vulnerabilities

Use of mutation testing in spring-boot-microservice-best-practices - Help needed

Hello there!

My name is Ana. I noted that you use the mutation testing tool in the project.
I am a postdoctoral researcher at the University of Seville (Spain), and my colleagues and I are studying how mutation testing tools are used in practice. With this aim in mind, we have analysed over 3,500 public GitHub repositories using mutation testing tools, including yours! This work has recently been published in a journal paper available at https://link.springer.com/content/pdf/10.1007/s10664-022-10177-8.pdf.

To complete this study, we are asking for your help to understand better how mutation testing is used in practice, please! We would be extremely grateful if you could contribute to this study by answering a brief survey of 21 simple questions (no more than 6 minutes). This is the link to the questionnaire https://forms.gle/FvXNrimWAsJYC1zB9.

Drop me an e-mail if you have any questions or comments ([email protected]). Thank you very much in advance!!

Explore and implement Contract Testing (pact)

NPE while performing dependency vulnerability scan with 6.1.3 & pitest 0.14

Created an issue in the original project. In the meanwhile, we will de bump the pitest dependency.
This issue is for the tracking purpose of the below original one.

jeremylong/DependencyCheck#3235

E2E Test execution is failing

Description:

The e2e test execution (Acceptance Test) is failing in the pipeline & locally.

When we ran the below command:

./gradlew test -Pe2e

Error:

No tests found for given includes: com.uk.companieshouse.e2e.*

Log trace:

> Configure project :
********************************************************
                   Executing E2E tests
********************************************************

> Task :compileJava
Note: /Users/abhishekrajput/dev/learn-concourse/spring-boot-microservice-best-practices/src/main/java/com/uk/companieshouse/expectionalhandler/ControllerExceptionHandler.java uses unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.

> Task :compileTestJava
Note: /Users/abhishekrajput/dev/learn-concourse/spring-boot-microservice-best-practices/src/test/java/com/uk/companieshouse/connector/CompaniesHouseConnectorTest.java uses unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.

> Task :test FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':test'.
> No tests found for given includes: [com.uk.companieshouse.e2e.*](filter.includeTestsMatching)

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 1m 13s
6 actionable tasks: 6 executed

Last successful execution:

https://app.circleci.com/pipelines/github/abhisheksr01/spring-boot-microservice-best-practices/362/workflows/7cf1fcbd-45d9-4c77-9cf1-d34ac2d9387f

Any other context

Tried with the 6.7.1 gradlew & iocucumber dependency of 6.10.0.
The current "./gradlew test" command doesn't execute the e2e directory evident from the build/reports as no cucumber reports are available.

Fix docker image vulnerability scanning (by trivy) in the circleci pipeline

Below vulnerabilities have been identified in the pipeline, fix them to make the pipeline green.

OWASP Dependency Vulnerability Scanning taking over 15 minutes in the CircleCI Pipeline

#Problem
Since dependency-check has moved from using the NVD data-feed to the NVD API. Without an NVD API Key dependency-check's updates will be extremely slow.

Solution

Users of dependency-check are highly encouraged to obtain an NVD API Key; see https://nvd.nist.gov/developers/request-an-api-key.Please see the documentation for the cli, maven, gradle, or ant integrations on how to set the NVD API key.

Hence we either need to start using the API key as recommended or cache the vulnerability data appropriately(which we were doing until now but seems to not working anymore)
https://github.com/jeremylong/Open-Vulnerability-Project/tree/main/vulnz#caching-the-nvd-cve-data

Rename master branch to main

As the new norm, rename the master branch to the main and update all the relevant references.

Update Dockerfile to use JDK 21 and Gradle 8.5

Vulnerable base docker image in ci.DockerFile

Cucumber HTML report not generating

After upgrading the cucumber version from 5.0.0 to 6.0.0
The plugin for generating HTML report is not working.

Class: CucumberTest.java

code snippet.
@CucumberOptions(features = "src/test/resources",
plugin = {
"pretty", "html:build/reports/cucumber", "json:build/reports/cucumber/"
})

Identify the write plugin options

Done Criteria:
All the E2E tests should pass & generate an HTML report.

Upgrade Java 17 to 21

Update all the pipelines, tool references and documentation to use Java 21.

• Councouse Pipeline
• GCP Cloud Build
• CircleCI
• Azure Pipeline
• Dockerfile
• ci.Dockerfile
• README
• sdkman config
• testhadolinttest.Dockerfile