This document describes the quality attributes currently measured for build scripts in this project and the low-level metrics used to characterize each attribute.
The README is intentionally methodology-first:
- it documents only metrics that are currently computed by the pipeline
- it groups metrics by quality attribute rather than by implementation script
- it focuses on metric definitions, formulas, and build-system-specific calculation rules
Supported build systems:
- Gradle Groovy DSL:
.gradleand.groovy - Gradle Kotlin DSL:
.gradle.kts - Maven:
pom.xml - Ant:
build.xml
- 1. Scope and Conventions
- 2. Quality Attribute Overview
- 3. Complexity
- 4. Dependency Quality
- 5. Determinism and Reproducibility
- 6. Maintainability
- 7. Coupling and Cohesion
- 8. Evolution and Change Activity
- 9. Security
- 10. Reliability
The before/after pipeline reports most metrics with paired fields:
*_Before*_After*_Deltawhen a direct delta is meaningful
When the GitHub commit runner is used, the pipeline also creates a per-commit output directory:
results/commits/<commit_sha>/
That directory contains one CSV per focused view for the changed build files in that commit:
summary_metrics.csvmaintainability_metrics.csvunderstandability_metrics.csvmodularity_metrics.csvsecurity_metrics.csvreliability_metrics.csv
Each CSV stores one row per changed build file in that commit, including the full File_Path, the File_Name, and the metrics for that quality view. The pipeline does not create separate per-file CSV folders inside the commit directory.
The formulas below are written for a single build file b. The pipeline applies the same definitions to both snapshots.
Several quality attributes reuse the same base quantities.
| Symbol / Metric | Definition | Calculation |
|---|---|---|
BLOC(b) |
Build lines of code for build file b. |
Obtained from scc as the Code count for the file. |
Lines(b) |
Total physical line count. | Obtained from scc as the Lines count. |
CommentLines(b) |
Number of comment lines. | Obtained from scc as the Comment count. |
NonEmptyLines(b) |
Number of non-blank lines. | Available as an auxiliary line-count measure when blank-line filtering is needed. |
Build_Modularity_*fields are referenced by the pipeline, but the implementation source is not present in the current workspace. They are therefore not treated as documented metrics in this README.- Conceptual or proposed metrics that are not currently computed by the pipeline are intentionally omitted.
- When a metric is unsupported for a specific build syntax, that limitation is stated explicitly in the relevant section.
| Quality Attribute | Low-Level Metrics | Derived / Aggregate Metrics |
|---|---|---|
| Complexity | BLOC, Cyclomatic_Complexity, Normalized_CC, Halstead_Volume, Normalized_HV |
none |
| Dependency Quality | Dependency_Count, Fixed_Dependency_Count, Dynamic_Dependency_Count, Snapshot_Dependency_Count, Unknown_Dependency_Count |
DSS |
| Determinism and Reproducibility | Non_Deterministic_Constructs, Non_Deterministic_Summary |
BDS |
| Understandability | Style_Conformance_Score, Comment_Ratio, Comment_Readability, Normalized_CC, Normalized_HV, Clone_Density |
methodology-level US |
| Maintainability | BLOC, Cyclomatic_Complexity, Normalized_CC, Halstead_Volume, Normalized_HV, Clone_Density, Maintainability_Smell_Count, Maintainability_Smell_Density, Maintainability_Smell_Summary |
methodology-level Maintainability |
| Coupling and Cohesion | CP_Internal, CP_External, CP_Total, NCP_Internal, NCP_External, Coupling_Ratio, Build_Cohesion |
EDR reuses coupling components |
| Modularity | CP_Internal, CP_External, NCP_Internal, NCP_External, Build_Cohesion, Clone_Density |
methodology-level MS; focused modularity_metrics.csv export |
| Evolution and Change Activity | Churn, Change_Frequency, Avg_Logical_LOC, Normalized_Churn, Normalized_Change_Frequency |
none |
| Security | Security_Smell_Count, Security_Smell_Density, Security_Smell_Summary, HARDCODED_CREDENTIALS, INSECURE_URLS, WILDCARD_USAGE, HARDCODED_PATHS_AND_URLS, DEPRECATED_DEPENDENCIES, OUTDATED_DEPENDENCIES |
focused security_metrics.csv export |
| Reliability | RE, DSS, EDR |
focused reliability_metrics.csv export |
Style_Conformance_Score is a normalized metric in the 0..1 range:
Style_Conformance_Score(b) = max(0, 1 - (violations(b) / BLOC(b)))
The pipeline also emits a focused understandability output file:
results/understandability_metrics.csv
Complexity captures the structural and cognitive burden of understanding and modifying a build script.
| Metric | Definition | Formula |
|---|---|---|
BLOC |
Build lines of code. | BLOC(b) = code_lines(b) |
Cyclomatic_Complexity |
File-level decision or build-logic complexity. | Build-system-specific calculation described below. |
Normalized_CC |
Size-normalized complexity. | Normalized_CC(b) = Cyclomatic_Complexity(b) / BLOC(b) |
Halstead_Volume |
Halstead volume of the build script. | HV(b) = (N1 + N2) * log2(n1 + n2) |
Normalized_HV |
Size-normalized Halstead volume. | Normalized_HV(b) = Halstead_Volume(b) / BLOC(b) |
Cyclomatic_Complexityis computed with CodeNarc and aggregated at file level.Halstead_Volumeis computed from a Groovy AST-based operator/operand extractor.
Cyclomatic_Complexityis computed with detekt by summing file-levelCyclomaticComplexMethodfindings.Halstead_Volumeis currently not implemented for Kotlin DSL in this pipeline, so the emitted value is0.0.
Cyclomatic_Complexity is implemented as a build-logic complexity heuristic:
BLC_maven = 1 + count(profile) + count(activation) + count(execution)
Halstead_Volume is computed from XML structure:
- operators: XML tag names
- operands: child-tag occurrences
Cyclomatic_Complexity is implemented as a build-logic complexity heuristic:
BLC_ant = 1
+ count(condition/operator tags)
+ count(if attributes)
+ count(unless attributes)
+ count(extra dependency edges from depends)
Halstead_Volume is computed from XML structure:
- operators: non-
projectand non-descriptionXML tags - operands: attribute names
Dependency quality reflects how explicitly and stably a build file declares the external artifacts it consumes.
| Metric | Definition | Formula |
|---|---|---|
Dependency_Count |
Total number of detected dependencies considered by the parser. | Count of parsed dependency declarations. |
Fixed_Dependency_Count |
Dependencies pinned to a fixed release version. | Count of dependencies classified as fixed. |
Dynamic_Dependency_Count |
Dependencies declared with dynamic or range-based versions. | Count of dependencies classified as dynamic. |
Snapshot_Dependency_Count |
Dependencies declared with snapshot-style versions. | Count of dependencies classified as snapshot. |
Unknown_Dependency_Count |
Dependencies whose version cannot be resolved to a concrete version from file contents. | Count of dependencies classified as unknown. |
DSS |
Dependency Stability Score. | DSS(b) = Fixed_Dependency_Count(b) / Dependency_Count(b) |
If Dependency_Count(b) = 0, the implementation returns:
DSS(b) = 0.0
| Class | Meaning | Examples |
|---|---|---|
fixed |
Explicit pinned release version. | 1.2.3 |
dynamic |
Range, floating, or mutable version expression. | 1.+, [1.0,2.0), latest.release |
snapshot |
Snapshot-style mutable version. | 1.2.3-SNAPSHOT |
unknown |
Version not concretely recoverable from file contents. | unresolved property or catalog reference |
- Dependency declarations are extracted from string-style and map-style dependency declarations.
- Local property-resolution heuristics are applied before classifying versions.
- Dependency versions are read from dependency declarations.
- If a dependency omits a local version, the pipeline consults local
dependencyManagementwhen available. - Local
<properties>are resolved before classification.
- Dependencies are inferred from versioned JAR references found in XML attribute values.
- Versions are extracted from JAR file names after resolving simple property references.
Determinism measures the degree to which a build script avoids constructs that can make repeated executions produce variable results.
| Metric | Definition | Formula |
|---|---|---|
Non_Deterministic_Constructs |
Number of detected non-deterministic constructs. | Count of lines matching any supported non-deterministic construct family. |
Non_Deterministic_Summary |
Set of non-deterministic construct families present. | Semicolon-separated set of detected construct labels. |
BDS |
Build Script Determinism Score. | BDS(b) = max(0, 1 - (Non_Deterministic_Constructs(b) / BLOC(b))) |
If BLOC(b) = 0, the implementation returns:
BDS(b) = 0.0
| Family | Meaning | Representative Examples |
|---|---|---|
TIME |
Time-dependent values introduced at build time. | System.currentTimeMillis(), Instant.now(), new Date(), ${maven.build.timestamp} |
RANDOMNESS |
Randomness sources that may change outputs between runs. | Math.random(), new Random(), SecureRandom, UUID.randomUUID() |
NON_REPRODUCIBLE_STEP |
Explicit mutable fetch or checkout steps. | curl, wget, Invoke-WebRequest, git clone, svn checkout, Ant <get> |
BDS is implemented for all supported build systems:
- Gradle
- Maven
- Ant
The current detector is text-based and scans the file contents for the construct families above.
Maintainability captures how easy a build script is to inspect, modify, and evolve without introducing extra effort or breakage.
For maintainability, this project now considers only:
BLOCCyclomatic_ComplexityHalstead_VolumeClone_Density- maintainability smells
The pipeline also emits a maintainability-focused output file:
results/maintainability_metrics.csv
A maintainability score can be expressed as:
Maintainability(b) =
(
1 / (1 + BLOC(b))
+ 1 / (1 + Normalized_CC(b))
+ 1 / (1 + Normalized_HV(b))
+ (1 - Clone_Density(b))
+ (1 - MSD(b))
) / 5
Where:
Normalized_CC(b)=Cyclomatic_Complexity(b) / BLOC(b)Normalized_HV(b)=Halstead_Volume(b) / BLOC(b)MSD(b)=Maintainability_Smell_Density
Interpretation:
- lower build size, lower control-flow complexity, and lower Halstead volume increase maintainability
- higher duplication reduces maintainability through
1 - Clone_Density(b) - higher smell density reduces maintainability through inverse normalization
| Metric | Definition | Formula |
|---|---|---|
BLOC |
Build lines of code. | BLOC(b) = code_lines(b) |
Cyclomatic_Complexity |
File-level decision or build-logic complexity. | Build-system-specific calculation described in Section 3.2. |
Normalized_CC |
Size-normalized cyclomatic complexity used in the maintainability CSV. | Normalized_CC(b) = Cyclomatic_Complexity(b) / BLOC(b) |
Halstead_Volume |
Halstead volume of the build script. | HV(b) = (N1 + N2) * log2(n1 + n2) |
Normalized_HV |
Size-normalized Halstead volume used in the maintainability CSV. | Normalized_HV(b) = Halstead_Volume(b) / BLOC(b) |
Clone_Density |
Fraction of duplicated build logic lines. | Clone_Density(b) = duplicated_build_logic_lines(b) / BLOC(b) |
| Metric | Definition | Formula |
|---|---|---|
Maintainability_Smell_Count |
Number of maintainability smell findings. | Count of detected maintainability smells. |
Maintainability_Smell_Density |
Smell density normalized by build lines of code. | Maintainability_Smell_Density(b) = Maintainability_Smell_Count(b) / BLOC(b) |
Maintainability_Smell_Summary |
Set of maintainability smell categories present. | Semicolon-separated set of smell identifiers. |
The maintainability-focused CSV includes:
BLOC_BeforeandBLOC_AfterCyclomatic_Complexity_BeforeandCyclomatic_Complexity_AfterNormalized_CC_BeforeandNormalized_CC_AfterHalstead_Volume_BeforeandHalstead_Volume_AfterNormalized_HV_BeforeandNormalized_HV_AfterClone_Density_BeforeandClone_Density_AfterMaintainability_Smell_Count_BeforeandMaintainability_Smell_Count_AfterMaintainability_Smell_Density_BeforeandMaintainability_Smell_Density_AfterMaintainability_Smell_Summary_BeforeandMaintainability_Smell_Summary_After
Tracked maintainability smell categories:
EMPTY_INCOMPLETE_TAGSINCONSISTENT_DEPENDENCY_MANAGEMENTLACK_OF_ERROR_HANDLINGMISSING_DEPENDENCY_VERSIONSUSPICIOUS_COMMENTSDEPRECATED_DEPENDENCIESOUTDATED_DEPENDENCIES
BLOC,Cyclomatic_Complexity, andHalstead_Volumereuse the definitions from Section 3.Clone_Densityis computed with PMD CPD when available, with a repeated-line-window fallback when the preferred duplication path is unavailable.- Maintainability smells are counted without weighting; each finding contributes equally to the smell count.
Coupling and cohesion describe the extent to which build logic is externally connected and internally related.
| Metric | Definition | Formula |
|---|---|---|
CP_Internal |
Internal coupling among elements inside the same build file. | CP_Internal(b) = T_int + V_shared + C_internal |
CP_External |
Coupling driven by external modules, artifacts, repositories, tools, and resources. | CP_External(b) = M + D + P + R + E + U |
CP_Total |
Total coupling. | CP_Total(b) = CP_Internal(b) + CP_External(b) |
NCP_Internal |
Size-normalized internal coupling. | NCP_Internal(b) = CP_Internal(b) / BLOC(b) |
NCP_External |
Size-normalized external coupling. | NCP_External(b) = CP_External(b) / BLOC(b) |
Coupling_Ratio |
Share of total coupling that is external. | Coupling_Ratio(b) = CP_External(b) / CP_Total(b) |
Component meanings:
T_int: internal task or target dependency linksV_shared: variables or properties shared by multiple internal elementsC_internal: internal configuration references reused across elementsM: inter-module referencesD: external dependenciesP: pluginsR: repositories or remote artifact sourcesE: external commands or build-script execution hooksU: environment variables, absolute paths, and URL-based resources
| Metric | Definition | Formula |
|---|---|---|
Build_Cohesion |
Average pairwise feature overlap among build elements. | Average pairwise Jaccard similarity across extracted element feature sets. |
The elements compared depend on the build system:
- Gradle: task feature sets
- Maven: plugin execution feature sets
- Ant: target feature sets
Modularity is not currently exported as a standalone implemented pipeline field in this workspace, but it can be operationalized from cohesion and coupling as:
MS(b) =
(
Build_Cohesion(b)
+ (1 - Coupling_Ratio(b))
+ 1 / (1 + NCP_External(b))
+ (1 - Clone_Density(b))
) / 4
Where:
MS(b)= modularity scoreBuild_Cohesion(b)rewards stronger internal relatedness of build elements1 - Coupling_Ratio(b)rewards a lower proportion of external coupling1 / (1 + NCP_External(b))rewards lower size-normalized external coupling1 - Clone_Density(b)rewards lower duplication across build logic
Interpretation:
- higher cohesion, lower external coupling, and lower duplication indicate stronger modularity
The pipeline also emits a focused modularity output file with the separately tracked low-level metrics:
results/modularity_metrics.csv
The modularity-focused CSV includes:
CP_Internal_BeforeandCP_Internal_AfterCP_External_BeforeandCP_External_AfterNCP_Internal_BeforeandNCP_Internal_AfterNCP_External_BeforeandNCP_External_AfterBuild_Cohesion_BeforeandBuild_Cohesion_AfterClone_Density_BeforeandClone_Density_After
Evolution metrics characterize how frequently a build file changes and how much code churn it experiences over time.
| Metric | Definition | Formula |
|---|---|---|
Churn |
Total added and deleted lines over the observation window. | Churn(f, T) = sum(added_lines + deleted_lines) |
Change_Frequency |
Number of commits touching the file during the observation window. | Change_Frequency(f, T) = number_of_commits_touching_f |
Avg_Logical_LOC |
Average logical LOC of the file over historical snapshots in the window. | Mean historical BLOC across the observation window. |
Normalized_Churn |
Size-normalized churn. | Normalized_Churn(f, T) = Churn(f, T) / Avg_Logical_LOC(f, T) |
Normalized_Change_Frequency |
Size-normalized change frequency. | Normalized_Change_Frequency(f, T) = (Change_Frequency(f, T) / Avg_Logical_LOC(f, T)) * 100 |
The current before/after pipeline uses a rolling observation window ending at the analyzed commit.
Security is characterized through smell-based indicators rather than a single aggregate security score.
A security score can be operationalized directly from the security smell density:
SS(b) = max(0, 1 - (Security_Smell_Count(b) / max(BLOC(b), 1)))
Equivalently, using the documented density metric:
SS(b) = max(0, 1 - Security_Smell_Density(b))
Where:
SS(b)= security scoreSecurity_Smell_Count(b)counts security smell findingsSecurity_Smell_Density(b)normalizes those findings by build lines of code
Interpretation:
- more security smells imply lower security
| Metric | Definition | Formula |
|---|---|---|
Security_Smell_Count |
Number of detected security smell findings. | Count of detected security smells. |
Security_Smell_Density |
Security smell density normalized by build lines of code. | Security_Smell_Density(b) = Security_Smell_Count(b) / BLOC(b) |
Security_Smell_Summary |
Set of security smell categories present. | Semicolon-separated set of smell identifiers. |
Tracked security smell categories:
HARDCODED_CREDENTIALSINSECURE_URLSWILDCARD_USAGEHARDCODED_PATHS_AND_URLSDEPRECATED_DEPENDENCIESOUTDATED_DEPENDENCIES
The pipeline also emits a focused security output file:
results/security_metrics.csv
The security-focused CSV includes:
Before_Security_Smell_CountandAfter_Security_Smell_CountBefore_Security_Smell_DensityandAfter_Security_Smell_DensityBefore_Security_Smell_SummaryandAfter_Security_Smell_SummaryBefore_HARDCODED_CREDENTIALSandAfter_HARDCODED_CREDENTIALSBefore_INSECURE_URLSandAfter_INSECURE_URLSBefore_WILDCARD_USAGEandAfter_WILDCARD_USAGEBefore_HARDCODED_PATHS_AND_URLSandAfter_HARDCODED_PATHS_AND_URLSBefore_DEPRECATED_DEPENDENCIESandAfter_DEPRECATED_DEPENDENCIESBefore_OUTDATED_DEPENDENCIESandAfter_OUTDATED_DEPENDENCIES
Reliability captures build-script robustness using issue density, dependency stability, and external-system reliance.
| Metric | Definition | Formula |
|---|---|---|
Reliability_Issues |
Total count of reliability-relevant smell findings. | RI(b) = HC(b) + IU(b) + WU(b) + HP(b) + DD(b) + OD(b) |
RE |
Issue-based reliability score. | RE(b) = max(0, 1 - (RI(b) / BLOC(b))) |
EDR |
External Dependency Risk. | EDR(b) = (D + P + R + E + U) / CP_Total(b) |
RM |
Overall reliability metric. | RM(b) = (RE(b) + DSS(b) + (1 - EDR(b))) / 3 |
Where:
HC= hardcoded credentials countIU= insecure URLs countWU= wildcard usage countHP= hardcoded paths or URLs countDD= deprecated dependencies countOD= outdated dependencies count
- higher
REmeans fewer reliability issues per build line - higher
DSSmeans a larger share of dependencies are pinned to fixed versions - lower
EDRmeans less reliance on external systems
The pipeline also emits a focused reliability output file:
results/reliability_metrics.csv
The reliability-focused CSV includes:
RE_BeforeandRE_AfterDSS_BeforeandDSS_AfterEDR_BeforeandEDR_After- higher
RMtherefore indicates a more reliable build file overall
The overall reliability metric used in this project is:
RM(b) = (RE(b) + DSS(b) + (1 - EDR(b))) / 3
This combines:
- issue-based reliability through
RE - dependency stability through
DSS - reduced external-system exposure through
1 - EDR
EDR is derived from the coupling model:
EDR(b) = (D + P + R + E + U) / CP_Total(b)
Local module links M are intentionally excluded from the EDR numerator because they represent project-internal structure rather than reliance on external systems.
The current implementation keeps two reliability views:
RE: issue-based reliability derived only from reliability issues andBLOCRM: overall reliability derived fromRE,DSS, andEDR
BDS is intentionally not folded into RM in the current pipeline; it remains a separate determinism attribute.