scijava / scijava-scripts Goto Github PK

View Code? Open in Web Editor NEW

15.0 17.0 16.0 621 KB

Shell scripts for scientific projects written in Java

License: The Unlicense

Shell 92.26% Perl 6.68% Python 1.06%

scijava-scripts's People

Contributors

Stargazers

Watchers

Forkers

thbar ralic gab1one hadim imagejan joshmoore imcf tferr kapoorlab skalarproduktraum ryan-feeley oeway jiapei100 apenshar houssoli celinetrebeau

scijava-scripts's Issues

Avoid failure if no sources with release-version.sh

When using release-version.sh in projects without attached sources, the script "fails" - but at that point the core artifact has already been uploaded. So maybe execution should just continue?

If we want jobs without sources to fail, the check should come beforehand (e.g. with an enforcer plugin)

Teach bump-versions-in-pom.sh about import scope dependencies

pom-imagej now imports pom-scifio...

Automate generation of release notes

Quoting myself from the scijava-common gitter channel:

I have been thinking of how to automate release notes management to some extent. At the moment, at every release I go through all commits and try to find bug fixes/new features/(interface breaking changes), e.g:
https://github.com/saalfeldlab/paintera/releases

This is, of course, unreliable because I may forget to do that or it can be hard to find all relevant contributions when there were many commits between releases. Instead I thought that maybe there could be a release-notes.md in the repository with all previous releases and a current release note. Every pull request would need to modify the current release note appropriately and after a release, the current release note could just simply be copied into the release note/tag for that version.

This would simplify the process of figuring out what changes happened for a PR and I think this could even be automated with the scijava-scripts: The content of the current release note could be attached to the tag automatically and the suggested release version could be inferred from the contents of the release note itself. In that case it may be better to use something like xml/json/yaml instead of md for the current release note. I know of jq as a very good json parser, so maybe json is the best choice here. I personally like yaml best for this use case but I don't think there is a good bash tool.

This is just something that came to my mind and there's probably some flaws but I would love to improve (gathering) release notes and ideally automate the process.

Melting pot with `-c ... -p` flags fails

I probably broke the melting pot's support for pruning out components not depending on a changed component. In pom-scjiava I ran:

tests/run.sh -c net.imagej:imagej-common:2.0.0-SNAPSHOT -p

Hoping I could build a mega-melt that tested only components depending on imagej-common, against a locally-built imagej-common 2.0.0-SNAPSHOT I had installed. But during the pruning phase, it died with:

[INFO] Checking relevance of component net.imagej/imagej-legacy
[INFO] Checking relevance of component net.imagej/imagej-matlab
[INFO] Checking relevance of component net.imagej/imagej-mesh-io
[ERROR] CannotMelting pot generation failed!

Would be nice to make this scenario work.

Enable logic to fail the release if SemVer is broken

The revapi-maven-plugin lets us programmatically test for API regressions (which should be SemVer major version bumps). Likely it can also flag API additions (which should be SemVer minor version bumps). We could hook this up to the release-version.sh script, such that whatever version bump is happening since last release is consistent with the reality of the API.

travis-build: also run javadoc generation for PRs

It seems that the maven-javadoc-plugin is not executed when Building the artifact locally only, i.e. for Travis builds that are triggered by open PRs.

See for example: https://travis-ci.org/ijpb/MorphoLibJ/builds/364542659

melting-pot: deal with clashing artifactId strings

Some dependencies of the SciJava component collection have the same artifactId name with different groupIds:

annotations (net.sourceforge.findbugs & org.jetbrains)
antlr (antlr & org.antlr)
jocl (org.jocl & org.jogamp.jocl)
kryo (com.esotericsoftware & com.esotericsoftware.kryo)
minlog (com.esotericsoftware & com.esotericsoftware.minlog)
trove4j (net.sf.trove4j & org.jetbrains.intellij.deps)

Unfortunately, the melting pot currently uses properties of the form <artifactId>.version to override component versions. We need to update the melting-pot script to support properties that successfully disambiguate between components, ideally without breaking backwards compatibility with the current scheme for the 99% of artifacts where artifactId is sufficient.

Travisify.sh: no repoSlug on Windows (GitBash)

I had a chance trying travisify.sh on my Windows system (using GitBash/MINGW64), and it errors out with:

[ERROR] Could not determine GitHub repository slug

because this line:

scijava-scripts/travisify.sh

Line 69 in cbc3ed2

 repoSlug=$(xmllint --xpath "//*[local-name()='project']/*[local-name()='scm']/*[local-name()='connection']" pom.xml|sed 's_.*github.com[:/]\(.*\)<.*_\1_') 

or more precisely this command:

 xmllint --xpath "//*[local-name()='project']/*[local-name()='scm']/*[local-name()='connection']" pom.xml

gives:

XPath set is empty

although the pom.xml contains a project/scm/connection object.
(This was tested with the pom.xml of our https://github.com/fmi-faim/fmi-ij2-plugins/ repo.)

melting-pot: what to do about multi-module builds?

When we create the mega-melt pom.xml we create a multi-module build where each dependency is a module.

If any of those dependencies are themselves modules of a multi-module build, where their aggregator is also their parent pom (i.e. https://github.com/axtimwalde/mpicbg), the now-orphaned modules appear to be unable to find their parents.

A heavy-handed suggestion: all child modules should include scijava repository blocks:

	<repositories>
		<repository>
			<id>scijava.public</id>
			<url>https://maven.scijava.org/content/groups/public</url>
		</repository>
	</repositories>

This way, the children will always know how to find their parents.

Fix git-synchronizer for SourceForge's insistence to disallow forced pushes

Moved from scijava/scijava-common#20.

We do allow forced pushes everywhere (even the master branches, although that is strongly discouraged). Therefore, we need to fix https://github.com/scijava/scijava-common/blob/master/bin/git-synchronizer.sh#L83 so that it deletes the offending branches from SourceForge's repositories first (the only work-around to "allow" forced updates).

release-version.sh failing on Windows Git Bash

As mentioned in ijpb/MorphoLibJ#62 (comment), it seems there's a general issue with release-version.sh when used in Git Bash. I was now able to reproduce the behavior in imagej-plugins-search-biii and get the following output:

Full output of release-version.sh

Version? [0.0.1]:
Detected DEFAULT version bump
From github.com:imagej/imagej-plugins-search-biii
 * branch            master     -> FETCH_HEAD
[INFO] Scanning for projects...
[INFO]
[INFO] ---------------< net.imagej:imagej-plugins-search-biii >----------------
[INFO] Building ImageJ Search Plugins for BIII 0.0.1-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- license-maven-plugin:2.0.0:update-project-license (default-cli) @ imagej-plugins-search-biii ---
[INFO] Adding a license repository jar:file:/C:/Users/eglijan/.m2/repository/org/codehaus/mojo/license-maven-plugin/2.0.0/license-maven-plugin-2.0.0.jar!/META-INF/licenses
[INFO] register COMMON DEVELOPMENT AND DISTRIBUTION LICENSE (CDDL) Version 1.0
[INFO] register GNU General Public License (GPL) version 3.0
[INFO] register MIT-License
[INFO] register GNU General Public License (GPL) version 2.0
[INFO] register GNU General Public License (GPL) version 1.0
[INFO] register GNU Free Documentation License (FDL) version 1.3
[INFO] register Eclipse Public License - v 2.0 with Secondary License
[INFO] register Eclipse Public + Distribution License - v 1.0
[INFO] register Eclipse Public License - v 2.0
[INFO] register Eclipse Public License - v 1.0
[INFO] register GNU Affero General Public License (AGPL) version 3.0
[INFO] register GNU General Lesser Public License (LGPL) version 3.0
[INFO] register GNU General Lesser Public License (LGPL) version 2.1
[INFO] register Apache License version 2.0
[INFO] register European Union Public License v1.1
[INFO] register BSD 3-Clause License
[INFO] register BSD 2-Clause License
[INFO]
[INFO] --- license-maven-plugin:2.0.0:update-file-header (default-cli) @ imagej-plugins-search-biii ---
[INFO] Adding a license repository jar:file:/C:/Users/eglijan/.m2/repository/org/codehaus/mojo/license-maven-plugin/2.0.0/license-maven-plugin-2.0.0.jar!/META-INF/licenses
[INFO] register COMMON DEVELOPMENT AND DISTRIBUTION LICENSE (CDDL) Version 1.0
[INFO] register GNU General Public License (GPL) version 3.0
[INFO] register MIT-License
[INFO] register GNU General Public License (GPL) version 2.0
[INFO] register GNU General Public License (GPL) version 1.0
[INFO] register GNU Free Documentation License (FDL) version 1.3
[INFO] register Eclipse Public License - v 2.0 with Secondary License
[INFO] register Eclipse Public + Distribution License - v 1.0
[INFO] register Eclipse Public License - v 2.0
[INFO] register Eclipse Public License - v 1.0
[INFO] register GNU Affero General Public License (AGPL) version 3.0
[INFO] register GNU General Lesser Public License (LGPL) version 3.0
[INFO] register GNU General Lesser Public License (LGPL) version 2.1
[INFO] register Apache License version 2.0
[INFO] register European Union Public License v1.1
[INFO] register BSD 3-Clause License
[INFO] register BSD 2-Clause License
[INFO] Will search files to update from root C:\code\imagej\imagej-plugins-search-biii\src
[INFO] Will search files to update from root C:\code\imagej\imagej-plugins-search-biii\target\generated-sources
[INFO] Scan 3 files header done in 30.847ms.
[INFO]
 * update header on 3 files.
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  1.607 s
[INFO] Finished at: 2023-01-24T13:13:08+01:00
[INFO] ------------------------------------------------------------------------
[INFO] Scanning for projects...
[INFO]
[INFO] ---------------< net.imagej:imagej-plugins-search-biii >----------------
[INFO] Building ImageJ Search Plugins for BIII 0.0.1-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- maven-release-plugin:2.5.3:prepare (default-cli) @ imagej-plugins-search-biii ---
[INFO] Verifying that there are no local modifications...
[INFO]   ignoring changes on: **\pom.xml.next, **\release.properties, **\pom.xml.branch, **\pom.xml.tag, **\pom.xml.backup, **\pom.xml.releaseBackup
[INFO] Executing: cmd.exe /X /C "git rev-parse --show-toplevel"
[INFO] Working directory: C:\code\imagej\imagej-plugins-search-biii
[INFO] Executing: cmd.exe /X /C "git status --porcelain ."
[INFO] Working directory: C:\code\imagej\imagej-plugins-search-biii
[WARNING] Ignoring unrecognized line: ?? release.properties
[INFO] Checking dependencies and plugins for snapshots ...
[INFO] Transforming 'ImageJ Search Plugins for BIII'...
[INFO] Not generating release POMs
[INFO] Executing goals 'clean verify'...
[INFO] [INFO] Scanning for projects...
[INFO] [INFO]
[INFO] [INFO] ---------------< net.imagej:imagej-plugins-search-biii >----------------
[INFO] [INFO] Building ImageJ Search Plugins for BIII 0.0.1
[INFO] [INFO] --------------------------------[ jar ]---------------------------------
[INFO] [INFO]
[INFO] [INFO] --- maven-clean-plugin:3.1.0:clean (default-clean) @ imagej-plugins-search-biii ---
[INFO] [INFO] Deleting C:\code\imagej\imagej-plugins-search-biii\target
[INFO] [INFO]
[INFO] [INFO] --- maven-enforcer-plugin:3.0.0:enforce (enforce-rules) @ imagej-plugins-search-biii ---
[INFO] [INFO] Adding ignore: module-info
[INFO] [INFO] Adding ignore: META-INF/versions/*/module-info
[INFO] [INFO] Adding ignore: com.google.inject.*
[INFO] [INFO] Adding ignore: javax.xml.namespace.QName
[INFO] [INFO] Adding ignore: jnr.ffi.*
[INFO] [INFO] Adding ignore: org.apache.hadoop.yarn.*.package-info
[INFO] [INFO] Adding ignore: org.apache.spark.unused.UnusedStubClass
[INFO] [INFO] Adding ignore: org.eclipse.aether.*
[INFO] [INFO] Adding ignore: org.hibernate.stat.ConcurrentStatisticsImpl
[INFO] [INFO] Adding ignore: org.jetbrains.kotlin.daemon.common.*
[INFO] [INFO] Adding ignore: org.junit.runner.Runner
[INFO] [INFO] Adding ignore: org.jzy3d.plot3d.pipelines.*
[INFO] [INFO] Adding ignore: module-info
[INFO] [INFO] Adding ignore: module-info
[INFO] [INFO]
[INFO] [INFO] --- maven-help-plugin:3.2.0:active-profiles (show-active-profiles) @ imagej-plugins-search-biii ---
[INFO] [INFO]
[INFO] Active Profiles for Project 'net.imagej:imagej-plugins-search-biii:jar:0.0.1':
[INFO]
[INFO] The following profiles are active:
[INFO]
[INFO]  - deploy-to-scijava (source: org.scijava:pom-scijava-base:15.0.3)
[INFO]  - scijava-platform-amd64 (source: org.scijava:pom-scijava-base:15.0.3)
[INFO]  - scijava-platform-windows (source: org.scijava:pom-scijava-base:15.0.3)
[INFO]
[INFO]
[INFO]
[INFO] [INFO]
[INFO] [INFO] --- build-helper-maven-plugin:3.2.0:regex-property (sanitize-version) @ imagej-plugins-search-biii ---
[INFO] [INFO]
[INFO] [INFO] --- build-helper-maven-plugin:3.2.0:regex-property (guess-package) @ imagej-plugins-search-biii ---
[INFO] [INFO]
[INFO] [INFO] --- buildnumber-maven-plugin:1.4:create (default) @ imagej-plugins-search-biii ---
[INFO] [INFO] Executing: cmd.exe /X /C "git rev-parse --verify HEAD"
[INFO] [INFO] Working directory: C:\code\imagej\imagej-plugins-search-biii
[INFO] [INFO] Storing buildNumber: 5714ca6521a21b8eb91fc527cfd7e80adb5fa36a at timestamp: 1674562394460
[INFO] [INFO] Storing buildScmBranch: master
[INFO] [INFO]
[INFO] [INFO] --- scijava-maven-plugin:2.1.0:set-rootdir (set-rootdir) @ imagej-plugins-search-biii ---
[INFO] [INFO] Setting rootdir: C:\code\imagej\imagej-plugins-search-biii
[INFO] [INFO]
[INFO] [INFO] --- jacoco-maven-plugin:0.8.7:prepare-agent (jacoco-initialize) @ imagej-plugins-search-biii ---
[INFO] [INFO] argLine set to -javaagent:C:\\Users\\eglijan\\.m2\\repository\\org\\jacoco\\org.jacoco.agent\\0.8.7\\org.jacoco.agent-0.8.7-runtime.jar=destfile=C:\\code\\imagej\\imagej-plugins-search-biii\\target\\jacoco.exec
[INFO] [INFO]
[INFO] [INFO] --- maven-resources-plugin:3.2.0:resources (default-resources) @ imagej-plugins-search-biii ---
[INFO] [INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] [INFO] Using 'UTF-8' encoding to copy filtered properties files.
[INFO] [INFO] skip non existing resourceDirectory C:\code\imagej\imagej-plugins-search-biii\src\main\resources
[INFO] [INFO]
[INFO] [INFO] --- maven-compiler-plugin:3.8.1:compile (default-compile) @ imagej-plugins-search-biii ---
[INFO] [INFO] Compiling 3 source files to C:\code\imagej\imagej-plugins-search-biii\target\classes
[INFO] [INFO]
[INFO] [INFO] --- maven-resources-plugin:3.2.0:testResources (default-testResources) @ imagej-plugins-search-biii ---
[INFO] [INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] [INFO] Using 'UTF-8' encoding to copy filtered properties files.
[INFO] [INFO] skip non existing resourceDirectory C:\code\imagej\imagej-plugins-search-biii\src\test\resources
[INFO] [INFO]
[INFO] [INFO] --- maven-compiler-plugin:3.8.1:testCompile (default-testCompile) @ imagej-plugins-search-biii ---
[INFO] [INFO] Nothing to compile - all classes are up to date
[INFO] [INFO]
[INFO] [INFO] --- maven-surefire-plugin:2.22.2:test (default-test) @ imagej-plugins-search-biii ---
[INFO] [INFO] No tests to run.
[INFO] [INFO]
[INFO] [INFO] --- maven-jar-plugin:3.2.0:jar (default-jar) @ imagej-plugins-search-biii ---
[INFO] [INFO] Building jar: C:\code\imagej\imagej-plugins-search-biii\target\imagej-plugins-search-biii-0.0.1.jar
[INFO] [INFO]
[INFO] [INFO] >>> maven-source-plugin:3.2.1:jar (attach-sources-jar) > generate-sources @ imagej-plugins-search-biii >>>
[INFO] [INFO]
[INFO] [INFO] --- maven-enforcer-plugin:3.0.0:enforce (enforce-rules) @ imagej-plugins-search-biii ---
[INFO] [INFO] Adding ignore: module-info
[INFO] [INFO] Adding ignore: META-INF/versions/*/module-info
[INFO] [INFO] Adding ignore: com.google.inject.*
[INFO] [INFO] Adding ignore: javax.xml.namespace.QName
[INFO] [INFO] Adding ignore: jnr.ffi.*
[INFO] [INFO] Adding ignore: org.apache.hadoop.yarn.*.package-info
[INFO] [INFO] Adding ignore: org.apache.spark.unused.UnusedStubClass
[INFO] [INFO] Adding ignore: org.eclipse.aether.*
[INFO] [INFO] Adding ignore: org.hibernate.stat.ConcurrentStatisticsImpl
[INFO] [INFO] Adding ignore: org.jetbrains.kotlin.daemon.common.*
[INFO] [INFO] Adding ignore: org.junit.runner.Runner
[INFO] [INFO] Adding ignore: org.jzy3d.plot3d.pipelines.*
[INFO] [INFO] Adding ignore: module-info
[INFO] [INFO] Adding ignore: module-info
[INFO] [INFO]
[INFO] [INFO] --- maven-help-plugin:3.2.0:active-profiles (show-active-profiles) @ imagej-plugins-search-biii ---
[INFO] [INFO]
[INFO] Active Profiles for Project 'net.imagej:imagej-plugins-search-biii:jar:0.0.1':
[INFO]
[INFO] The following profiles are active:
[INFO]
[INFO]  - deploy-to-scijava (source: org.scijava:pom-scijava-base:15.0.3)
[INFO]  - scijava-platform-amd64 (source: org.scijava:pom-scijava-base:15.0.3)
[INFO]  - scijava-platform-windows (source: org.scijava:pom-scijava-base:15.0.3)
[INFO]
[INFO]
[INFO]
[INFO] [INFO]
[INFO] [INFO] --- build-helper-maven-plugin:3.2.0:regex-property (sanitize-version) @ imagej-plugins-search-biii ---
[INFO] [INFO]
[INFO] [INFO] --- build-helper-maven-plugin:3.2.0:regex-property (guess-package) @ imagej-plugins-search-biii ---
[INFO] [INFO]
[INFO] [INFO] --- buildnumber-maven-plugin:1.4:create (default) @ imagej-plugins-search-biii ---
[INFO] [INFO]
[INFO] [INFO] --- scijava-maven-plugin:2.1.0:set-rootdir (set-rootdir) @ imagej-plugins-search-biii ---
[INFO] [INFO]
[INFO] [INFO] --- jacoco-maven-plugin:0.8.7:prepare-agent (jacoco-initialize) @ imagej-plugins-search-biii ---
[INFO] [INFO] argLine set to -javaagent:C:\\Users\\eglijan\\.m2\\repository\\org\\jacoco\\org.jacoco.agent\\0.8.7\\org.jacoco.agent-0.8.7-runtime.jar=destfile=C:\\code\\imagej\\imagej-plugins-search-biii\\target\\jacoco.exec
[INFO] [INFO]
[INFO] [INFO] <<< maven-source-plugin:3.2.1:jar (attach-sources-jar) < generate-sources @ imagej-plugins-search-biii <<<
[INFO] [INFO]
[INFO] [INFO]
[INFO] [INFO] --- maven-source-plugin:3.2.1:jar (attach-sources-jar) @ imagej-plugins-search-biii ---
[INFO] [INFO] Building jar: C:\code\imagej\imagej-plugins-search-biii\target\imagej-plugins-search-biii-0.0.1-sources.jar
[INFO] [INFO]
[INFO] [INFO] --- jacoco-maven-plugin:0.8.7:report (jacoco-site) @ imagej-plugins-search-biii ---
[INFO] [INFO] Skipping JaCoCo execution due to missing execution data file.
[INFO] [INFO]
[INFO] [INFO] --- maven-javadoc-plugin:3.3.0:jar (attach-javadocs) @ imagej-plugins-search-biii ---
[INFO] [INFO] No previous run data found, generating javadoc.
[INFO] [INFO]
[INFO] Loading source file C:\code\imagej\imagej-plugins-search-biii\src\main\java\net\imagej\search\biii\BIIIeuSearcher.java...
[INFO] Loading source file C:\code\imagej\imagej-plugins-search-biii\src\main\java\net\imagej\search\biii\BIIIeuSearchResult.java...
[INFO] Loading source file C:\code\imagej\imagej-plugins-search-biii\src\main\java\net\imagej\search\biii\OpenBIIIeuInBrowserActionFactory.java...
[INFO] Loading source files for package net.imagej.search.biii...
[INFO] Constructing Javadoc information...
[INFO] Standard Doclet version 1.8.0_361
[INFO] Building tree for all the packages and classes...
[INFO] Generating C:\code\imagej\imagej-plugins-search-biii\target\apidocs\net\imagej\search\biii\BIIIeuSearcher.html...
[INFO] Generating C:\code\imagej\imagej-plugins-search-biii\target\apidocs\net\imagej\search\biii\BIIIeuSearchResult.html...
[INFO] Generating C:\code\imagej\imagej-plugins-search-biii\target\apidocs\net\imagej\search\biii\OpenBIIIeuInBrowserActionFactory.html...
[INFO] Generating C:\code\imagej\imagej-plugins-search-biii\target\apidocs\net\imagej\search\biii\package-frame.html...
[INFO] Generating C:\code\imagej\imagej-plugins-search-biii\target\apidocs\net\imagej\search\biii\package-summary.html...
[INFO] Generating C:\code\imagej\imagej-plugins-search-biii\target\apidocs\net\imagej\search\biii\package-tree.html...
[INFO] Generating C:\code\imagej\imagej-plugins-search-biii\target\apidocs\constant-values.html...
[INFO] Generating C:\code\imagej\imagej-plugins-search-biii\target\apidocs\net\imagej\search\biii\class-use\BIIIeuSearcher.html...
[INFO] Generating C:\code\imagej\imagej-plugins-search-biii\target\apidocs\net\imagej\search\biii\class-use\BIIIeuSearchResult.html...
[INFO] Generating C:\code\imagej\imagej-plugins-search-biii\target\apidocs\net\imagej\search\biii\class-use\OpenBIIIeuInBrowserActionFactory.html...
[INFO] Generating C:\code\imagej\imagej-plugins-search-biii\target\apidocs\net\imagej\search\biii\class-use\OpenBIIIeuInBrowserActionFactory.html...
[INFO] Generating C:\code\imagej\imagej-plugins-search-biii\target\apidocs\net\imagej\search\biii\class-use\BIIIeuSearchResult.html...
[INFO] Generating C:\code\imagej\imagej-plugins-search-biii\target\apidocs\net\imagej\search\biii\class-use\BIIIeuSearcher.html...
[INFO] Generating C:\code\imagej\imagej-plugins-search-biii\target\apidocs\net\imagej\search\biii\package-use.html...
[INFO] Building index for all the packages and classes...
[INFO] Generating C:\code\imagej\imagej-plugins-search-biii\target\apidocs\overview-tree.html...
[INFO] Generating C:\code\imagej\imagej-plugins-search-biii\target\apidocs\index-all.html...
[INFO] Generating C:\code\imagej\imagej-plugins-search-biii\target\apidocs\deprecated-list.html...
[INFO] Building index for all classes...
[INFO] Generating C:\code\imagej\imagej-plugins-search-biii\target\apidocs\allclasses-frame.html...
[INFO] Generating C:\code\imagej\imagej-plugins-search-biii\target\apidocs\allclasses-noframe.html...
[INFO] Generating C:\code\imagej\imagej-plugins-search-biii\target\apidocs\index.html...
[INFO] Generating C:\code\imagej\imagej-plugins-search-biii\target\apidocs\help-doc.html...
[INFO] 6 warnings
[INFO] [WARNING] Javadoc Warnings
[INFO] [WARNING] javadoc: warning - Error fetching URL: https://imagej.net/ij/developer/api/
[INFO] [WARNING] javadoc: warning - Error fetching URL: https://javadoc.scijava.org/MorphoLibJ/
[INFO] [WARNING] javadoc: warning - Error fetching URL: https://javadoc.scijava.org/Eclipse/
[INFO] [WARNING] javadoc: warning - Error fetching URL: https://javadoc.scijava.org/JGraphT/
[INFO] [WARNING] javadoc: warning - Error fetching URL: https://javadoc.scijava.org/JOML/
[INFO] [WARNING] javadoc: warning - Error fetching URL: https://javadoc.scijava.org/Javassist/
[INFO] [INFO] Building jar: C:\code\imagej\imagej-plugins-search-biii\target\imagej-plugins-search-biii-0.0.1-javadoc.jar
[INFO] [INFO]
[INFO] [INFO] --- maven-jar-plugin:3.2.0:test-jar (default) @ imagej-plugins-search-biii ---
[INFO] [INFO] Skipping packaging of the test-jar
[INFO] [INFO]
[INFO] [INFO] --- maven-failsafe-plugin:2.22.2:integration-test (default) @ imagej-plugins-search-biii ---
[INFO] [INFO] No tests to run.
[INFO] [INFO]
[INFO] [INFO] --- maven-failsafe-plugin:2.22.2:verify (default) @ imagej-plugins-search-biii ---
[INFO] [INFO] ------------------------------------------------------------------------
[INFO] [INFO] BUILD SUCCESS
[INFO] [INFO] ------------------------------------------------------------------------
[INFO] [INFO] Total time:  37.181 s
[INFO] [INFO] Finished at: 2023-01-24T13:13:49+01:00
[INFO] [INFO] ------------------------------------------------------------------------
[INFO] Checking in modified POMs...
[INFO] Executing: cmd.exe /X /C "git add -- pom.xml"
[INFO] Working directory: C:\code\imagej\imagej-plugins-search-biii
[INFO] Executing: cmd.exe /X /C "git rev-parse --show-toplevel"
[INFO] Working directory: C:\code\imagej\imagej-plugins-search-biii
[INFO] Executing: cmd.exe /X /C "git status --porcelain ."
[INFO] Working directory: C:\code\imagej\imagej-plugins-search-biii
[WARNING] Ignoring unrecognized line: ?? pom.xml.releaseBackup
[WARNING] Ignoring unrecognized line: ?? release.properties
[INFO] Executing: cmd.exe /X /C "git commit --verbose -F C:\Users\eglijan\AppData\Local\Temp\maven-scm-660226626.commit pom.xml"
[INFO] Working directory: C:\code\imagej\imagej-plugins-search-biii
[INFO] Tagging release with the label imagej-plugins-search-biii-0.0.1...
[INFO] Executing: cmd.exe /X /C "git tag -F C:\Users\eglijan\AppData\Local\Temp\maven-scm-1213442929.commit imagej-plugins-search-biii-0.0.1"
[INFO] Working directory: C:\code\imagej\imagej-plugins-search-biii
[INFO] Executing: cmd.exe /X /C "git ls-files"
[INFO] Working directory: C:\code\imagej\imagej-plugins-search-biii
[INFO] Transforming 'ImageJ Search Plugins for BIII'...
[INFO] Not removing release POMs
[INFO] Checking in modified POMs...
[INFO] Executing: cmd.exe /X /C "git add -- pom.xml"
[INFO] Working directory: C:\code\imagej\imagej-plugins-search-biii
[INFO] Executing: cmd.exe /X /C "git rev-parse --show-toplevel"
[INFO] Working directory: C:\code\imagej\imagej-plugins-search-biii
[INFO] Executing: cmd.exe /X /C "git status --porcelain ."
[INFO] Working directory: C:\code\imagej\imagej-plugins-search-biii
[WARNING] Ignoring unrecognized line: ?? pom.xml.releaseBackup
[WARNING] Ignoring unrecognized line: ?? release.properties
[INFO] Executing: cmd.exe /X /C "git commit --verbose -F C:\Users\eglijan\AppData\Local\Temp\maven-scm-260965389.commit pom.xml"
[INFO] Working directory: C:\code\imagej\imagej-plugins-search-biii
[INFO] Release preparation complete.
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  40.768 s
[INFO] Finished at: 2023-01-24T13:13:50+01:00
[INFO] ------------------------------------------------------------------------
The release preparation step failed -- look above for errors and fix them.
Use "mvn javadoc:javadoc | grep error" to check for javadoc syntax errors.

It looks like the maven release plugin finishes release preparation successfully, and the respective changes were committed locally:

commit 3dfb9947b1665ad5afa61128bae4eef3aeea30ae (HEAD -> master)
Author: Jan Eglinger <[email protected]>
Date:   Tue Jan 24 13:13:50 2023 +0100

    [maven-release-plugin] prepare for next development iteration

commit 502842b900bcc0f4b8f160ecf711a6df0397a351 (tag: imagej-plugins-search-biii-0.0.1)
Author: Jan Eglinger <[email protected]>
Date:   Tue Jan 24 13:13:49 2023 +0100

    [maven-release-plugin] prepare release imagej-plugins-search-biii-0.0.1

So it seems the script is running until here:

scijava-scripts/release-version.sh

Lines 333 to 337 in 224ea62

 $DRY_RUN mvn $BATCH_MODE release:prepare -DpushChanges=false -Dresume=false $TAG \ 

 $PROFILE $DEV_VERSION -DreleaseVersion="$VERSION" \ 

 "-Darguments=-Dgpg.skip=true ${EXTRA_ARGS# }" || 

 die 'The release preparation step failed -- look above for errors and fix them. 

 Use "mvn javadoc:javadoc | grep error" to check for javadoc syntax errors.'

... but then maven returns a non-zero exit code?!

Indeed, runnning:

 mvn --batch-mode release:prepare -DpushChanges=false -Dresume=false -Pdeploy-to-scijava -DreleaseVersion=0.0.1 -Darguments=-Dgpg.skip=true; echo "result=$?"

... results in:

[...]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  39.127 s
[INFO] Finished at: 2023-01-24T13:45:35+01:00
[INFO] ------------------------------------------------------------------------
result=1

@ctrueden any ideas?

Error with GPL_3, asks for gpl_v3

albert@local ~/workspace/jython-autocompletion (master)$ git checkout main
Switched to branch 'main'
Your branch is up to date with 'origin/main'.
albert@san ~/workspace/jython-autocompletion (main)$ ../scijava-scripts/release-version.sh 
Version? [0.0.1]: 
Detected DEFAULT version bump
From github.com:fiji/jython-autocompletion
 * branch            main       -> FETCH_HEAD
[INFO] Scanning for projects...
[INFO] 
[INFO] -----------------< org.scijava:jython-autocompletion >------------------
[INFO] Building Jython Autocompletion 0.0.1-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] 
[INFO] --- license-maven-plugin:1.20:update-project-license (default-cli) @ jython-autocompletion ---
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  1.451 s
[INFO] Finished at: 2020-12-15T15:33:03Z
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.codehaus.mojo:license-maven-plugin:1.20:update-project-license (default-cli) on project jython-autocompletion: could not init goal UpdateProjectLicenseMojo for reason : License named 'GPL_3' is unknown, use one of [cddl_v1, gpl_v3, mit, gpl_v2, gpl_v1, fdl_v1_3, epl_v2, epl_v1, epl_only_v2, epl_only_v1, agpl_v3, lgpl_v3, lgpl_v2_1, apache_v2, eupl_v1_1, bsd_3, bsd_2] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
Failed to update copyright blurbs.
You can skip the license update using the --skip-license-update flag

Release tags point to non-existing commits in github

It seems that the version tags created by release-version.sh tag a local commit that is never pushed to github (beside the downloadable zip?). Therefore the tags cannot be easily associated to any point in the github repository code history.
GitHub says:

This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.

I'd propose that either the release commit could be pushed to master or to a "release branch" explicitly or, alternatively, the Bump to next development cycle commit could be improved to Released x.y.z. Bump to next development cycle and putting the tag x.y.z. to point here.

Create a script to add missing tags to components

Some components in the SciJava ecosystem were/are released in a way that does not create a corresponding SCM tag. We need a script to check for that situation across all release versions of a given GA, and push new tags as needed.

This can easily be accomplished by extracting the Implementation-Build entry from the JAR manifest of each release artifact.

These tags are instrumental in the melting pot script's proper functioning.

melting-pot: Report which dependency versions are overridden for each component

When a melting pot build succeeds, it is nice to know, for a given component, which dependency versions were different than the release build was.

This might be nice so that e.g. authors can be notified that their component's dependencies are safe to update to newer versions. In practice this will often likely work well, but we do need to be careful not to report such things to authors with already-outdated versions of their components—we are not testing against the latest master here, after all.

/cc @axtimwalde

release-version: newline style is fragile with WSL

@imagejan wrote:

When trying release-properties.sh from WSL terminal, I get:
-bash: ../../scijava/scijava-scripts/release-version.sh: /bin/sh^M: bad interpreter: No such file or directory

Double check that this isn't an issue with newline style in the committed file.

Melting-pot: Detecting incompatible bytecode

The melting pot will currently not catch scenarios that do not involve API changes but require recompilation: for example, a change in generic parameters. This deficiency is due to the fact that we recompile each component before testing.

Some thoughts:

For each project, compile the test sources only and run with the deployed .jar
JPMS module versioning could be used as a way to indicate incompatibility, which presumably would be caught even if recompiling each project.
Rev API might have some solutions?

Consider japi-tracker integration into CI

Consider integration of japi-tracker in continuous integration to monitor the API state of all components managed on the SciJava/ImageJ maven repository. The reports could be hosted as static webpages similar to the javadoc, e.g. at api-history.scijava.org or some such.

For libraries released to Maven Central, the reports exist here:

Allow the Bump-POM-SciJava Jenkins job to bump preemptively

When, say, scijava-common is released in a new version, we usually bump scijava-common's version first, then pom-scijava's (after adjusting the scijava-common.version property).

This poses the following problem: the new pom-scijava version references the new scijava-common version in its <properties> section, but the same scijava-common version has the previous pom-scijava version as parent.

I propose to allow the Jenkins job to adjust the scijava-common.version property in pom-scijava first (i.e. before it is valid), deploy pom-scijava, increase both the scijava-common version as well as the pom-scijava parent version in scijava-common's pom.xml and then deploy scijava-common, too.

We could easily adjust the Jenkins job we use to bump the pom-scijava version already by adding another parameter that allows us to specify which artifact (or artifact's) to release. The user would have to specify the new version as well, and the Jenkins job would then have to call release-version.sh, too, making sure that the children (i.e. the projects whose parent version needs to be bumped) are handled according to the order implicitly given by the project dependencies.

@ctrueden @hinerm what do you think?

release-version: bad behavior when previous release files are present

@imagejan wrote:

When running release-version.sh on Windows (git bash), I now get the following warnings:
[WARNING] Ignoring unrecognized line: ?? pom.xml.releaseBackup
[WARNING] Ignoring unrecognized line: ?? release.properties
... and no release.properties file gets created, but the run reports success nevertheless. Also, and likely related, I observed the newly created tag is now part of the main branch.
(Sorry, I see release.properties is created but not committed)

Reference to non-existing wiki page

This page doesn't seem to exist, and I failed to find it anywhere else:

scijava-scripts/release-version.sh

Line 26 in 818b971

See also: https://github.com/scijava/pom-scijava/wiki/GPG-Signing'

Deal with github renaming the default branch from master to main

For newly created repositories, github names the default branch "main" by default now.

travisify.sh, release-version.sh, and others(?) hvae to be updated to work on both "master" and "main"

Refactor bump-POM-SciJava automation

The logic for the current automation of bump-POM-SciJava via jenkins is distributed between the jenkins job itself and the local script.

Instead, this logic should be entirely contained in shell scripts in this project.

Also, the current execution conflates two tasks:

Analyze the current pom-scijava to detect if it needs to be updated
Analyze downstream projects to see if their pom-scijava version needs to be updated

These tasks should be split into two separate scripts, the former calling the latter.

And finally, any transient information (such as e-mail addresses) should be discovered or input instead of being hard-coded.

Build Fiji using the melting pot on Travis CI

Jenkins needs a job which builds Fiji using the melting pot on a regular basis, to verify that the current runtime state of Fiji is sane.

This will entail testing the melting pot script with sc.fiji:fiji:2.0.0-SNAPSHOT extensively, ironing out any problems, creating the Jenkins job, and deciding under what conditions it will be triggered.

Workflow compatibility in scripts

We had a bit of a discussion about issues with the release-version.sh script on the sciview repo scenerygraphics/sciview#322

This is just to link it over here and conclude a bit. I'd like to point out that it isn't that I necessarily have an issue with the recent changes to the script, but since there isn't any semvar versioning it isn't necessarily obvious to someone trying to naively use the scripts that an update would introduce so much extra work just to cut a release. This hurt a bit when I was trying to cut a release quickly for our I2K workshop and all of the licensing/git restrictions/etc. made it impossible. However, in all fairness, I had wonked other things in the process.

I don't have any solutions to offer, or necessarily any observations of a specific problem. If we'd been releasing more often then this wouldn't have happened at a mission critical time. On the other hand, I don't love how much enforcement is happening by default, even if in the name of best practice.

Fix bump-pom-scijava.sh to update versions "in advance"

Moved from scijava/scijava-common#22.

The current bump-pom-scijava.sh has some limitations which we should address before pushing artifacts to Maven Central en masse.

The following scenario recently occurred:

pom-scijava 1.100 has imglib2.version of 2.0.0-beta-20 and imagej.version of 2.0.0-beta-7.5.
pom-scijava 1.101 bumped imglib2.version to 2.0.0-beta-21, an incompatible change since subsequent betas do not follow SemVer yet.
pom-scijava 1.105 further bumped imglib2.version to 2.0.0-beta-22, another incompatible change.
In particular, although we try to deprecate old functionality in a release before removing it in a subsequent release, the fact remains that between ImgLib2 versions 2.0.0-beta-20 and 2.0.0-beta-22, there are definitely API incompatibilities such as the CompositeXYProjector class migrating to a new package.
However, ImageJ version 2.0.0-beta-7.5 continues to depend on ImgLib2 2.0.0-beta-20. So consequently, pom-scijava 1.105 and above recommend incompatible library versions of ImgLib2 and ImageJ2, respectively.

Really, when a library is bumped to an incompatible (i.e., major) new version, all downstream projects of that library also need to be updated and released and bumped inside pom-scijava as well, or else the version recommendations clash.

A solution that @dscho proposed is to bump versions "preemptively"—i.e., before they have actually been cut. Then the release itself can have the new pom-scijava version as its parent, and the version properties of the release itself will be consistent.

However, ultimately, whenever a new major version of an upstream library is released, we still need to do the work of cutting a release of each downstream library as well (probably all with new major version numbers).

We should get ImgLib2 and SCIFIO out of beta as soon as possible, and start respecting SemVer conventions and avoiding major version bumps, to avoid these problems whenever possible. But when major releases are deemed necessary, it would be good to be clear on how bump-pom-scijava.sh should behave in those cases.

Releasing with release-version.sh stops halfway with Git Bash on Windows

When having unstashed/uncommited changes in the working copy of a repository and running release-version.sh, it proceeds to create a tag but then leaves you with a 'detached HEAD' state without finalizing the release.

UPDATE: this might be unrelated to the changes, it also left me halfway through the process when starting with a clean state. So this might rather be related to me using Git Bash on Windows...

Redesign bump-pom-scijava logic to be foolproof

The required steps to bump and release new versions of pom-scijava, using the Bump-POM-SciJava script and corresponding bump-pom-scijava.sh, is somewhat convoluted and highly error prone when breaking changes are involved.
Basically, we would like to move to a system where we can lean on the tools to help us decide what needs to be done (e.g. when there are breaking changes, what components need to be updated) and avoid ever needing to manually edit pom-scijava (e.g. to preemptively account for what versions of software will be released).

Thus a new overall workflow is proposed:

There will be a single bump-pom-scijava Jenkins job, with no parameters. It will perform the following operations:

If there is a pom-scijava topic branch, check it out.
Scan the complete software stack.
- If there are new versions of any components, add commits increasing their version.property, and push to the topic branch.
Run mvn install on pom-scijava. This will allow us to test each component against the new version properties without updating their pom-scijava version. NB: requires unified pom-scijava versions (which would be maintained by this script once established)
for each scijava project that we have push rights to, starting at the lowest levels of dependency:
- Checkout the latest master and attempt to build/run tests
  - If successful, continue
  - If unsuccessful, push a topic branch for that project equal to the current pom version, send notification of failure, abort bump-pom-scijava.
    - MANUAL STEP: fix the broken downstream project's topic branch to compile and test successfully against the pom-scijava topic branch. Merge to master and release the next version. Run bump-pom-scijava again.
When all projects have passed:
- Merge pom-scijava topic branch to master & delete the topic branch. Increase pom-scijava version and deploy. (optional: rebase and squish all the changes to a single commit)
- for each scijava project that we have push rights to:
  - update to the newly created pom-scijava version and push to master

This will create a simple workflow where we just run bump-pom-scijava, respond to the feedback it gives us about failing projects, perform the necessary updates, and keep running bump-pom-scijava until everything passes. There is no ambiguity about what flags need to be selected in which circumstances, and no need to ever manually edit pom-scijava, or spend time trying to figure out what components need to be updated.

@dscho @ctrueden : any thoughts on feasibility, concerns, etc?

travisify: make sure build.sh is committed as executable on Windows

When running travisify.sh on Windows (Git Bash), the committed build.sh script is not executable. Apparently, setting with chmod isn't sufficient on Windows:

scijava-scripts/travisify.sh

Line 137 in 818b971

chmod +x "$tmpFile"

In my experience, using git update-index is required:

git update-index --chmod=+x .travis/build.sh

My bash scripting skills are limited, so I was unsure where to add this line, but think it would likely belong into the update() function:

scijava-scripts/travisify.sh

Lines 41 to 63 in 818b971

 update() { 

 file=$1 

 msg=$2 

 test "$msg" || msg="Travis: update $file" 

 if [ -e "$file" ] 

 then 

 if diff -q "$file" "$tmpFile" >/dev/null 

 then 

 info "$file is already OK" 

 else 

 info "Updating $file" 

 $EXEC rm -rf "$file" 

 $EXEC mv -f "$tmpFile" "$file" 

 fi 

 else 

 info "Creating $file" 

 $EXEC mkdir -p "$(dirname "$file")" 

 $EXEC mv "$tmpFile" "$file" 

 fi 

 rm -rf "$tmpFile" 

 $EXEC git add "$file" 

 $EXEC git diff-index --quiet HEAD -- || $EXEC git commit -m "$msg" 

 }

release_version.sh currently failing bc of unavailable javadoc resources

Dear Scijava team,

I was trying to bump the version for the ilastik fiji plugin, and it fails during javadoc run because of some missing unrelated resources (omero, bio-formats). Would it be possible to add a switch to the script to skip javadoc run, or is it required to release a new version?

Cheers
Dominik

	$DRY_RUN mvn $BATCH_MODE release:prepare -DpushChanges=false -Dresume=false $TAG \
	$PROFILE $DEV_VERSION -DreleaseVersion="$VERSION" \
	"-Darguments=-Dgpg.skip=true ${EXTRA_ARGS# }" \|\|
	die 'The release preparation step failed -- look above for errors and fix them.
	Use "mvn javadoc:javadoc \| grep error" to check for javadoc syntax errors.'

	update() {
	file=$1
	msg=$2
	test "$msg" \|\| msg="Travis: update $file"
	if [ -e "$file" ]
	then
	if diff -q "$file" "$tmpFile" >/dev/null
	then
	info "$file is already OK"
	else
	info "Updating $file"
	$EXEC rm -rf "$file"
	$EXEC mv -f "$tmpFile" "$file"
	fi
	else
	info "Creating $file"
	$EXEC mkdir -p "$(dirname "$file")"
	$EXEC mv "$tmpFile" "$file"
	fi
	rm -rf "$tmpFile"
	$EXEC git add "$file"
	$EXEC git diff-index --quiet HEAD -- \|\| $EXEC git commit -m "$msg"
	}

scijava / scijava-scripts Goto Github PK

scijava-scripts's People

Contributors

Stargazers

Watchers

Forkers

scijava-scripts's Issues

Recommend Projects

Recommend Topics

Recommend Org

Jobs