scala / scala-asm

A fork of https://gitlab.ow2.org/asm/asm for the Scala compiler

A fork of ASM for the Scala Compiler

This repository contains a fork of the ASM Java bytecode manipulation and analysis framework for the Scala compiler.

Structure

This is a fork (a git clone) of the official asm repository at https://gitlab.ow2.org/asm/asm.

Tags of the form ASM_X_Y* come from the upstream repo. Tags named vX.Y.Z-scala-n are created in this repo and used to release our fork under "org.scala-lang.modules" % "scala-asm".

For each ASM release there's a corresponding branch in this repo, e.g., s-9.5 for ASM 9.5. These branches start at the corresponding ASM release tag, our customizations are commits on top.

The following changes are applied:

• The package name is changed to scala.tools.asm
• Unused files are removed, including certain source files (e.g., package org.objectweb.asm.xml)
• Customizations are applied in commits marked [asm-cherry-pick]

We start a new branch for each ASM release and re-apply our changes in order to keep track what we changed. Besides, it would be non-trivial to merge in changes from upstream, as we deleted many files and moved all sources to a different directory.

Upgrading to a new version of ASM

Set up remotes

To avoid confusion, let's not call the remote origin:

• git remote rename origin upstream-github

If you haven't done so, add the upstream repository (https://gitlab.ow2.org/asm/asm) as a remote

• git remote add upstream-asm https://gitlab.ow2.org/asm/asm

Make the new version

Pull changes / tags from upstream

• git fetch upstream-asm --tags

Review the upstream changes to see if there's anything that requires attention, updates to scripts in this repo, or similar

• https://asm.ow2.io/versions.html
• git push upstream-github --tags
• https://github.com/scala/scala-asm/compare/ASM_9_4...ASM_9_5

Create a new branch s-x.y.z starting at the ASM tag for version x.y.z:

• git checkout -b s-9.5 ASM_9_5

Get the script to prepare the sources

• In the history, find the commit "Script for deleting unused files and preparing sources"
  • git log --oneline upstream-github/s-9.4 | grep 'Script for deleting'

    af285877 [asm-cherry-pick] Script for deleting unused files and preparing sources
    

• Cherry-pick
  • git cherry-pick af285877

Run the script

• ./scripts/selectAndPrepareSources

Push the branch, verify that the commits created by the script look correct

• git push --set-upstream upstream-github s-9.5

Get the build infrastructure

• In the history, find the commit "Build infrastructure" commit
  • git log --oneline upstream-github/s-9.4 | grep 'Build infra'

    06ff86de [asm-cherry-pick] Build infrastructure
    

• Cherry-pick the commit
  • git cherry-pick 06ff86de

Check that the build works correctly

• make sure you're on JDK 8 (java -version)
• sbt clean update test publishLocal
• for f in target/*.jar; do unzip -l $f; done

Cherry-pick all commits that went on top of the previous branch

• Compare the history of the previous branch to the current branch
  • git log --oneline --graph --first-parent upstream-github/s-9.4 | grep asm-cherry-pick
  • git log --oneline --graph --first-parent @ | grep asm-cherry-pick
• Cherry-pick the commits that are not yet included
  • git cherry-pick ..., repeated for each missing SHA

Rebase and clean up

• Make sure that all commits have the [asm-cherry-pick] flag
• Clean up the history
• If there are changes to the build infrastructure, squash them into the "Build infrastructure" commit

Check that the build works correctly

• sbt clean update test publishLocal
• for f in target/*.jar; do unzip -l $f; done

Push the branch to scala/scala-asm

• git push upstream-github s-9.5
• Check the build on travis: https://app.travis-ci.com/github/scala/scala-asm/builds

Create and push a tag to create a release

• git tag -s -m "scala-asm 9.5.0-scala-1" v9.5.0-scala-1 s-9.5
• git push upstream-github --tags
• Check the build on travis: https://app.travis-ci.com/github/scala/scala-asm/builds
• Check and release the staging repository on sonatype: https://oss.sonatype.org/
• Make yourself a sandwich while waiting for artifacts to appear at https://repo1.maven.org/maven2/org/scala-lang/modules/scala-asm/

scala-asm's Issues

Publish version 9.7

https://repo1.maven.org/maven2/org/scala-lang/modules/scala-asm/9.7.0-scala-1/

Performance optimization opportunities in Analyzer

Identified by @retronym scala/scala#6075 (comment)

---

https://github.com/scala/scala-asm/blob/v5.1.0-scala-2/src/main/java/scala/tools/asm/tree/analysis/Frame.java#L623-L632

• Type.getArgumentTypes(desc).length / Type.getReturnType(desc) == Type.VOID_TYPE could be directly implemented without constructing a Type isntance
• The same can be done in the Scala compiler (https://github.com/scala/scala/blob/v2.12.3/src/compiler/scala/tools/nsc/backend/jvm/analysis/InstructionStackEffect.scala#L63-L64)

---

In Type.java, a number of String to char[] conversions are performed to call into getType. If this method was duplicated to operatoe on String directly, those conversions could be avoided.

public static Type getReturnType(final String methodDescriptor) {
        char[] buf = methodDescriptor.toCharArray();
        return getType(buf, methodDescriptor.indexOf(')') + 1);
    }

Could use lastIndexOf to search from the end of the descriptor.

Rebase on ASM 6.2

$ git remote add upstream-asm https://gitlab.ow2.org/asm/asm
$ git fetch upstream-asm --tags
$ git push origin ASM_6_2

ASM_6_0...ASM_6_2

$ git log  --oneline ASM_6_0..v6.0.0-scala-1
fdb06854 (tag: v6.0.0-scala-1, origin/s-6.0) [asm-cherry-pick] Fill exception message for max String literal length
ea125bd4 [asm-cherry-pick] Fix findItemByIndex in case of hash collisions
0e5b0ab9 [asm-cherry-pick] Call interpreter.copyInstruction consistently
3210f654 [asm-cherry-pick] Multiple methods for initializing analysis values
41733544 [asm-cherry-pick] Allow setting stack values in analysis frames
75c9cfb8 [asm-cherry-pick] Clarify the doc of MethodInsnNode.owner
8f28b7f2 [asm-cherry-pick] Fix typos
f71d8867 [asm-cherry-pick] Use MethodWriter to compute maxLocals / maxStack
84e71cf2 [asm-cherry-pick] Log names on method size overflow
6665e1ff [asm-cherry-pick] Associate LabelNodes with their corresponding label
958c576a [asm-cherry-pick] Ensure instructions belong only to one list
4f7acc2e [asm-cherry-pick] asm.CustomAttribute class
145c8595 [asm-cherry-pick] Build infrastructure
665ea289 Remove trailing whitespace
937747f7 Update `@links`, `@associates`
7a11f0df Update imports
1ae83b79 Update package clauses
794b6628 Move sources to src/main/java/scala/tools/asm
273fcf82 Convert all line endings to unix
8e73bfa4 Remove unused files
000a5395 [asm-cherry-pick] Script for deleting unused files and preparing sources

$ git cherry-pick 679bfd52
[s-6.2 d324049b] [asm-cherry-pick] Script for deleting unused files and preparing sources

$ bash -x scripts/selectAndPrepareSources

$ git push --set-upstream retronym s-6.2

$ git cherry-pick 145c8595
[s-6.2 ef09c20b] [asm-cherry-pick] Build infrastructure

[success] Total time: 0 s, completed 30/05/2018 8:38:07 PM
[warn] MiMa will NOT run because no mimaPreviousVersion is provided.
[info] Compiling 94 Java sources to /Users/jz/code/scala-asm/target/classes...
[warn] bootstrap class path not set in conjunction with -source 1.5
[warn] source value 1.5 is obsolete and will be removed in a future release
[warn] -options.
[error] /Users/jz/code/scala-asm/src/main/java/scala/tools/asm/tools/ModuleInfoBndPlugin.java:40: package aQute.bnd.header does not exist
[error] import aQute.bnd.header.Attrs;
[error] /Users/jz/code/scala-asm/src/main/java/scala/tools/asm/tools/ModuleInfoBndPlugin.java:41: package aQute.bnd.header does not exist
[error] import aQute.bnd.header.Parameters;
[error] /Users/jz/code/scala-asm/src/main/java/scala/tools/asm/tools/ModuleInfoBndPlugin.java:42: package aQute.bnd.osgi does not exist
[error] import aQute.bnd.osgi.Analyzer;
[error] /Users/jz/code/scala-asm/src/main/java/scala/tools/asm/tools/ModuleInfoBndPlugin.java:43: package aQute.bnd.osgi does not exist
[error] import aQute.bnd.osgi.Constants;
[error] /Users/jz/code/scala-asm/src/main/java/scala/tools/asm/tools/ModuleInfoBndPlugin.java:44: package aQute.bnd.osgi does not exist
[error] import aQute.bnd.osgi.EmbeddedResource;
[error] /Users/jz/code/scala-asm/src/main/java/scala/tools/asm/tools/ModuleInfoBndPlugin.java:45: package aQute.bnd.osgi does not exist
[error] import aQute.bnd.osgi.Jar;
[error] /Users/jz/code/scala-asm/src/main/java/scala/tools/asm/tools/ModuleInfoBndPlugin.java:46: package aQute.bnd.service does not exist
[error] import aQute.bnd.service.AnalyzerPlugin;
[error] /Users/jz/code/scala-asm/src/main/java/scala/tools/asm/tools/ModuleInfoBndPlugin.java:48: cannot find symbol
[error]   symbol: class AnalyzerPlugin
[error] public class ModuleInfoBndPlugin implements AnalyzerPlugin {
[error] /Users/jz/code/scala-asm/src/main/java/scala/tools/asm/tools/ModuleInfoBndPlugin.java:56: cannot find symbol
[error]   symbol:   class Analyzer
[error]   location: class scala.tools.asm.tools.ModuleInfoBndPlugin
[error]   public boolean analyzeJar(Analyzer analyzer) throws Exception {
[error] /Users/jz/code/scala-asm/src/main/java/scala/tools/asm/tools/ModuleInfoBndPlugin.java:57: cannot find symbol
[error]   symbol:   variable Constants
[error]   location: class scala.tools.asm.tools.ModuleInfoBndPlugin
[error]     String moduleName = analyzer.getProperty(MODULE_NAME, analyzer.getProperty(Constants.BUNDLE_SYMBOLICNAME));
[error] /Users/jz/code/scala-asm/src/main/java/scala/tools/asm/tools/ModuleInfoBndPlugin.java:58: cannot find symbol
[error]   symbol:   variable Constants
[error]   location: class scala.tools.asm.tools.ModuleInfoBndPlugin
[error]     String moduleVersion =  analyzer.getProperty(MODULE_VERSION, analyzer.getProperty(Constants.BUNDLE_VERSION));
[error] /Users/jz/code/scala-asm/src/main/java/scala/tools/asm/tools/ModuleInfoBndPlugin.java:60: cannot find symbol
[error]   symbol:   variable Constants
[error]   location: class scala.tools.asm.tools.ModuleInfoBndPlugin
[error]     String exportPackages = analyzer.getProperty(MODULE_EXPORTS, analyzer.getProperty(Constants.EXPORT_PACKAGE));
[error] /Users/jz/code/scala-asm/src/main/java/scala/tools/asm/tools/ModuleInfoBndPlugin.java:75: cannot find symbol
[error]   symbol:   class Parameters
[error]   location: class scala.tools.asm.tools.ModuleInfoBndPlugin
[error]       Parameters requireParams = analyzer.parseHeader(requireModules);
[error] /Users/jz/code/scala-asm/src/main/java/scala/tools/asm/tools/ModuleInfoBndPlugin.java:77: cannot find symbol
[error]   symbol:   class Attrs
[error]   location: class scala.tools.asm.tools.ModuleInfoBndPlugin
[error]         Attrs attrs = requireParams.get(requireName);
[error] /Users/jz/code/scala-asm/src/main/java/scala/tools/asm/tools/ModuleInfoBndPlugin.java:86: cannot find symbol
[error]   symbol:   class Parameters
[error]   location: class scala.tools.asm.tools.ModuleInfoBndPlugin
[error]       Parameters exportParams = analyzer.parseHeader(exportPackages);
[error] /Users/jz/code/scala-asm/src/main/java/scala/tools/asm/tools/ModuleInfoBndPlugin.java:104: cannot find symbol
[error]   symbol:   class Jar
[error]   location: class scala.tools.asm.tools.ModuleInfoBndPlugin
[error]     Jar jar = analyzer.getJar();
[error] /Users/jz/code/scala-asm/src/main/java/scala/tools/asm/tools/ModuleInfoBndPlugin.java:105: cannot find symbol
[error]   symbol:   class EmbeddedResource
[error]   location: class scala.tools.asm.tools.ModuleInfoBndPlugin
[error]     EmbeddedResource moduleInfo = new EmbeddedResource(bytecode, System.currentTimeMillis());
[error] /Users/jz/code/scala-asm/src/main/java/scala/tools/asm/tools/ModuleInfoBndPlugin.java:105: cannot find symbol
[error]   symbol:   class EmbeddedResource
[error]   location: class scala.tools.asm.tools.ModuleInfoBndPlugin
[error]     EmbeddedResource moduleInfo = new EmbeddedResource(bytecode, System.currentTimeMillis());
[error] (compile:compileIncremental) javac returned nonzero exit code
[error] Total time: 1 s, completed 30/05/2018 8:38:08 PM
/code/scala-asm on s-6.2*

$ git commit -a -m "Remove unneeded parts of asm: ModuleInfoBndPlugin / Retrofitter"
[s-6.2 048c0792] Remove unneeded parts of asm: ModuleInfoBndPlugin / Retrofitter
 3 files changed, 398 deletions(-)
 delete mode 100644 src/main/java/scala/tools/asm/tools/ModuleInfoBndPlugin.java
 delete mode 100644 src/main/java/scala/tools/asm/tools/Retrofitter.java
 delete mode 100644 src/main/java/scala/tools/asm/tools/jdk1.5.0.12.txt.gz

$ git commit -a -m "Disable javadoc linting"
[s-6.2 0a56eca7] Disable javadoc linting
 1 file changed, 1 insertion(+), 1 deletion(-)

$ git cherry-pick 145c8595..fdb06854

$ git rebase -i 257c4b95 # squash `-Xdoclint:none` change into `Build infrastructure"

$ git diff head
diff --git a/versions.properties b/versions.properties
index ad86745dd7..d8dc24c899 100644
--- a/versions.properties
+++ b/versions.properties
@@ -7,5 +7,5 @@ starr.version=2.13.0-M4
 # Other usages:
 #  - scala-asm: jar content included in scala-compiler
 #  - jline: shaded with JarJar and included in scala-compiler
-scala-asm.version=6.0.0-scala-1
+scala-asm.version=6.0-22-gdbf7433
 jline.version=2.14.6

$ git co 2.13.x; sbt 'partest --pos --neg --run --terse'
...
[success] Total time: 437 s, completed 30/05/2018 9:05:35 PM

Retire this fork

Revisit each of our customizations and either find way to make this in the scala/scala repo (as a client of ASM 6.2) or push changes upstream.

Move customization to scala/scala

Changes to scala/scala that enable these reversions in scala-asm

Upstream

That leaves a smaller diff that make sense to upstream.

"Allow setting stack values in analysis frames"

eb6050e / retronym/asm#2

We should upstream this change. It would help if we could motivate it with a standalone ASM analysis that it enables

"Multiple methods for initializing analysis values"

a2791a0 / retronym/asm#3

This change appears to backwards compatible, visitor implementations that only override newValue will be called as before. So hopefully it could be considered upstream.

Once again, a motivating example will help us sell the change. Could we consider submitting AliasingFrame, NullnessFrame NullnessAnalyzer NullnessInterpreter and copyPropagation to asm-commons?

"Call interpreter.copyInstruction consistently"

a6a43f7 / retronym/asm#4

This appears to be a bugfix. Internally in asm, the copyOperation hook is only used in BasicVerifier, which doesn't deal with the DUP* instructions for which we've added extra calls.

Assorted error message improvements

retronym/asm#5
retronym/asm#6

Include the class/method name in "code too large" or "string literal too large" exceptions. Hopefully uncontroversial to upstream.

Publish version 9.4