se-sic / jdime Goto Github PK

This is related to https://travis-ci.org/xai/jdime/builds/112626677

Our test JDime93a3d27 fails with the combined strategy. It tries to merge the input file using a linebased strategy first, which produces a whitespace conflict.

Here, we would expect that our combined strategy detects this conflict and in its next step remerges the input file using a structured merge strategy, which then would be able to merge the file with no conflicts.

But instead, it just misses that there was a conflict and prints the result as provided by the linebased merge.

Affected are all branches.

I ran this command:

$JDIME_DIR/build/install/JDime/bin/JDime -r --accept-non-java --mode linebased,semistructured,structured \
--output checker-framework-fork-smillst-branch-java8inference-7b64ac8a86a828195dd8438b698bdff955e7d0dd-jdime-merged \
checker-framework-fork-smillst-branch-java8inference-9be94d8675 \
checker-framework-fork-smillst-branch-java8inference-c237f52596524124e44ca79dcb6c3a9fb4656e11 \
checker-framework-fork-smillst-branch-java8inference-93f64d9f29

The output is in the attached file output.txt.
output.txt

You can obtain the input files from http://homes.cs.washington.edu/~mernst/tmp33333/jdime-bug-report.zip (warning: big file).

Request to whitelist libglpk-java is here: travis-ci/apt-package-safelist#1330

Some stuff is outdated, we should care more about README because that's what people look at first.
As an example, @ttben mentioned that -m unstructured (as an alias for -m linebased) is still listed in README but no longer works. I bet there's more. We should also be careful about the differences of master and develop here. We (devs) live in develop and beyond (in private forks of develop), but master, and its documentation, is still important.

Hello!

I am trying to perform a 3-way merge with JDime to show it to my students (and show what Git would have done), and I can't make it work as expected.

The context is easy and is the following:

• a base that contains a Class C with only one dummy method.
• a right branch that renames, and only renames, Class C into Class D
• a left branch that add a single method to Class C (since it exists in her branch)

After building JDime, I run the following:

./JDime -m linebased -p -r ~/Desktop/test/left/ ~/Desktop/test/base/ ~/Desktop/test/right/
(by the way, the option unstructured is specified in the readme but does not exist)

I expected

• a full deletion of C and full addition of D by the right branch,
• an small addition in C by the left branch
• no conflict yielded

Instead, the merge result is:

• only the right branch has been applied. ie: the method added on the left branch is not in the merged result and the class C no longer exists.

I tried to run this command:
./JDime -m structured -p -r ~/Desktop/test/left/ ~/Desktop/test/base/ ~/Desktop/test/right/
and I have exactly the same result as before.

Am I doing something wrong here?

Thanks!

I tried to follow the instructions for running JDime after building it.

The JDime script does not appear at the location stated in README.md. See #23.

Running build/scripts/JDime yields

Error: Could not find or load main class de.fosd.jdime.Main
Caused by: java.lang.ClassNotFoundException: de.fosd.jdime.Main

I edited build/scripts/JDime by hand to change build/lib/JDime.jar to build/libs/JDime.jar.

Then, running build/scripts/JDime yields

Error: Unable to initialize main class de.fosd.jdime.Main
Caused by: java.lang.NoClassDefFoundError: org/apache/commons/cli/ParseException

I noticed that on one of my systems, a linebased merge conflict was not recognized as such.

• System: Debian GNU/Linux 9.8 (stretch)
• Java: build 1.8.0_181-8u181-b13-2~deb9u1-b13
• libgit2: 0.24.6 (Debian version: 0.25.1+really0.24.6-1) <- lol
• JDime: 7f5f127 (xai/develop)
• JNativeMerge: 83bf998 (seibt/master)
• Case study: https://github.com/xai/jdime <- ;-)
• Merge triple (left base right): 566e874 cd7f5ad b3d6e61
• File: src/de/fosd/jdime/strategy/StructuredStrategy.java

Steps to reproduce (use a real bash instead of /bin/sh because of process substitution):

jdimesrc=$(mktemp -d)
issue=$(mktemp -d)
git clone -q https://github.com/xai/jdime $jdimesrc
cd $jdimesrc
git show 566e874:src/de/fosd/jdime/strategy/StructuredStrategy.java > $issue/left.java
git show b3d6e61:src/de/fosd/jdime/strategy/StructuredStrategy.java > $issue/right.java
git show cd7f5adfa8da968998f97273531379d8f633ad14:src/de/fosd/jdime/strategy/StructuredStrategy.java > $issue/base.java
cd $issue
output=$(jdime -p -log FINEST -m linebased -s left.java base.java right.java 2>&1)
egrep '^(<<<<<<<|=======|>>>>>>>)' <(echo "$output") # there are conflicts
egrep -A1 '^General:' <(echo "$output") # output for affected systems is: Conflicts: 0

The output was this:

<<<<<<< left.java
=======
>>>>>>> right.java
General:
    Conflicts: 0

I hit that issue when I incidentally tried a combined merge using -m linebased,structured
that produced a conflict that -m structured did not.

This is also where the bug becomes a real problem: it breaks CombinedStrategy

Some debugging revealed that jdime did not consider the linebased conflict and did not bother to run the second strategy.

As a next step, I tried the same on my Fedora machine with the following relevant changes to the environment:

• System: Fedora release 29 (Twenty Nine)
• Java: build 1.8.0_201-b09
• libgit2: 0.27.8 (Fedora version: 0.27.8-1.module_f29+2764+dbb9cdda)

The rest of the setup remains unchanged, but the output is different:

<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< left.java
=====================================================
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> right.java
General:
    Conflicts: 1

The main difference here seems to be the libgit2 version:
0.24 on debian uses a conflict marker size of 7 and does not recognize the conflict.
0.27 on fedora uses a conflict marker size of huge?! and does recognize the conflict.

So, my next step was to control the marker size, which I did by patching both JNativeMerge and JDime to ensure a conflict marker size of 7 (which git itself uses, so I consider it a sane default):

• JDime: https://github.com/xai/jdime/tree/fixes/marker_size
• JNativeMerge: https://gitlab.infosun.fim.uni-passau.de/lessenic/JNativeMerge/tree/fixes/marker_size

After applying these patches, my fedora machine ran into the same issue as my Debian stretch machine.

Any ideas on what that might really be and how we can fix it?

In some cases, semistructured merge produces an additional newline after a method, which results in some failing tests: https://travis-ci.org/xai/jdime/builds/112415660

This currently affects only our feature/semistructured branch.

Hi, guys!

I would like to inform that the class Unordered Merge is looping indefinitely with the attached example, at
https://github.com/se-passau/jdime/blob/master/src/de/fosd/jdime/merge/UnorderedMerge.java#L105

This is the version I am currently using: 63ffc34 and the following command

-p -mode structured ".../left.java" "...base.java" ".../right.java"

Please, let me know if it gets fixed! Thanks in advance.

unordered merger loop.zip

Especially -d should set an appropriate log level and provide dumps of the matched ASTNodeArtifacts.

Hi all!

I have tried to install JDime on last version of MacOS X but no success so far.

I installed libgit2 (brew install libgit2), and compiled JDime from develop (./gradlew installDist). I ran into an error:

Logging configuration file JDimeLogging.properties does not exist. Falling back to defaults.
JDime [10:41:38] GRAVE: Uncaught exception.
java.lang.LinkageError: Failed to load the git2 native library.
	at de.uni_passau.fim.seibt.LibGit2.<clinit>(LibGit2.java:105)
	at de.fosd.jdime.strategy.LinebasedStrategy.<init>(LinebasedStrategy.java:59)
	at de.fosd.jdime.config.merge.MergeContext.<init>(MergeContext.java:235)
	at de.fosd.jdime.Main.run(Main.java:137)
	at de.fosd.jdime.Main.main(Main.java:113)
Caused by: java.io.IOException: Native library (darwin/libgit2.dylib) not found in resource path ([file:/Users/falleri/work/code/jdime/build/install/JDime/lib/JDime.jar, file:/Users/falleri/work/code/jdime/build/install/JDime/lib/extendj.jar, file:/Users/falleri/work/code/jdime/build/install/JDime/lib/commons-io-2.6.jar, file:/Users/falleri/work/code/jdime/build/install/JDime/lib/commons-cli-1.4.jar, file:/Users/falleri/work/code/jdime/build/install/JDime/lib/commons-math3-3.6.1.jar, file:/Users/falleri/work/code/jdime/build/install/JDime/lib/commons-codec-1.11.jar, file:/Users/falleri/work/code/jdime/build/install/JDime/lib/xstream-1.4.10.jar, file:/Users/falleri/work/code/jdime/build/install/JDime/lib/gson-2.8.2.jar, file:/Users/falleri/work/code/jdime/build/install/JDime/lib/kvconfig-1.0.jar, file:/Users/falleri/work/code/jdime/build/install/JDime/lib/JNativeMerge.jar, file:/Users/falleri/work/code/jdime/build/install/JDime/lib/xmlpull-1.1.3.1.jar, file:/Users/falleri/work/code/jdime/build/install/JDime/lib/xpp3_min-1.1.4c.jar, file:/Users/falleri/work/code/jdime/build/install/JDime/lib/jna-5.2.0.jar])
	at com.sun.jna.Native.extractFromResourcePath(Native.java:1089)
	at com.sun.jna.Native.extractFromResourcePath(Native.java:1047)
	at de.uni_passau.fim.seibt.LibGit2.<clinit>(LibGit2.java:103)
	... 4 more

Any I idea what I am doing wrong?

Hi, I am trying to run the nway mode, but everytime I run it with any code it shows a null pointer exception.
As a minimal example I created 3 classes. Class1.java contains the following code.
public class Class1{
public int method1(){
return 1;
}
}
Class2.java and Class3.java are similar but returning values "2" and "3" accordingly.

I tried to use JDime to merge them using the following command:
../jdime//build/install/JDime/bin/JDime --mode nway -f --output ClassExit.java Class1.java Class2.java Class3.java

But everytime I get an error as follows:
Logging configuration file JDimeLogging.properties does not exist. Falling back to defaults.
JDime [18:46:17] WARNING: JDime.properties can not be used as a config file as it does not exist.
JDime [18:46:17] SEVERE: Exception while merging:
JDime [18:46:17] SEVERE: A: Class1.java
JDime [18:46:17] SEVERE: B: Class2.java
JDime [18:46:17] SEVERE: C: Class3.java
JDime [18:46:17] SEVERE: java.lang.NullPointerException
JDime [18:46:17] SEVERE: Uncaught exception.
java.lang.Error: java.lang.NullPointerException
at de.fosd.jdime.strategy.NWayStrategy.merge(NWayStrategy.java:149)
at de.fosd.jdime.artifact.file.FileArtifact.merge(FileArtifact.java:647)
at de.fosd.jdime.operations.MergeOperation.apply(MergeOperation.java:166)
at de.fosd.jdime.Main.merge(Main.java:446)
at de.fosd.jdime.Main.run(Main.java:156)
at de.fosd.jdime.Main.main(Main.java:113)
Caused by: java.lang.NullPointerException
at de.fosd.jdime.merge.Merge.merge(Merge.java:73)
at de.fosd.jdime.artifact.ast.ASTNodeArtifact.merge(ASTNodeArtifact.java:471)
at de.fosd.jdime.operations.MergeOperation.apply(MergeOperation.java:166)
at de.fosd.jdime.strategy.NWayStrategy.merge(NWayStrategy.java:109)
... 5 more

I have also tried with 4 classes. Same error.
This error does not happen with Linebased or Structured modes.

Thanks in advance for the help.

The JDime script says "Licensed under the Apache License", which conflicts with README.md which says "JDime is covered by the GNU Lesser General Public License.".

I tried to run the auto-tuned mode by using a combination of line based and structured. I was performing my experiments over the curated DrJava source code made available from the first JDime evaluations. I was running the tool in the following manner - "JDime -r -m linebased,structured -f -o mergeDir LeftDIr BaseDir RightDir "

I keep getting the following error:
java.lang.RuntimeException: beaver.Scanner$Exception: illegal escape sequence "'\u"
at de.fosd.jdime.util.parser.Parser.getTokenCount(Parser.java:184)"
As a result of this error, the tool falls back to line based approach "JDime [20:05:28] SEVERE: Falling back to line based strategy. "

I thought this is a windows issue and how it uses "" for directory separators so I tried the same command in Linux and through Eclipse by using "\" but I kept getting the same error. Oddly, this does not happen when I pass either only linebased or only structured as the mode.

Attached is the stack trace

stacktrace.txt

Hi,
I don´t understand if there is a right way to use the output option.
If I run it with just a path, like this:
JDime --mode structured --output somePath/ somePath/versionLeft.java somePath/versionBase.java somePath/versionRight.java

I get this error:
Logging configuration file JDimeLogging.properties does not exist. Falling back to defaults.
JDime [18:23:05] WARNING: JDime.properties can not be used as a config file as it does not exist.
The output file or directory exists. Use -f to force overwriting.

If I add the "-f" argument as indicated in the message like this:
JDime --mode structured -f --output somePath/ somePath/versionLeft.java somePath/versionBase.java somePath/versionRight.java

I get this error:
Logging configuration file JDimeLogging.properties does not exist. Falling back to defaults.
JDime [18:30:29] WARNING: JDime.properties can not be used as a config file as it does not exist.
The output must be a file when merging files.

If I add an output file as indicated in the message like this (with or without the "-f" I get the same error):

JDime --mode structured -f --output somePath/outputFijle.java somePath/versionLeft.java somePath/versionBase.java somePath/versionRight.java

I get this error:
Logging configuration file JDimeLogging.properties does not exist. Falling back to defaults.
JDime [18:31:41] WARNING: JDime.properties can not be used as a config file as it does not exist.
JDime [18:31:41] SEVERE: Uncaught exception.
java.lang.IllegalArgumentException: File 'somePath/outputFile.java' does not exist.
at de.fosd.jdime.artifact.file.FileArtifact.(FileArtifact.java:205)
at de.fosd.jdime.artifact.file.FileArtifact.(FileArtifact.java:181)
at de.fosd.jdime.config.merge.MergeContext.configInputOutput(MergeContext.java:686)
at de.fosd.jdime.config.merge.MergeContext.configureFrom(MergeContext.java:333)
at de.fosd.jdime.Main.parseCommandLineArgs(Main.java:368)
at de.fosd.jdime.Main.run(Main.java:139)
at de.fosd.jdime.Main.main(Main.java:113)

So, what's the correct way to use the output option?
Thanks!

PS: I'm using java version "1.8.0_131"

I tried to use the following commands to install JDime:

mkdir -p ~/tmp
cd ~/tmp
rm -rf jdime JNativeMerge
git clone https://github.com/se-sic/jdime.git
cd jdime
./gradlew installDist

Under Java 8, I get the error:

FAILURE: Build failed with an exception.

* What went wrong:
org/eclipse/jgit/storage/file/FileRepositoryBuilder has been compiled by a more recent version of the Java Runtime (class file version 55.0), this version of the Java Runtime only recognizes class file versions up to 52.0

Running ./gradlew --debug installDist gives (among voluminous other output):

[DEBUG] [org.gradle.api.internal.artifacts.ivyservice.ivyresolve.DynamicVersionResolver] Attempting to resolve version for org.eclipse.jgit:org.eclipse.jgit:[4.8.0.201706111038-r,) using repositories [Gradle Central Plugin Repository]
[DEBUG] [org.gradle.cache.internal.DefaultCacheAccess] Creating new cache for metadata-2.53/module-versions, path /home/mernst/.gradle/caches/modules-2/metadata-2.53/module-versions.bin, access org.gradle.cache.internal.DefaultCacheAccess@3bfeacf
[DEBUG] [org.gradle.cache.internal.btree.BTreePersistentIndexedCache] Opening cache module-versions.bin (/home/mernst/.gradle/caches/modules-2/metadata-2.53/module-versions.bin)
[DEBUG] [org.gradle.api.internal.artifacts.ivyservice.ivyresolve.CachingModuleComponentRepository] Using cached module metadata for module 'org.eclipse.jgit:org.eclipse.jgit:6.7.0.202309050840-r' in 'Gradle Central Plugin Repository'
[DEBUG] [org.gradle.api.internal.artifacts.ivyservice.ivyresolve.DynamicVersionResolver] Using org.eclipse.jgit:org.eclipse.jgit:6.7.0.202309050840-r from Maven repository 'Gradle Central Plugin Repository'
[DEBUG] [org.gradle.api.internal.artifacts.ivyservice.ivyresolve.DynamicVersionResolver] Attempting to resolve version for org.eclipse.jgit:org.eclipse.jgit.ui:[4.8.0.201706111038-r,) using repositories [Gradle Central Plugin Repository]
[DEBUG] [org.gradle.api.internal.artifacts.ivyservice.ivyresolve.CachingModuleComponentRepository] Using cached module metadata for module 'org.eclipse.jgit:org.eclipse.jgit.ui:6.7.0.202309050840-r' in 'Gradle Central Plugin Repository'
...
[DEBUG] [org.gradle.internal.component.model.ComponentAttributeMatcher] Selected match org.eclipse.jgit:org.eclipse.jgit.ui:6.7.0.202309050840-r configuration runtime from candidates [org.eclipse.jgit:org.eclipse.jgit.ui:6.7.0.202309050840-r configuration runtime] for {}
[DEBUG] [org.gradle.internal.component.model.ComponentAttributeMatcher] Selected match org.eclipse.jgit:org.eclipse.jgit:6.7.0.202309050840-r configuration runtime from candidates [org.eclipse.jgit:org.eclipse.jgit:6.7.0.202309050840-r configuration runtime] for {}

Class file version 55.0 corresponds to Java 11, so I tried building under Java 11, but that gave a different error:

* What went wrong:
Could not determine java version from '11.0.20.1'.

Currently travis breaks for develop (see report), as native libraries are required.