dice-group / limes

Link Discovery Framework for Metric Spaces.

LIMES - Link Discovery Framework for Metric Spaces.

Running LIMES

To bundle LIMES as a single jar file, do

mvn clean package shade:shade -Dmaven.test.skip=true

Then execute it using

java -jar limes-core/target/limes-core-${current-version}.jar

Using Docker

For running LIMES server in Docker, we expose port 8080. The image accepts the same arguments as the limes-core.jar, i.e. to run a configuration at ./my-configuration:

docker run -it --rm \
  -v $(pwd):/data \
  dicegroup/limes:latest \
    /data/my-configuration.xml

To run LIMES server:

docker run -it --rm \
  -p 8080:8080 \
  dicegroup/limes:latest \
    -s

To build and run Docker with WordNet:

docker build -f wordnet.Dockerfile . -t limes-wordnet

docker run -it --rm \
  -v $(pwd):/data \
  limes-wordnet \
    /data/my-configuration.xml

Maven

<dependencies>
    <dependency>
        <groupId>org.aksw.limes</groupId>
        <artifactId>limes-core</artifactId>
        <version>1.7.5</version>
    </dependency>
</dependencies>

<repositories>
    <repository>
        <id>maven.aksw.internal</id>
        <name>University Leipzig, AKSW Maven2 Internal Repository</name>
        <url>http://maven.aksw.org/repository/internal/</url>
    </repository>

    <repository>
        <id>maven.aksw.snapshots</id>
        <name>University Leipzig, AKSW Maven2 Snapshot Repository</name>
        <url>http://maven.aksw.org/repository/snapshots/</url>
    </repository>
</repositories>

How to cite

@article{KI_LIMES_2021,
  title={{LIMES - A Framework for Link Discovery on the Semantic Web}},
  author={Axel-Cyrille {Ngonga Ngomo} and Mohamed Ahmed Sherif and Kleanthi Georgala and Mofeed Hassan and Kevin Dreßler and Klaus Lyko and Daniel Obraczka and Tommaso Soru},
  journal={KI-K{\"u}nstliche Intelligenz, German Journal of Artificial Intelligence - Organ des Fachbereichs "Künstliche Intelligenz" der Gesellschaft für Informatik e.V.},
  year={2021},
  url = {https://papers.dice-research.org/2021/KI_LIMES/public.pdf},
  publisher={Springer}
}

More details

• Demo
• User manual
• Developer manual

limes's Issues

NoClassDefFoundError when using jar on Windows 7

After trying to load classes in the process of making a new Config the following exception is thrown:
java.lang.NoClassDefFoundError: Could not initialize class org.apache.jena.query.ARQ
at org.aksw.limes.core.gui.util.sparql.SPARQLHelper.queryExecution(SPARQLHelper.java:322)
at org.aksw.limes.core.gui.util.sparql.SPARQLHelper.querySelect(SPARQLHelper.java:377)
at org.aksw.limes.core.gui.util.sparql.SPARQLHelper.rootClassesUncached(SPARQLHelper.java:151)
at org.aksw.limes.core.gui.model.GetClassesTask.call(GetClassesTask.java:72)
at org.aksw.limes.core.gui.model.GetClassesTask.call(GetClassesTask.java:24)
at javafx.concurrent.Task$TaskCallable.call(Task.java:1423)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)

Measures and getSimilarity function

Dear all,
Helios and Dynamic planners give the opportunity to use a child LS as filter applied to the resulting mapping of the other child LS. In order to use a LS as a filter, each similarity measure implemented in the measures package SHOULD implement the public double getSimilarity(Instance a, Instance b, String property1, String property2); function.

SLF4J issue

SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/Users/tom/.m2/repository/org/slf4j/slf4j-log4j12/1.7.6/slf4j-log4j12-1.7.6.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/Users/tom/.m2/repository/org/apache/jena/jena-jdbc-driver-bundle/1.1.2/jena-jdbc-driver-bundle-1.1.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]

Error while trying to save config

When using persons11.nt and persons21.nt as endpoints, the first problem is that ID has to be specified, so this should be either obligatory when making new config or set some default value.
If ID is set, the whole GUI closes when trying to save throwing ERROR [RDFConfigurationWriter] Undefined prefix www in console

Check existence of setting in ML algorithm

boolean isImplemented(LearningSetting s);

Cannot use SPARQL 1.1 in Restriction

Restrictions are not accepted if they contain SPARQL 1.1 constructs, such as the NOT EXISTS filter.

Unwanted behaviour of setDefaultParameter() method of AWombat

AWombat setDefaultParameters() method adds the default parameters without clearing the list first, leading to an ever increasing list of parameters if the method is called more often

GUI: loading wrong filetype as config throws nullpointer exception

Just as title says - trying to load wrong filetype as a config throws NullPointer exception after "unknown filetype" dialogue box.

Steps to reproduce:

1. Start GUI
2. File -> Load config
3. Select random file (e.g. *.png, *.md)
4. See the "unknown filetype" dialogue box
5. See the NullPointer dialogue box

Needs a fix.
It also would be better to restrict user to only allow selecting files with correct extensions.

CSVMappingReader is not working correctly

Using DataSetChooser i load DBPLINKEDMDB. Since there are 2 different versions (one with tab-seperated .csv files the other with comma seperated .csv files) the behaviour is different for each.

Using the tab seperated (reference.csv, source.csv, target.csv):
InstanceURIs in the caches have the form
The URIs in the referenceMapping don't have these angle brackets. Therefore I am unable to get Instances out of the caches by using URIs. The problem seems to be line 96 in CSVMappingReader:

m.add(split[0].substring(1, split[0].length() - 1), split[1].substring(1, split[1].length() - 1), 1.0);

which works fine for comma-seperated files, but not with tab-seperated. I would propose to change it, given the typical form of the .csv files, by using:

 if(split[0].startsWith("\"")){
     m.add(split[0].substring(1, split[0].length() - 1), split[1].substring(1, split[1].length() - 1), 1.0);  
}else{
     m.add(split[0],split[1],1.0);
}

This does not fix another problem in commaseperated files though: For example the line in reference2.csv

"<http://dbpedia.org/resource/Elton_John:_Me,_Myself_&_I>","<http://data.linkedmdb.org/resource/film/78002>" 

gets split on the wrong comma. Therefore in the AMapping, which is the result of CSVMappingReader reading the file, this looks like this:

[<http://dbpedia.org/resource/Elton_John:_M -> (Myself_&_I>|1.0)]

Also the mapping contains the line

[iddbpedia -> (idlinkedmdb|1.0)]

which probably should not be there.

Include user_manual.md and developer_manual.md in new gitbook documentation

1. Subdivide user_manual.md and developer_manual.md into smaller Markdown files for maintainability
2. Link to user_manual.md and developer_manual.md in SUMMARY.md and README.md

Problem with Jena and Maven packaging

When running the following: java -jar limes-core-1.0.0-SNAPSHOT-jar-with-dependencies.jar -f rdf path/to/example.ttl I get the following error stacktrace:
http://pastebin.com/TAZYkNZB

A common solution for this is using the shade plugin in the pom.xml (see here ), but this doesn't seem to resolve this issue every time . Until now I didn't have any success with it.

log4j issue

log4j:WARN No appenders could be found for logger (org.apache.jena.riot.system.stream.JenaIOEnvironment).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.

add optional properties

add optional properties to configuration file

Batch learning not fully implemented

There is currently no way to provide a training mapping

limes.dtd is not found when using build jar

I am testing the build jar. At first I thought it was a GUI issue, but I also get it when running it in command-line. When I run java -jar git/LIMES-dev2/limes-core/target/limes-core-1.0.0-SNAPSHOT-jar-with-dependencies.jar -f xml path/to/example.xml I get the following error stacktrace: http://pastebin.com/GbC3Rfse

I guess the problem is that since the limes.dtd is part of the jar the file does not exist and cannot be accessed

Crossvalidation in Evaluator class is not working

The crossValidate function initializes algorithms without a target cache which leads to errors, since most algorithms rely on having a source AND target cache

Logging path should be configurable

I want to be able to specify where LIMES should save the .log file. At the moment, it just writes it to the same folder as .xml configuration (which is fine as default).

Possibility to use FileChooser for Endpoints

More user friendly than having to input the whole path

Eagle parameters are hard coded

Eagle parameters such as allowed metrics and operators are hard coded.

Add CHANGELOG file

We need a CHANGELOG.md file that'll describe changes in new builds/releases.
It's possible to use something like git-extras to generate it, but we need a nice commit messages to do so.

Multi-module Maven setup

Adopt multi module setup, e.g. like https://github.com/GeoKnow/Facete2

One parent pom, which declares all dependency versions, and the modules then use the artifacts versions from the parent.

Cancel buttons don't cancel background threads

The cancel buttons after pressing "learn" in machine learning don't actually cancel the process, see the continuing console output. See also EditEndpointController

Mappers: Validate threshold

For string mappers, when the getMapping function is called, then if the threshold is 0 or negative, the the program throws InvalidThresholdException and it terminates.

Please perform similar threshold checks to the other Mappers based on their functionality.

GUI: entering non-URLs into "Endpoint URL" throws

Steps to reproduce:

1. File -> New
2. Enter "asd" into "Endpoint URL"
3. See two error dialogue boxes with stack traces

Input validation required.

Tests should not have error stack traces in output

See Travis logs for examples.

Make parameter strings public in AWombat

If those strings are public I can use them as labels in the GUI and it would be easier to change if it would be necessary

GUI: entering non-numerical value in page size throws

Steps to reproduce:

1. File -> New
2. Enter text in "Page size" box
3. Hit "Next"

No errors are shown, but console displays the following:

Exception in thread "JavaFX Application Thread" java.lang.NumberFormatException: For input string: "-asd"
        at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
        at java.lang.Integer.parseInt(Integer.java:580)
        at java.lang.Integer.parseInt(Integer.java:615)

Input validation and display of the errors to user required.

ML algorithms and Execution Engine

Dear all,
if you are responsible for a ML algorithm and your algorithm requires an execution engine type and a planner type, please give the config instance as parameter to your constructor. No hardcoded engines and planner types. Check the Controller class for further information on how to invoke the execution engine and what parameters the (new) execute function needs. Ideally, the execution engine should interact with your ML class ONLY with the execute function and none other execute functions.

Possibility to close NodeContextMenu by clicking somewhere in the graph builder

This would lead to a more pleasant user experience

RDFConfigurationReader file path bug

When handing over a relative path to RDFConfigurationReader, it throws an Exception. Given an absolute path like /resources/lgd-lgd-ml.ttl it uses it as relative path (i.e., $CWD/resources/lgd-lgd-ml.ttl).
Probably just a missing +"/"+.

No check in Parser for complex LS with wrong Operator

I am doing test for the CanonicalPlanner. I wanted to check what happens if the LS has an operator that is not included in the list of operators. It seems that the LinkSpecification readSpec function assumes that the input LS is atomic because of the Parser class.

EagleTest.testSupervisedBatch occasionally fails

See Travis logs for more info, here's a partial copy of output:

testSupervisedBatch(org.aksw.limes.core.ml.EagleTest)  Time elapsed: 1.122 sec  <<< FAILURE!
java.lang.AssertionError: null
    at org.aksw.limes.core.ml.EagleTest.testSupervisedBatch(EagleTest.java:104)
Running org.aksw.limes.core.ml.DecisionTreeLearningTest
11:00:07.214 [main] INFO  org.aksw.limes.core.ml.algorithm.decisionTreeLearning.DecisionTreeLearning - Getting initial training mapping...
11:00:07.724 [main] INFO  org.aksw.limes.core.ml.algorithm.decisionTreeLearning.DecisionTreeLearning - false & true = false
11:00:07.724 [main] INFO  org.aksw.limes.core.ml.algorithm.decisionTreeLearning.DecisionTreeLearning - Training Data contains only positive/negative examples. Using Wombat
11:00:08.234 [main] INFO  org.aksw.limes.core.ml.algorithm.decisionTreeLearning.DecisionTreeLearning - true & true = true
11:00:08.265 [main] INFO  org.aksw.limes.core.ml.algorithm.decisionTreeLearning.DecisionTreeLearning - Building classifier....
11:00:08.404 [main] INFO  org.aksw.limes.core.ml.algorithm.decisionTreeLearning.DecisionTreeLearning - Parsing tree to LinkSpecification...
11:00:08.407 [main] INFO  org.aksw.limes.core.ml.algorithm.decisionTreeLearning.DecisionTreeLearning - Learned LinkSpecification: (jaccard(x.surname,y.surname)|1.00, 1.0, null, null)
digraph J48Tree {
N0 [label="positive (2.0/1.0)" shape=box style=filled ]
}

Inconsistency between ExecutionPlannerFactory and Configuration

The default executionPlan of the Configuration class is "simple" which is unknown to ExecutionPlannerFactory which has "default" instead. So one or the other has to be changed.

Define Data Sources with direct use of SPARQL

in configuration file add the ability to use user defined sparql directly for data input

Add possibility to select properties from a property if it is itself a resource

In the propertyMatchingView if the selected property is a resource make a new window pop up where it's properties can be selected and the property is added as "resource/property"

changed the type of the mapping

@mommi84 The type of a mapping is not the mapping name, see Line 90 org.aksw.limes.core.evaluation.evaluator

Restrict resizing of Windows

Set a minimum height/width to prohibit indefinite resizing

Measure Factories

@MSherif : Please remove any Type_of_Metric_Factory.java class from internal measure packages.
The execution engine recognizes only one measure factory, the MeasureFactory inside the measures package.
-We avoid duplication of code.
-We avoid to unnecessary maintenance.

Support Big Data Technologies

Can the current workflow deal with big datasets (i.e., when it's impossible to store them in-memory)?

{XML,RDF}ConfigurationReader only works correctly with paths relative to working directory

Since in the constructor both classes call

super(System.getProperty("user.dir") + "/" + fileNameOrUri);

Given an absolute path the working the directory is always taken as prefix. A possible solution could be:

super(fileNameOrUri);

But I don't know if this would break something else.
Another option could be to make another read method which takes absolute paths

Measures that have no implementation for getRuntimeApproximation

Helios creates filterInstructions considering runtime approximations of the measures included in the filter. I found a set of measures that do not implement getRuntimeApproximation. Please check measure.measures.MeasureFactory.getMeasure function. If you are responsible for one or more of the measures that have the comment "//NO getRuntimeApproximation", it would be nice if you could fix it :)

Move code to dev branch

EditPropertyMatchingView adds "null" as properties

When clicking on the empty "added source/target properties" fields "null" gets added

add km to geo-spatial distances

in the metric exper. add km/m/... to the distance instead of the threshold

Matching equal language tags

I propose a feature that only equal language tags get matched. This increases precision and reduces runtime from |N|*|M| to |N \cap M|.

Example:

dbpedia Germany                    linkedgeodata Germany
"Germany"@en                       "Germany"@en
"Deutschland"@de                  "Deutschland"@en
"Alemagne"@fr                       "?"@es

The way it is now, there are 9 comparisons, but with the new method there would only be 2. Question is what to do with no language tag, compare to others without or also to everything else, should be an option.

Idea:

<property>rdfs:label RENAME label TAGMATCH notagtoall</property>
<property>rdfs:label RENAME label TAGMATCH notagtonotag</property>

MetricFormatException when trying to add source/target properties of example config

When trying to add the source/target properties of the example config the following error is thrown: Exception in thread "JavaFX Application Thread" org.aksw.limes.core.gui.model.metric.MetricFormatException: id "x.geom:geometry/geos:asWKT" does not confirm to the regex \w+\.\w+:?\w+ at org.aksw.limes.core.gui.model.metric.Property.<init>(Property.java:30) at org.aksw.limes.core.gui.view.ToolBox.generateProperty(ToolBox.java:157) .....

NullPointerException in Parser Constructor

I am doing a test for HeliosPlanner and I am creating a plan that has a null measure expression in its filtering instruction (aka filter without condition but threshold only). Helios is trying to find get costs for this filtering instruction by finding the metric(s) in the measure expression and therefore, a function in Helios creates a Parser instance. I have fixed Helios to handle LS with filtering instructions with null measure expressions but that issue could also be raised somewhere else i.e. when someone gives an empty LS as input.

Renaming of properties in GUI

Renaming of properties does not have any effect in machine learning part of GUI, leading to errors

Long property names make graphic nodes look ugly

The text goes over the borders of the nodes, also "source" and "target" are floating somewhere outside of the node

Summary table after cross-validation

The summary table has to show the average values of the selected quantitative measures.