giusepperizzo / neeleval

#Microposts2014 NEEL Challenge Evaluation Framework

neeleval

#Microposts2014 NEEL Challenge Evaluation Framework

Prerequisites

• Java v1.6+
• Maven v3+

Evaluate: compute F1, R, P figures

java -cp neeleval.jar neeleval.Evaluate GS TS

where:

• GS (Gold Standard) tab separated file. Each line should contain i) tweetid, entityMention1, entityURI1, ... , entityMentionN, entityURIN
• TS (Test Set) tab separated file. Each line should contain i) tweetid, entityMention1, entityURI1, ... , entityMentionN, entityURIN

outputs:

• precision, recall, and F1

EvaluateRank @x : compute F1, R, P figures

java -cp neeleval.jar neeleval.EvaluateRank GS TS x

where:

• GS (Gold Standard) tab separated file. Each line should contain i) tweetid, entityMention1, entityURI1, ... , entityMentionN, entityURIN
• TS (Test Set) tab separated file. Each line should contain i) tweetid, entityMention1, entityURI1;entityURI2, ... , entityMentionN, entityURIN;entityURIN+1
• x (cut-off of the rank) integer from 1 ... n

outputs:

• precision@x, recall@x, and F1@x

Redirect: resolve redirects

java -cp neeleval.jar neeleval.ResolveRedirect TS /path/where/to/save/TS/with/redirects

where:

• TS (Test Set) tab separated file. Each line should contain i) tweetid, entityMention1, entityURI1, ... , entityMentionN, entityURIN
• TS (Test Set) which contains all redirects (where needed)

neeleval's Issues

Tests failing

It looks like the tests are expecting precision and recall values between 0 and 1, when the Main.java actually returns percentages.

Removing "100 *" from Main.java and Evaluate.java fixes this issue.

---SOLVED--- Exception in URI with double quotes

---SOLVED---
Got this exception

java.net.URISyntaxException: Illegal character in path at index 35: http://dbpedia.org/resource/Olivia_"Bong"_Coo

at java.net.URI$Parser.fail(URI.java:2809)
at java.net.URI$Parser.checkChars(URI.java:2982)
at java.net.URI$Parser.parseHierarchical(URI.java:3066)
at java.net.URI$Parser.parse(URI.java:3014)
at java.net.URI.<init>(URI.java:578)
at neeleval.Pair.<init>(Pair.java:19)
at neeleval.Main.readSet(Main.java:33)
at neeleval.Main.main(Main.java:95)