yuz998 / parser-triples

The web contains vast repositories of unstructured text. We investigate the opportunity for building a knowledge graph from these text sources. We generate a set of triples which can be used in knowledge gathering and integration. We define the architecture of a language compiler for processing subject-predicate-object triples using the OpenNLP parser. We implement a depth-first search traversal on the POS tagged syntactic tree appending predicate and object information. A parser enables higher precision and higher recall extractions of syntactic relationships across conjunction boundaries. We are able to extract 2-2.5 times the correct extractions of ReVerb. The extractions are used in a variety of semantic web applications and question answering. We verify extraction of 50,000 triples on the ClueWeb dataset.

parser-triples

Shaun D’Souza. Parser extraction of triples in unstructured text. arXiv preprint arXiv:1811.05768, 2018. url: https://arxiv.org/abs/1811.05768

• Steps to compile and run jar

• System requirements

  • Install Java JDK
  • Add jdk\bin to Windows PATH

• Compile source files to generate opennlp-parser-svo-new.jar

javac -cp opennlp-tools-1.6.0.jar ParseKnowledgeNpVisited.java ParseKnowledgeNpVisitedMap.java -d .
jar cvf opennlp-parser-svo-new.jar ./opennlp

• Unstructured text can be parsed as per the Apache OpenNLP developer guide [Chapter 8. Parser] (https://opennlp.apache.org/docs/1.6.0/manual/opennlp.html#tools.parser)
  • Download en-parser-chunking.bin
  • A sample parsed file is uploaded in ie-parser.txt

java -cp opennlp-tools-1.6.0.jar opennlp.tools.cmdline.CLI Parser en-parser-chunking.bin < input.txt > output-parser.txt

• SVO Triples are extracted using the command
  • Unix shell uses colon (:) as the path separator

java -cp opennlp-parser-svo-new.jar;opennlp-tools-1.6.0.jar opennlp.tools.parser.ParseKnowledgeNpVisitedMap -fun -pos head_rules < ie-parser.txt

• Expected output

Google is located in Mountain view
0       "Google"        "is located"    "in Mountain view"
Mountain view is in California
1       "Mountain view" "is"    "in California"
Google will acquire YouTube , announced the New York Times .
2       "Google"        "will acquire"  "YouTube"
2       "Google"        "announced"     "the New York Times"
Google and Apple are headquartered in California .
3       "Google and Apple"      "are headquartered"     "in California"

• Alternate code command

java -cp opennlp-parser-svo-new.jar;opennlp-tools-1.6.0.jar opennlp.tools.parser.ParseKnowledgeNpVisited -fun -pos head_rules < ie-parser.txt