GithubHelp home page GithubHelp logo

rdfio / rdfio-vagrantbox Goto Github PK

View Code? Open in Web Editor NEW
1.0 1.0 2.0 72 KB

An automated Virtual Machine setup (Vagrant box) for trying out the RDFIO extension for Semantic MediaWiki.

License: MIT License

Shell 66.09% Vim Script 33.91%

rdfio-vagrantbox's Introduction

RDFIO Extension for Semantic MediaWiki

Build Status Test Coverage Code Climate Rating Code Climate Issues Codacy Grade Latest Stable Version Licence

Screenshot of the SPARQL Endpoint shipped with RDFIO

Updates

Sep 4, 2017: Our paper on RDFIO was just published! If you use RDFIO in scientific work, please cite:
Lampa S, Willighagen E, Kohonen P, King A, Vrandečić D, Grafström R, Spjuth O
RDFIO: extending Semantic MediaWiki for interoperable biomedical data management
Journal of Biomedical Semantics. 8:35 (2017). DOI: 10.1186/s13326-017-0136-y.

Introduction

This extension extends the RDF import and export functionality in Semantic MediaWiki by providing import of arbitrary RDF triples (not only OWL ontologies, as before (see about Ontology import, and a SPARQL endpoint that allows write operations.

Technically, RDFIO implements the PHP/MySQL based triple store (and its accompanying SPARQL Endpoint) provided by the ARC2 library. For updating wiki pages with new triples on import/sparql update, the WOM extension is used.

The RDF import stores the original URI of all imported RDF entities (in a special property), which can later be used by the SPARQL endpoint, instead of SMW's internal URIs, which thus allows to expose the imported RDF data "in its original formats", with its original URIs. This allows to use SMW as a collaborative RDF editor, in workflows together with other semantic tools, from which it is then possible to "export, collaboratively edit, and import again", to/from SMW.

This extensions was initially developed as part of a Google Summer of Code 2010 project, and further extended as part of a FOSS OPW 2014 project.

Installation

Easiest: Use the ready-made Virtual Machine

The absolute easiest way to try out RDFIO is to import the Ready-made Virtual Machine with RDFIO 3.0.2 (with MW 1.29 and SMW 2.5) into VirtualBox or VMWare, and just start browsing the local wiki installation.

Steps:

  1. Download the .ova file from doi.org/10.6084/m9.figshare.5383966.v1
  2. In VirtualBox (should be similar in VMWare), select "File > Import appliance"
  3. Click the folder icon
  4. Locate the .ova file you downloaded
  5. Click "Next", "Agree" to the license, and finally "Import", to start the import
  6. Start the virtual machine
  7. Click log in (No password required)
  8. Click the icon on the desktop
  9. You will now see a local wiki installation with an RDFIO enabled wiki, in a browser!
  10. Enjoy!

Easy: Vagrant box

Another quite easy way, is to use the RDFIO Vagrant box, which will automatically set up MediaWiki, SemanticMediaWiki and RDFIO in a virtual machine in under 20 minutes.

Medium-hard: Install semi-manually using composer

Install dependencies

$smwgShowFactbox = SMW_FACTBOX_NONEMPTY;

Installation steps

Assuming you have followed the steps above to install the dependencies for RDFIO:

  1. Install RDFIO by executing the following commands in a terminal:

    cd <wiki_folder>
    composer require rdfio/rdfio --update-no-dev
  2. Log in to your wiki as a super user

  3. Browse to http://[your-domain]/wiki/Special:RDFIOAdmin

  4. Click the "Setup" button to set up ARC2 database tables.

  5. If you already have semantic annotations in your wiki, you need to go to the article "Special:SMWAdmin" in your wiki, and click "Start updating data", and let it complete, in order for the data to be available in the SPARQL endpoint.

Optional but recommended steps

  • Edit the MediaWiki:Sidebar page and add the following wiki snippet, as an extra menu (I use to place it before just the "* SEARCH" line), which will give you links to the main functionality with RDFIO from the main links in the left sidebar on the wiki:

    * Semantic Tools
    ** Special:RDFIOAdmin|RDFIO Admin
    ** Special:RDFImport|RDF Import
    ** Special:SPARQLEndpoint|SPARQL Endpoint
    ** Special:SPARQLImport|SPARQL Import
    
  • Create the article "MediaWiki:Smw_uri_blacklist" and make sure it is empty (you might need to add some nonsense content like {{{<!--empty-->}}}).

Test that it works

  • Access the SPARQL endpoint at http://[url-to-your-wiki]/Special:SPARQLEndpoint
  • Access the RDF Import page at http://[url-to-your-wiki]/Special:RDFImport
  • Access the SPARQL Import page at http://[url-to-your-wiki]/Special:SPARQLImport
  • Optionally, if you want to really see that it works, try adding some semantic data to wiki pages, and then check the database (using phpMyAdmin e.g.) to see if you get some triples in the table named arc2store_triple.

Additional configuration

These are some configuration options that you might want to adjust to your specific use case. That goes into your LocalSettings.php file. Find below a template with the default options, which you can start from, add to your LocalSettings.php file and modify to your liking:

# ---------------------------------------------------------------
#  RDFIO Configuration
# ---------------------------------------------------------------
# An associative array with base uris as keys and corresponding 
# prefixes as the items. Example:
# array( 
#       "http://example.org/someOntology#" => "ont1",
#       "http://example.org/anotherOntology#" => "ont2"
#      );
# $rdfiogBaseURIs = array();
# ---------------------------------------------------------------
# Query by /Output Equivalent URIs SPARQL Endpoint 
# (overrides settings in HTML Form)
# 
# $rdfiogQueryByEquivURI = false;
# $rdfiogOutputEquivURIs = false;
#
# $rdfiogTitleProperties = array(
#  'http://semantic-mediawiki.org/swivt/1.0#page',
#  'http://www.w3.org/2000/01/rdf-schema#label',
#  'http://purl.org/dc/elements/1.1/title',
#  'http://www.w3.org/2004/02/skos/core#preferredLabel',
#  'http://xmlns.com/foaf/0.1/name',
#  'http://www.nmrshiftdb.org/onto#spectrumId'
# );
# ---------------------------------------------------------------
# Allow edit operations via SPARQL from remote services
#
# $rdfiogAllowRemoteEdit = false;
# ---------------------------------------------------------------

Dependencies

Known limitations

Bugs, new feature request and contact information

Please reports bugs and feature requests in the issue tracker here on Github.

Links

Related work

rdfio-vagrantbox's People

Contributors

samuell avatar

Stargazers

 avatar

Watchers

 avatar  avatar  avatar

Forkers

samuell atcares

rdfio-vagrantbox's Issues

provisioning fails during python setup

Adding the two steps to the pre_tasks: section solved the problem.

pre_tasks:
- name: 'prep update 1'
raw: sudo apt-get update
- name: 'prep update 2'
raw: sudo apt-get -y upgrade

Semantic-Media-Wiki setup fails ....

during upgrade step:

TASK [semantic_mediawiki : Update MediaWiki database] **************************

fatal: [virtualbox]: FAILED! => {"changed": true, "cmd": ["php", "/var/www/html/w/maintenance/update.php"], "delta": "0:00:05.628181", "end": "2018-01-10 08:22:40.714518", "failed": true, "msg": "non-zero return code", "rc": 1, "start": "2018-01-10 08:22:35.086337", "stderr": "Set $wgShowExceptionDetails = true; in LocalSettings.php to show detailed debugging information.", "stderr_lines": ["Set $wgShowExceptionDetails = true; in LocalSettings.php to show detailed debugging information."], "stdout": "MediaWiki 1.29.0 Updater\n\nYour composer.lock file is up to date with current dependencies!\nGoing to run database updates for smw\nDepending on the size of your database this may take a while!\nAbort with control-c in the next five seconds (skip this countdown with --quick) ... 5\b4\b3\b2\b1\b0\nTurning off Content Handler DB fields for this part of upgrade.\n...ipblocks table does not exist, skipping new field patch.\n...ipblocks table does not exist, skipping new field patch.\n...already have interwiki table", "stdout_lines": ["MediaWiki 1.29.0 Updater", "", "Your composer.lock file is up to date with current dependencies!", "Going to run database updates for smw", "Depending on the size of your database this may take a while!", "Abort with control-c in the next five seconds (skip this countdown with --quick) ... 5\b4\b3\b2\b1\b0", "Turning off Content Handler DB fields for this part of upgrade.", "...ipblocks table does not exist, skipping new field patch.", "...ipblocks table does not exist, skipping new field patch.", "...already have interwiki table"]}

VM, rdfio302_on_smw254_mw129.ova setup issue

refs https://sourceforge.net/p/semediawiki/mailman/message/36028997/

First of all, it is delight to have such simple setup by just running the VM without having to deal with all the vagrant, ansible setup.

The following issues appeared with the https://doi.org/10.6084/m9.figshare.5383966.v1 file:

VM description

The VM description says "..with MediaWiki 1.9, ..." which should state 1.29 instead.

Display resolution

I need to install sudo apt-get install virtualbox-guest-utils virtualbox-guest-x11 virtualbox-guest-dkms to allow for a better display resolution

LocalSettings.php

  • Link to the LocalSettings.php on the desktop
  • Add $wgAllowDisplayTitle = true; $wgRestrictDisplayTitle = false; to use the DISPLAYTITLE
  • $smwgOWLFullExport is not required (doesn't exists since 1.6)

Outdated cURL extension

I had to manually update cURL sudo apt-get install php-curl as tests complained with "Use of undefined constant CURLE_GOT_NOTHING - assumed 'CURLE_GOT_NOTHING'"

php.ini

Some reports of Maximum execution time of 60 seconds exceeded hence required to change the php.ini - max_execution_time.

XDebug

XDebug is enabled by default which kills performance and it should be disabled using "sudo phpdismod -s cli xdebug".

Accessing the Virtualbox from the outside

Accessing your Virtualbox Guest from your Host OS (https://2buntu.com/articles/1513/accessing-your-virtualbox-guest-from-your-host-os/)

Running SMW standard tests

After resolving all above issues, I was finally able to the run composer phpunit (or composer integration for those integration tests) and the encounter the following issue:

In /var/www/html/w/extensions/Rdfio/stores/SMW_ARC2Store.php:110 there is no such method as SMWWikiPageValue::setValues not in 2.5.* and not in 1.8* (for comparison see https://github.com/SemanticMediaWiki/SemanticMediaWiki/blob/1.8.0.5/includes/datavalues/SMW_DV_WikiPage.php).

ubuntu@ubuntu-xenial:/var/www/html/w/extensions/SemanticMediaWiki$ composer integration -- --stop-on-error
> composer phpunit -- --testsuite=semantic-mediawiki-integration '--stop-on-error'
> php ../../tests/phpunit/phpunit.php -c phpunit.xml.dist '--testsuite=semantic-mediawiki-integration' '--stop-on-error'
Using PHP 7.0.22-0ubuntu0.16.04.1

Semantic MediaWiki: 2.5.4 (SMWARC2Store, mysql)
MediaWiki:          1.29.0 (MediaWiki vendor autoloader)
Site language:      en

Execution time:     2017-09-07 06:43
Debug logs:         Disabled
Xdebug:             Disabled (or not installed)

PHPUnit 4.8.35 by Sebastian Bergmann and contributors.

Runtime:	PHP 7.0.22-0ubuntu0.16.04.1
Configuration:	/var/www/html/w/extensions/SemanticMediaWiki/phpunit.xml.dist

............................................................E
SMW\Tests\Integration\JSONScript\JsonTestCaseScriptRunnerTest::executeTestCases executeTestCases with data set "f-0303.json" ran for 10.069 seconds

SMW\Tests\Integration\JSONScript\JsonTestCaseScriptRunnerTest::executeTestCases executeTestCases with data set "p-0207.json" ran for 43.698 seconds


Time: 3.64 minutes, Memory: 76.00MB

There was 1 error:

1) SMW\Tests\Integration\JSONScript\JsonTestCaseScriptRunnerTest::executeTestCases with data set "p-0405.json" ('/var/www/html/w/extensions/Se...5.json')
Error: Call to undefined method SMWWikiPageValue::setValues()

/var/www/html/w/extensions/Rdfio/stores/SMW_ARC2Store.php:110
/var/www/html/w/extensions/SemanticMediaWiki/includes/StoreUpdater.php:276
/var/www/html/w/extensions/SemanticMediaWiki/includes/StoreUpdater.php:247
/var/www/html/w/extensions/SemanticMediaWiki/includes/StoreUpdater.php:212
/var/www/html/w/extensions/SemanticMediaWiki/includes/StoreUpdater.php:142
/var/www/html/w/extensions/SemanticMediaWiki/includes/StoreUpdater.php:96
/var/www/html/w/extensions/SemanticMediaWiki/includes/ParserData.php:406
/var/www/html/w/extensions/SemanticMediaWiki/src/Updater/DeferredCallableUpdate.php:236
/var/www/html/w/extensions/SemanticMediaWiki/src/Updater/TransactionalDeferredCallableUpdate.php:134
/var/www/html/w/extensions/SemanticMediaWiki/src/Updater/DeferredCallableUpdate.php:265
/var/www/html/w/extensions/SemanticMediaWiki/includes/ParserData.php:422
/var/www/html/w/extensions/SemanticMediaWiki/src/MediaWiki/Hooks/LinksUpdateConstructed.php:94
/var/www/html/w/extensions/SemanticMediaWiki/src/MediaWiki/Hooks/HookRegistry.php:359
/var/www/html/w/includes/Hooks.php:186
/var/www/html/w/includes/deferred/LinksUpdate.php:160
/var/www/html/w/includes/content/AbstractContent.php:234
/var/www/html/w/includes/page/WikiPage.php:2168
/var/www/html/w/includes/page/WikiPage.php:1789
/var/www/html/w/includes/libs/rdbms/database/Database.php:2693
/var/www/html/w/includes/deferred/AtomicSectionUpdate.php:35
/var/www/html/w/includes/deferred/DeferredUpdates.php:257
/var/www/html/w/includes/deferred/DeferredUpdates.php:210
/var/www/html/w/includes/deferred/DeferredUpdates.php:127
/var/www/html/w/includes/deferred/DeferredUpdates.php:289
/var/www/html/w/includes/deferred/DeferredUpdates.php:97
/var/www/html/w/includes/page/WikiPage.php:1801
/var/www/html/w/includes/page/WikiPage.php:1619
/var/www/html/w/extensions/SemanticMediaWiki/tests/phpunit/Utils/PageCreator.php:92
/var/www/html/w/extensions/SemanticMediaWiki/tests/phpunit/Utils/PageCreator.php:71
/var/www/html/w/extensions/SemanticMediaWiki/tests/phpunit/JsonTestCaseContentHandler.php:147
/var/www/html/w/extensions/SemanticMediaWiki/tests/phpunit/JsonTestCaseContentHandler.php:112
/var/www/html/w/extensions/SemanticMediaWiki/tests/phpunit/JsonTestCaseScriptRunner.php:239
/var/www/html/w/extensions/SemanticMediaWiki/tests/phpunit/Integration/JSONScript/JsonTestCaseScriptRunnerTest.php:216
/var/www/html/w/extensions/SemanticMediaWiki/tests/phpunit/Integration/JSONScript/JsonTestCaseScriptRunnerTest.php:154
/var/www/html/w/extensions/SemanticMediaWiki/tests/phpunit/JsonTestCaseScriptRunner.php:160
/var/www/html/w/extensions/SemanticMediaWiki/tests/phpunit/MwDBaseUnitTestCase.php:131
/var/www/html/w/maintenance/doMaintenance.php:111

We should be able to run composer phpunit from the SMW root directory without problems with or without enabled SMWARC2Store.

Test-import fails

After bringing up the vagrant box the data import with rdfio fails with following error:

`[6b7e64af6040433a16b7bebe] /w/index.php?title=Special:RDFImport Error from line 727 of /vagrant/w/vendor/semsol/arc2/store/ARC2_Store.php: Call to a member function escape() on null

Backtrace:

#0 /vagrant/w/vendor/semsol/arc2/store/ARC2_StoreQueryHandler.php(31): ARC2_Store->getTermID(string, string)
#1 /vagrant/w/vendor/semsol/arc2/store/ARC2_StoreSelectQueryHandler.php(1023): ARC2_StoreQueryHandler->getTermID(string, string)
#2 /vagrant/w/vendor/semsol/arc2/store/ARC2_StoreSelectQueryHandler.php(996): ARC2_StoreSelectQueryHandler->getTriplePatternSQL(array, string)
#3 /vagrant/w/vendor/semsol/arc2/store/ARC2_StoreSelectQueryHandler.php(1005): ARC2_StoreSelectQueryHandler->getPatternSQL(array, string)
#4 /vagrant/w/vendor/semsol/arc2/store/ARC2_StoreSelectQueryHandler.php(996): ARC2_StoreSelectQueryHandler->getDefaultPatternSQL(array, string)
#5 /vagrant/w/vendor/semsol/arc2/store/ARC2_StoreSelectQueryHandler.php(1005): ARC2_StoreSelectQueryHandler->getPatternSQL(array, string)
#6 /vagrant/w/vendor/semsol/arc2/store/ARC2_StoreSelectQueryHandler.php(996): ARC2_StoreSelectQueryHandler->getDefaultPatternSQL(array, string)
#7 /vagrant/w/vendor/semsol/arc2/store/ARC2_StoreSelectQueryHandler.php(924): ARC2_StoreSelectQueryHandler->getPatternSQL(array, string)
#8 /vagrant/w/vendor/semsol/arc2/store/ARC2_StoreSelectQueryHandler.php(475): ARC2_StoreSelectQueryHandler->getWHERESQL()
#9 /vagrant/w/vendor/semsol/arc2/store/ARC2_StoreSelectQueryHandler.php(68): ARC2_StoreSelectQueryHandler->getQuerySQL()
#10 /vagrant/w/vendor/semsol/arc2/store/ARC2_StoreSelectQueryHandler.php(36): ARC2_StoreSelectQueryHandler->getSQL()
#11 /vagrant/w/vendor/semsol/arc2/store/ARC2_Store.php(646): ARC2_StoreSelectQueryHandler->runQuery(array, integer)
#12 /vagrant/w/vendor/semsol/arc2/store/ARC2_Store.php(596): ARC2_Store->runQuery(array, string, integer, string)
#13 /vagrant/w/extensions/Rdfio/classes/RDFIO_ARC2StoreWrapper.php(138): ARC2_Store->query(string)
#14 /vagrant/w/extensions/Rdfio/classes/RDFIO_ARC2StoreWrapper.php(162): RDFIOARC2StoreWrapper->getURIForEquivURI(string, boolean)
#15 /vagrant/w/extensions/Rdfio/classes/parsers/RDFIO_URIToWikiTitleConverter.php(68): RDFIOARC2StoreWrapper->getWikiTitleByEquivalentURI(string)
#16 /vagrant/w/extensions/Rdfio/classes/parsers/RDFIO_URIToWikiTitleConverter.php(51): RDFIOURIToTitleConverter->getExistingTitleForURI(string)
#17 /vagrant/w/extensions/Rdfio/classes/parsers/RDFIO_ARC2ToWikiConverter.php(31): RDFIOURIToTitleConverter->convert(string)
#18 /vagrant/w/extensions/Rdfio/classes/RDFIO_RDFImporter.php(77): RDFIOARC2ToWikiConverter->convert(array, array, array)
#19 /vagrant/w/extensions/Rdfio/classes/RDFIO_RDFImporter.php(25): RDFIORDFImporter->importFromArc2Data(array, array, array)
#20 /vagrant/w/extensions/Rdfio/specials/SpecialRDFImport.php(83): RDFIORDFImporter->importRdfXml(string)
#21 /vagrant/w/extensions/Rdfio/specials/SpecialRDFImport.php(35): RDFImport->importData(RDFIORequestData)
#22 /vagrant/w/includes/specialpage/SpecialPage.php(522): RDFImport->execute(NULL)
#23 /vagrant/w/includes/specialpage/SpecialPageFactory.php(578): SpecialPage->run(NULL)
#24 /vagrant/w/includes/MediaWiki.php(287): SpecialPageFactory::executePath(Title, RequestContext)
#25 /vagrant/w/includes/MediaWiki.php(862): MediaWiki->performRequest()
#26 /vagrant/w/includes/MediaWiki.php(523): MediaWiki->main()
#27 /vagrant/w/index.php(43): MediaWiki->run()
#28 {main}`

Do you have an idea what might cause this problem?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.