wouterbeek / prolog_library_collection

A collection of Prolog libraries.

Prolog-Library-Collection

A collection of Prolog libraries that have proven useful in various projects. These libraries are intended to extend the functionality that is already available in the SWI-Prolog standard libraries.

Dependencies

Install SWI-Prolog.

Installation

Install this library:

swipl -g 'pack_install(prolog_library_collection)' -t halt

Use

Once installed, modules from this library are loaded as follows:

:- [library(atom_ext)].

Module overview

Module archive_ext

This module extends the standard library archive:

• archive_extension(?Extension:atom)

  Succeeds if Extension is a default file name extension for an archive filter or format, as declared in library [[media_type]].

• archive_media_type(?MediaType:media)

  Succeeds if MediaType is the Media Type of an archive filter or format.

• archive_open(+In:stream, -Archive:blob)

  Opens an archive over all supported and sensible archive filters and formats. This specifically excludes format mtree, which is a plain text format that is almost never used yet leads to many false positives in practice.

Module assoc_ext

This module extends the standard library assoc:

• merge_assoc(+New:assoc, +Old:assoc, -Merge:assoc)

  Merges two assocs into a new one. If the same key exists in New and Old, the format replaces the latter in Merge. These semantics are inspired by those of the standard library predicate merge_options/3 in library option.

• transpose_assoc(+Assoc:assoc, -Transposed:assoc)

  Turns an assoc of (key,value) pairs into one with(value,key) pairs.

Module atom_ext

This module provides additional support for working with atoms:

• atom_capitalize(+Original:atom, ?Capitalized:atom)

  Succeeds if Capitalized is a copt of Orginal where the first character is in upper case (if applicable).

• atom_ellipsis(+Original:atom, ?MaxLength:between(2,inf), ?Ellipsed:atom)

  Succeeds if Ellipsed is like Original, but has ellipsis applied in order to have MaxLength.

• atom_postfix(+Original:atom, ?PostFix:atom)

• atom_postfix(+Original:atom, ?Length:nonneg, ?PostFix:atom)

  Succeeds if Postfix is a postfix of Original consisting of Length characters.

• atom_prefix(+Original:atom, ?PostFix:atom)

• atom_prefix(+Original:atom, ?Length:nonneg, ?PostFix:atom)

  Succeeds if Prefix is a prefix of Original consisting of Length characters.

• atom_strip(+Original:atom, ?Stripped:atom)

• atom_strip(+Original:atom, +Strip:list(char), ?Stripped:atom)

  Succeeds if Stripped is a copy of Original where leading and trailing characters in Strip have been removed.

• atom_terminator(+Original:atom, +Terminator:atom, ?Terminated:atom)

  Succeeds if Terminated is a copy of Original which is ensured to end with the Terminator character.

• atom_truncate(+Original:atom, +MaxLenhgt:noneng, ?Truncated:atom)

  Like atom_prefix/3, but the Truncated atom is the Original atom in case MaxLength exceeds the Original atom length.

call_ext

meta-predicates

closure

code_ext

This module extends support for working with character-denoting numeric codes:

• put_codes(+Codes:list(code))
• put_codes(+Out:stream, +Codes:list(code))

conf_ext

This module introduces a generic way for dealing with external configuration files:

• cli_arguments(-Args:list(opt)) is det.
• conf_json(-Conf:dict) is det.

counter

csv_ext

Streamed processing of CSV files.

date_time

dcg

Definite Clause Grammars

In directory /dcg you will find a collection of Definite Clause Grammar (DCG) modules.

dcg/dcg_abnf

Advanced Bauckus-Naur Form (ABNF)

While DCGs are nice, they can be a bit verbose for expressing common repetition patterns. To make DCGs that include repetitions less verbose, this module implements variable repetition as defined in [[https://tools.ietf.org/html/rfc5234][RFC 5234: Augmented BNF for Syntax Specifications: ABNF]].

A simple example

Suppose we want to parse sentences, which are non-empty sequences of words:

sentence1([H|T]) -->
  word(H),
  sentece2(T).

sentence2([H|T]) -->
  word(H),
  sentence2(T)
sentence2([]) --> "".

When this module is loaded, the same can be written as follows:

sentence(L) -->
  +(word, L).

definition

variable repetition

Variable repetition is a metasyntactic construct which states that at least M and at most N occurrences of :Dcg_0 must be processed:

'm*n'(?M:nonneg, ?N:nonneg, :Dcg_0)//

specific repetition

Specific repetition is a metasyntactic construct which states that exactly N occurrences of Dcg_0 must be processed:

'#'(?N:nonneg, :Dcg_0)//

Specific repetition is a special case of [[variable repetition]], because #(N, Dcg_0) is the same as 'm*n'(N, N, Dcg_0).

Kleene

Kleene star is a metasyntactic construct which states that zero or more occurrences of Dcg_0 must be processed:

*(?N:nonneg, :Dcg_0)//

Kleene star is a special case of [[variable repetition]], because *(N, Dcg_0) is the same as 'm*n'(_, _, Dcg_0).

Kleene sum

Kleene sum is a metasyntactic construct which states that one or more occurrences of Dcg_0 must be processed:

+(?N:nonneg, :Dcg_0)//

Kleene sum is a special case of [[variable repetition]], because +(N, Dcg_0) is the same as 'm*n'(1, _, Dcg_0).

optional sequence

Optional sequence is a metasyntactic construct which states that Dcg_0 should either be processed once or not at all:

?(:Dcg_0)//

Optional sequence is a special case of [[variable repetition]], because ?(Dcg_0) is the same as 'm*n'(0, 1, Dcg_0).

DCG	Meaning	Name
#(?N, :Dcg_0)//	Process Dcg_0 exactly N times.	specific repetition
*(:Dcg_0)//	Process Dcg_0 0 or more times.	Kleene star
'*n'(?N, :Dcg_0)//	Process Dcg_0 at most N times.
+(:Dcg_0)//	Process Dcg_0 1 or more times.	Kleene sum
?(:Dcg_0)//	Process Dcg_0 0 or 1 times.	optional sequence
'm*'(?M, :Dcg_0)//	Process Dcg_0 at least M times.
'm*n'(?M, ?N, :Dcg_0)//	Process Dcg_0 at least M and at most N times.	variable repetition

It contains the following modules:

Type	Definition
media	A compound term of the form media(Super:atom/Sub:atom,Parameters:list(opt))
opt	A unary compound term whose predicate letter is an option name and whose argument is a corresponding option value.

dcg/dcg_ext

debug_ext

default

dict

Dictionaries.

dlist

Difference lists.

file_ext

Handling files and directories.

graph/gml

graph/graph_ext

graph/jgf

hash_ext

http/http_client2

http/http_generic

http/http_pagination

http/http_resource

http/http_server

json_ext

This module provides extended JSON support on top of the standard library http/json:

• json_load(+File:atom, -Structure:dict) is det.

• json_save(+File:atom, +Structure:dict) is det.

list_ext

math_ext

media_type

nlp/nlp_lang

os_ext

Running external processes, streaming to/from external processes.

pagination

pair_ext

pp

pure

sort_ext

stream_ext

Support for recoding, unpacking, sorting, and hasing streams.

string_ext

task

term_ext

thread_ext

uri_ext

Constructing/decomposing URIs.

xml_ext

This module allows Prolog goals to be called on a stream that encodes an XML DOM:

• call_on_xml(+In:stream, +Names:list(atom), :Goal_1) is det.

The following predicates allow the encoding of an XML file or stream to be determined:

• xml_encoding(+In:stream, -Encoding:atom) is semidet.
• xml_file_encoding(+File:atom, -Encoding:atom) is semidet.

xsd

Support for XML Schema 1.1 Part 2: Datatypes.

• xsd_date_time/3 for translating between XSD date/time representations and date/time representations as supported by [[https://github.com/wouterbeek/Prolog-Library-Collection][Prolog-Library-Collection]].

• xsd_date_time_type/1 for checking/enumerating the XSD date/time datatype IRIs.

• xsd_encode_string//0 a DCG rule for encoding strings of characters according to the restrictions of the XSD string datatype.

• xsd_numeric_type/1 enumerates XSD numeric datatype IRIs.

• xsd_strict_subtype/2 and xsd:subtype/2 allow the hierarchy of XSD datatype IRIs to be queried.

prolog_library_collection's Issues

Goodbye functor/3 (V7)

Use compound_name_arguments/3 and compound_name_arity/3?

Use run extensions for reporting on external program dependencies.

I find the run extensions to be quite helpful when starting PraSem. This could be better use while loading a module which has a dependency.

RDF graph naming

RDF graph naming poses the following difficulties:

• Sometimes we want to provide a graph name, sometimes not (generate an arbitrary name).
• Sometimes a graph with that name already exists. Sometimes we want the old graph to be overloaded; sometimes we want a slightly different graph naming (leaving the old one intact).
• Sometimes we have a file name and we want the graph to have the same name.

This is now handled in different ways for different predicates. Unify this.

Refresh Web messages page

Refresh Web messages page using JS.

Implement the CKAN API

Implement the CKAN API.

This consists of the following tasks:

• Only implement API version 3 (not versions 1 and 2).
• Implement all the functions that are specified for the (not yet released) CKAN version 2.2.
• Also implement the deprecated parameters for all functions (many CKAN sites run CKAN versions 2.0 and 2.1).
• Some functions result in a server error (HTTP status codes 5xx) when called. For this we have to use pagination mode (= requesting the data in chunks called "pages").
• Convert the JSON output to Prolog terms, and then to RDF (using a generic Prolog2RDF converter).
• Use XML Schema 2: Datatypes W3C Recommendation 1.1 for asserting booleans, integers, dates, times, dataTimes, floats, ... (Assert the rest as RDF literals.)

The following sub-APIs will be implemented:

• Action API,
• Datastore API, http://docs.ckan.org/en/latest/datastore.html#the-datastore-api
• Do not implement sub-APIs that are legacy (i.e. pre-2.2) only. E.g. do not implement the Filestore API.

Cleanup op http_goal/4

Cleanup the http status and exception clauses.

Also, think of intelligent ways to react to specific http status codes.

Examples:

• smtp send upon 403.
• retry later upon 5xx.
• retry immediately upon socket error.

Replace prolog_file_type/2 with Internet Content Types

Steps:

1. Implement RFC 2046.
2. Convert the IANA registry of MIME types to RDF, probably by reusing my previous implementation of record-jar-to-RDF.
3. Create a persistency-like module for MIME type registration and usage in PGC.
4. Replace all occurrences of prolog_file_type/2 in PGC and in all projects that depend on it by occurrences of predicates in the module written in step 3.
5. Integrate the registration of external applications, currently in run_ext, with the module written in step 3.

Remove the ugly pure CSS menu.

We need a menu that does not mess up the DOM too much.

Look at Beekeeper project for inspiration.

ABNF #rule

Implement the ABNF =|#rule|=, as defined in RFC 2616 (HTTP 1.1).

A construct # is defined, similar to *,
for defining lists of elements.
The full form is <n>#<m>element indicating
at least n and at most m elements,
each separated by one or more commas
and OPTIONAL linear white space ('LWS'//).

Motivation & example

This makes the usual form of lists very easy;
a rule such as

( *LWS element *( *LWS "," *LWS element ))

can be shown as

1#element

Null elements

Wherever this construct is used, null elements are allowed,
but do not contribute to the count of elements present.
That is (element), , (element) is permitted,
but counts as only two elements.
Therefore, where at least one element is required,
at least one non-null element MUST be present.

Default values

Default values are 0 and infinity so that #element allows any number,
including zero; 1#element requires at least one;
and 1#2element allows one or two.

Differences from RFC 4234 (ABNF)

This grammatical construct is not defined in RFC 4234 (ABNF).

Error for json_object/1 directive.

The following error occurs only when PraSem is loaded using debug.pl (not when it is loaded using run.pl):

?- use_module(test).
ERROR: /home/wbeek/PraSem/test.pl:3:
    Wrong context: json_object/1 can only be used in a directive
Warning: /home/wbeek/PraSem/test.pl:3:
    Goal (directive) failed: test:json_object(api_key(user:atom,organization:atom,service:atom,key:atom)+[type=api_key])
% test compiled into test 0.00 sec, 2 clauses
true.

variable names

I could have used variable names dcgs for some language today, issue for me to add them.

Replace AP statistics tracking

Replace module ap_stat with a version that generated process overviews based on the new RDF-assertions in AP.

Some uses of statistics tracking outside of AP will have to be accommodated in a different way.

Setup-call-cleanup

Use call_cleanup/2 for cases that end in cleaning (which may not be reached). Do I have these?

Add rdf_call_cleanup/2 to module RDF_AP. Use it in module EL_PARSE.

Logical constants as meta-DCG arguments

Ik wil inderdaad een meta-DCG maken met a//1 en b//1 als argumenten, en niet alleen a//0 en b//0. Ik wil de argumenten van a//1 resp. b//1 teruggeven in een lijst.

Merk op dat dit op zich prima werkt:

dcg_multi(ab, 3-3, Args1, `aba`, [])

Ik heb de code uit jouw email minimaal uitgebreid zodat dit werkt (ook als attachement meegestuurd).

Het enige wat ik nog wil bereiken is dat ik ab//1 niet nodig heb, want dit betekent dat ik voor disjuncties in de grammatica steeds een nieuwe DCG regel moet aanmaken (en dat lijkt me een onnodig gecompliceerde notatie).

Mijn vraag kan ik dus als volgt herformuleren: Is er een manier waarop ik voor:

dcg_multi((a ; b), 3-3, Args, `aba`, [])

hetzelfde gedrag krijg als voor:

dcg_multi(ab, 3-3, Args, `aba`, [])
ab(X) --> a(X).
ab(X) --> b(X).

Hierbij jouw voorbeeld, aangevuld voor DCGs met 1 argument:

:- module(test, []).
:- meta_predicate(dcg_multi(//,+,-,?,?)).
:- meta_predicate(dcg_multi(//,+,+,-,?,?)).

a(a) --> "a".
b(b) --> "b".
ab(X) --> a(X).
ab(X) --> b(X).

dcg_multi(DCG, Min-Max, Args) -->
  dcg_multi(DCG, 0, Min-Max, Args).

dcg_multi(_DCG, Max, _Min-Max, []) --> [].
dcg_multi(DCG, C0, Min-Max, [H|T]) -->
  call(DCG, H),
  {C1 is C0 + 1},
  dcg_multi(DCG, C1, Min-Max, T).
dcg_multi(_DCG, C, Min-_Max, []) --> {C >= Min}.

:- initialization(test).
test:-
  dcg_multi(ab, 3-3, Args1, `aba`, []), writeln(Args1),
  dcg_multi((a ; b), 3-3, Args2, `aba`, []), writeln(Args2).

Als ik vervolgens make/0 doe krijg ik bovendien de volgende warnings:

Warning: The predicates below are not defined. If these are defined
Warning: at runtime using assert/1, use :- dynamic Name/Arity.
Warning:
Warning: test:a/2, which is referenced by
Warning:     /home/wbeek/Git/PraSem/test.pl:25:13: 1-st clause of test:test/0
Warning: test:ab/2, which is referenced by
Warning:     /home/wbeek/Git/PraSem/test.pl:23:12: 1-st clause of test:test/0
Warning: test:b/2, which is referenced by
Warning:     /home/wbeek/Git/PraSem/test.pl:25:17: 1-st clause of test:test/

Accept-Language HTTP headers are processable

Process Accept-Language HTTP headers according to RFC 2616 (HTTP 1.1).

Application in WebQR, where the descriptions of model elements could be tailored towards the language preferences specified by a user.

This is very difficult to implement!, because of the following reasons:

• The ABNF contains a list-building construct that is not in RFC 4234 (ABNF), this needs to be supported in module HTTP_ABNF.
• The semantics of 0.0 is not "lowest in total order", but is "leave out of total order completely" (i.e. "unacceptable").
• It is requires that the user be able to select the language tags from within the application.
• The language tags must be matched. This is a specification of its own, and its wuite difficult!

Add tabular navigation GUI for RDF datasets

RDF datasets are internally described using VoID.

Integrate Java Maven JAR code.

Project IOTW now uses a JAR created by IOTW-OWL.
It would be nice to integrate a Java Maven project inside PGC.
The Java Maven project could generate JARs for generic taks such as OWL-materializing a collection of RDF data files.

Add unit tests for module `dcg_multi`

Module dcg_multi is quite complex and is expected to contain some bugs.

Write unit tests for the 3 main DCG rules.

RDF consolidation

The RDF modules need to be cleaned up.

1. Converting RDF files.
2. Cleaning RDF files.
3. Graph traversal, search
4. RDFS backward chaining, improve/use more of semweb here?

No tags can be used in HTML's title elements

I currently generate HTML title content like [1].

<title>Overview of RDF graph <span class="rdf-graph"><a href="/rdf_tabular?graph=swag">swag</a></span></title>

Uniform API for download-extract-RDFload series

Uniformly download Web resources to folders like [0].

[0]   DATA-SUBDIRECTORY/http/downloads.dbpedia.org/3.9/en/

The API allows one to

• download files to a URL-based folder, [3].
• also extract any compressed files, [2].
• also load any RDF files into the Semweb triples store, [1].

[1]   rdf_serial:rdf_downlaod_extract_load(+Url:url, +Options:list(nvpair))
[2]   uri_ext:download_and_extract_to_files(+Url:url, +Options:list(nvpair), -Files:ordset(atom))
[3]   download_to_file(+Url:url, -File:atom)

Update all existing file downloads, e.g. mime, uri_scheme, dbpedia_categories.

Integrate RDF_NAME and RDF_DCG

Integrate format/write/atom-based RDF generator with the new DCG grammar approach. Also use DCG_COLLECTION.

RDF consolidation

The RDF modules need to be cleaned up.

1. Converting RDF files.
2. Cleaning RDF files.
3. Graph traversal, search
4. RDFS backward chaining, improve/use more of semweb here?

Add pagination to RDF tabular pages

Add pagination to RDF tabular pages.
This obsoletes the use of list truncation for this.

Test HTML tables with RDF data

Due to the use of `dcg_nth0_call//3.

Make all CSS files available for offline usage.

Make all CSS files available for offline usage.

Minified menu does not have click event defined

The contents of http://purecss.io/js/ui.js need to be loaded in order to have the toggle behavior.

Problems with starting PGC from within PraSem

The following stalls:

swipl -s debug.pl PGC/

Fix predicates for safe directory+contents deletion

Fix predicates for safe directory+contents deletion.

Append subdirectories

The following does not work on Windows:

directory_to_subdirectories(Dir1, Subdirs1),
directory_to_subdirectories(Dir2, Subdirs2),
append(Subdirs1, Subdirs2, Subdirs3),

Drive letter should be stripped from file paths.

Add timeout error to http_goal catcher

[UNRECOGNIZED-EXCEPTION]
error(timeout_error(read,(0x7f128041c1a0)),context(read_util:read_line_to_codes/2,_G666))

Disentangle some of Doyle

Disentangle some of the Doyle predicates (relocate to tms).
Also see whether RDF labeled nodes can be created in a separate module.

Files are dropped outside of /Data subdirectory

E.g. mime.ttl appears after swipl -s debug.pl PGC/ starts.

Prolog to RDF conversion

Add conversion clauses for:

• XSD datatypes: parse using dcg_phrase, then rdf_assert_datatype/5.
  • boolean (also 0, 1, True, False, @(true), @(false))
  • dateTime
  • integer
  • float
  • decimal
• Compund term: recursive call.
• List: maplist recursive call.
• Null: no assertion.
• Oneof: meberchk/2
• Or: Try conversions from left to right. The order is significant, since False is converted to a boolean for or([boolean,atom]), but to an atom for or([atom,boolean]).

Improve VoID support

We have the following objects:

• RDF dataset
  A compound term of a atom denoting the default graph and a list of IRIs denoting named graphs.
• VoID dataset
  An IRI denoting an RDF graph that is defined using the VoID vocabulary.
• VoID file
  An atom denoting a file containing serialized VoID content.

We want to implement the following API:

• [1] loads VoID datasets whose description is serialized File into an RDF dataset.
• [2] saves RdfDataset into a File.
• [3] Translates between VoID graphs and the RDF datasets described by them.

[1]   void_load(+File:atom, -RdfDataset:compound) is det.
[2]   void_save(+RdfDataset:compound, ?File:atom) is det.
[3]   void_dataset(?VoidGraph:atom, ?RdfDataset:compound) is nondet.

RdfDataset is a compound term of the form [4].

rdf_dataset(?DefaultGraph:atom, ?NamedGraphs:list(iri))

Use Bootrap instead of Pure

Use Bootrap instead of Pure.

HTML table: text with URLs

Use DCG HTML tables for module web_message.

Phrase text with URLs by default.

Learn how to properly use command-line arguments

Use the ClioPatria codebase for inspiration.
This will fix #45

More consistent handling of numeric values in XSD support.

Right now the float arithmetic in the XSD modules results in quite long lexical expressions, e.g. 0.1000000000009 for 0.1. Is this right? We could use writeq/1 in order to write 0.1 as 0.1 instead.

`rdfs_individual/4` in module `rdfs_read` does not halt

rdfs_individual/4 in module rdfs_read does not halt.

Not all language tagged strings appear in Semweb's literal enumeration

The follow succeeds:

:- use_module(library(semweb/rdf_db)).
inconsistent_language_tagged_string(X, Y):-
  rdf(_, _, literal(lang(X,Y))),
  \+ ((
    rdf_current_literal(Z),
    Z = lang(X, Y)
  )).

Create a reproducible case.

Improve RDF conversion of CKAN

Use existing vocabularies for dstaset descriptions.

What to do with module XFN?

The XFN module needs either updating (e.g. export via GV_DCG and GV_FILE) or needs to be removed.

Create inspection tools for RDF(S) entailment

Preferably Web-based.

Show:

• Which triples are deduced.
• When materialization is completed.
• For each proposition: hos it was derived (possibly multiple answers).

This ties in which modules tms_web and rdf_web.

Support date-time formats for HTTP 1.1

RFC 2616 specifies the following date/time representations:

• Sun, 06 Nov 1994 08:49:37 GMT ; RFC 822, updated by RFC 1123
• Sunday, 06-Nov-94 08:49:37 GMT ; RFC 850, obsoleted by RFC 1036
• Sun Nov 6 08:49:37 1994 ; ANSI C's asctime() format

Implement these using DCGs with a parse tree argument.

Unify graph traversal and transitive closure predicates

Some modules:

• rdf_search
• set_theroy:transitive_closure/3
• graph_traversal
• graph_trav
• graph_closure
• shortest_path

Optimize RFC 3987

Although is seems to parse IRIs correctly, the DCG implementation for RFC 3987 is quite slow on some IRI inputs. Since this module has never been optimized for greedy rule application, there are bound to be some potential cuts that improve performance drastically.

Move image thumbnail boxes and box grids from project SWAG

Project SWAG uses image thumbnail grids and boxes. These are generic enough to be move to the PGC.

image_rows//3 and image_row//2 need to be adapted to make them more generic: giving them a list argument of file names.

Store identical resources efficiently

For N identical resources there are currently N^2 propositions.

RESTmodule

A module which defines a declaration for HTTP methods, in combination with the existing http_handler/3 declaration.

Example

:- 'REST-handler'(root(aap), get, aap).

% Automatically generated code.
:- http_handler(root(aap), aap, []).
aap(Request):-
  memberchk(method(HTTP_Method), Request),
  'REST-handler'(aap, get, Pred),
  call(Pred, Request).

% Automatically generated code
aap_options(Request):-
  findall(Method, 'REST-handler'(_, Method, aap), Methods),
  phrase(
    'Response'(Tree, version(1,1), status(200, 'OK'), [allow-Methods], []),
    Codes
  ),
  put_codes(Codes).

% Custom code starts like this.
aap_get(Request):-
  ...