oeg-upm / ttla

TTLA

This application is meant to be an automated experiment and not an application by it self to annotated numeric columns. Nonetheless, we are planning to create an application based on this approach details will be mentioned here once we start.

Install via pip

pip install ttla

Run the experiments

To download the data of T2Dv2 automatically

python data/preprocessing.py

Detection

python experiments/web_commons_v2.py detect

Labeling

1. Label (may take up to an hour, it needs to be connected to the internet)

python experiments/web_commons_v2.py label

2. Get the kinds (offline, quick)

python experiments/web_commons_v2.py addkinds
 

3. Show scores (offline, quick)

python experiments/web_commons_v2.py scores
 

Tests

Quick tests (test the algorithms, but does not include the t2d experiment)

sh run_tests.sh

run tests with the T2Dv2 experiment (may take up to an hour)

sh run_t2dv2_tests.sh

not that some tests may fail overtime as they depend on dbpedia

Coverage:

Coverage of the quick tests

sh run_cov.sh

Coverage of T2Dv2 tests

sh run_t2dv2_cov.sh

To publish

python setup.py sdist bdist_wheel
twine upload dist/*

Contribution

To contribute, please read the below to follow the same convention

Code structure

• The source code related to detection of data types (e.g. categorical, continuous, ...) is located under detect.
• while the files related to the annotation of the semantic types (e.g. height of a person) are located under label.

ttla's Issues

use easysparql class

The easysparql class uses cache. which would make it much faster.

Fix features for categorical

see function compute_features.

new idea for detecting counts

Maybe we can "re-express" counts using square roots and computer the std or something like that.

Ordinal Detection with non-numeric

See this file: 21801207_0_8144839668470123042.tar.gz

We should ignore the non-numeric values, order then and could be correctly detected

improve getTrim

Instead of fetching the numbers from the string. Remove strings that has numbers (e.g. 2a3b shouldn't be 23).