buda-base / autocomplete-prototype

prototype for an autocomplete service for BDRC

Vagrant scripts for BUDA platform instanciation

The base platform is built using Vagrant and VirtualBox:

1. Install Vagrant and VirtualBox.
2. Download or git clone this repository.
3. cd into the unzipped directory or git clone
4. install VirtualBox guest additions with vagrant plugin install vagrant-vbguest
5. run vagrant up to summon a local instance

Or for an AWS EC2 instance:

1. install the vbguest plugin: vagrant plugin install vagrant-vbguest
2. and run the command: vagrant up or rename Vagrantfile.aws to Vagrantfile and run vagrant up --provider=aws

This will grind awhile installing all the dependencies of the BUDA platform.

Once the initial install has completed the command: vagrant ssh will connect to the instance where development, customization of the environment and so on can be performed as for any headless server.

Similarly, the jena-fuseki server will be listening on:

http://localhost:13180/fuseki

Lds-pdi application is accessible at :

http://localhost:13280/

(see https://github.com/buda-base/lds-pdi/blob/master/README.md for details about using this rest services)

The command: vagrant halt will shut the instance down. After halting (or suspending the instance) a further: vagrant up will simply boot the instance without further downloads, and vagrant destroy will completely remove the instance.

If running an AWS instance, after provisioning access the instance via ssh -p 15345 and delete Port 22 from /etc/ssh/sshd_config and sudo systemctl restart sshd. This will further secure the instance from attacks on port 22.

autocomplete-prototype's Issues

Sort by priority

Sort by priority

No fuzzy matching before 3 chars

If I type "c", or pretty much any char, I get this:

dbyangs 'char/
dbyangs yig
dbyangs gsal dang yig rigs/
dbyangs 'char ba'i dus sbyor gyi 'bras bu nye bar mkho ba/
dbyangs gsal gyi mgur ma/
陪你这一生
dbyangs gsal gyi brda chad/
dbyangs yig

Start fuzzy match when user typed the third character.

Autosuggest in Vespa

Create autosuggest in Vespa in order to combine prefix search with fuzzy match.

Improvements over the current version

• match inside titles, not only beginning (this will fix several know UX cases)
• match any length, not only 20 chars
• allow scoped suggestions, like texts of a collection or works by or about a person (this would work in ES as well. Crucial in the new prototyped UX)

FYI @eroux

Prioritise non-fuzzy matches

When I type "chod", I get

chos grags ye shes/
chos rgyal khri srong lde btsan/
chos sku/
chos rgyal dri med kun ldan/
chos grags/
chos sku'i ye shes/
chos grags bzang po/
chos grags, rdor 'bum/
chos sku sangs rgyas yar byon/
chos/

...although there would be
chod nyid nam mkha'i klong mdzod las/_bka' srung gnod sbyin btsan rgod

The main principle would be to suggest non-fuzzy matches first. We can later see if we want to balance this a bit with popularity, but not yet.

Query time unicode conversion

It would be good to start testing autocomplete with unicode Tibetan.

Adjust Levenshtein distance according to string length

The matches are too fuzzy with short strings.

For example, the suggestions for the random string "chod " are

chod nyid nam mkha'i klong mdzod las/_bka' srung gnod sbyin btsan rgod
chen po'i bskang gsol 'dod don lhun grub/

The second suggestion should not appear, because it is at edit distance 2.

I think the edit distances should be something like this, but this has to be tested carefully when the system is more mature.
Query length: 0-2 -> edit distance 0
Query length 3-8 -> edit distance 1
Query length 9+ -> edit distance 2