Comments (8)
Good catch, thanks! Considering it's a typo, I've changed it directly on the master as a hotfix.
from core.
Sweet...thanx
from core.
You're welcome, can I ask what you're using it for? Always nice to know the user base :).
from core.
Was mainly doing a test of the application to see if it could be used here in Norway. We have some large datasets. To get tings "speedy" it is not an alternative to read the CSV file on the fly, so i wanted to test speed when using a Mongodb as backend for the dataset.
from core.
I see, the solution we use for one of our clients is that we ingest it in Elasticsearch automatically.
We use the package tdt/input to configure a job that reads data from the CSV file, ingests it into an index of your choosing (pick a dedicated index for it though) automatically, just with configuration. Then when that's done, you can make an Elasticsearch datasource in order to read the data from Elasticsearch and have a default wildcard parameter to search in that data out of the box.
As far as MongoDB goes, I dont think we support exporting data into MongoDB directly, maybe there are other tools for that, or you can do a pull request to the tdt/input package ;)
from core.
I see...the Elasticsearch seems to be a better choise then :-) Does that mean that the user can search Elasticsearch data through the "datatank" interface?
In regards to Mongo, i just did an commandline import of the CSV file directly into the database.
from core.
That is correct, the datatank interface will return json, but you're able to pass a "q" request parameter with a string which will be passed to the Elasticsearch controller as a query.
Be advised though, the standard queue'ing system is "sync", which means that the job is immediately executed and depending on your file can take while. This also means that adding several of those jobs after each other, one will have to wait until the first is done. For queueing, beanstalkd is also supported out of the box by laravel 4. (the framework this was built in)
Mongo-wise: neat! Whatever works :)
from core.
Cool...i will se if i can test this with an Elastic database :-)
from core.
Related Issues (20)
- Incorrect pagination in SPARQL CONSTRUCT dataset HOT 5
- caching policy HOT 1
- request throttling any kind of requests HOT 1
- add hydra pagedCollection info to dcat feed
- Demo data install fails HOT 3
- support for GeoJSON? HOT 2
- New milestone? Is 6.12 fixed?
- support GeoCSV
- Not possible to create demo dataset with php artisan db:seed --class=DemoDataSeeder HOT 1
- Feature Request: Add Map format if Geography data
- What's the DCAT_AP URI of the demo site? HOT 2
- logging frequency configuration HOT 3
- URL Encode metadata fields (e.g. license) HOT 2
- Catch failing jobs
- updating license proces
- Nodes on the map stop rendering properly
- hydra paging
- Project status
- Bug Report
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from core.