Comments (7)
I really like this idea
from osm-gpt.
We can get sufficient query , that won't be a problem at the end its a pair between language keywords with overpass query !
For the model itself
My Initial Thoughts :
llama : Released By Meta
https://ai.meta.com/llama/
I haven't tested this out yet intensively but looks like a promising model as they advertise , They say its free for researchers and commercial use but I am concerned with their license , Lot of people say its not open source : https://blog.opensource.org/metas-llama-2-license-is-not-open-source/
I have tried their LLaMA2-70B Model , Not good as chatgpt , and produces random query . I wonder if it can be retrained then this could work
Another option is stable LM
https://github.com/Stability-AI/StableLM
And this one binding of llma cpp in python that can be used as api :
https://github.com/abetlen/llama-cpp-python
from osm-gpt.
Update on LLama model training :
Here is the sample dataset that can be used to train LLama with RLHF
https://huggingface.co/datasets/HuggingFaceH4/stack-exchange-preferences
Now we need similar training dataset to be prepared for overpass questions and query ! There will be some challenges : We might search for training data , in order to retrain or play with it , it needs massive GPU and machine , I tried a demo with collab couldn't go through on free version
Two references :
https://lightning.ai/pages/community/tutorial/accelerating-llama-with-fabric-a-comprehensive-guide-to-training-and-fine-tuning-llama/
https://huggingface.co/blog/stackllama
Looks like it can run on 8GB of GPU which is good a standard personal computer nowadays have this , but a Solid GPU and training dataset is needed ,Training dataset is something we can generate by asking community , bootstrapping overpass query examples , Challenge is the machine
from osm-gpt.
Thank you @kshitijrajsharma for sharing the sample dataset and references for training LLama with RLHF, as well as the insight into the challenges ahead. The stack-exchange-preferences dataset seems valuable, and it's promising that LLama can potentially run on an 8GB GPU. Although generating a similar training dataset for overpass questions will require effort, we are now equipped with the knowledge to proceed. I appreciate the progress made so far and believe we are ready to collect data for testing fine-tuning. Collaboration within the community will be vital in overcoming the challenge and achieving success. Let's continue to build on these findings and unlock LLama's full potential!
from osm-gpt.
@kshitijrajsharma Thank you for showing interest and offering to contribute! I really appreciate your enthusiasm. Your idea of using open-source LLMs to save on API costs is excellent. We can explore options like Meta's Llama2, a recently released LLM, or other advanced models.
I'd love to hear your ideas and suggestions on how we can acquire a sufficient amount of OSM query data to train the model effectively. Please share any potential sources or data collection methods, you think would be beneficial.
Let's collaborate and make this project cost-effective!
from osm-gpt.
Yes ! There is a option for collecting data from the tool within though ! Need to check licensing for it ! Now I am looking at training datasets , let me know if you find any options
from osm-gpt.
Hello and thanks for this beautiful repo:)
If you'd like to integrate a self hosted solution, please consider Ollama.
Best,
Orkut
from osm-gpt.
Related Issues (9)
- I need to be able to upload my own aoi and get the data for that area.
- export option : cendroids
- How to change the map coordinates?
- Rate limit error while query to https://osm-gpt.rohitgautam.com.np/
- Rate limited: message HOT 2
- Geojson Extent To Zoom Into Layer HOT 2
- Combined Searches HOT 2
- Feature properties(attributes) in Popup
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from osm-gpt.