Comments (7)
Hey, thanks for giving this a start! It turned out that I had made this repo a while ago, probably looking to do a similar thing, but then didn't end up doing it.
I just tried the package, but couldn't seem to get it to instantiate - Julia was unable to download the initial artifact, and when I tried that link on Github it hung indefinitely. Strangely, even when I tried to get the raw file from Github I had the same issue.
We could download the Shapefile zips directly from the source, which is hosted by AWS and has a considerably faster download. That means that we'd need to somewhat "roll our own" artifacts, and we couldn't use artifact-strings in that case, using artifacts only as a storage solution.
I wouldn't mind writing that up, and it shouldn't require too much maintenance either. Discovery is actually possible through their Github repo, so there's that advantage as well. Unfortunately, in this case, we would not be able to track by version as we might on Github,
All in all I think the design looks good!
We should definitely download from tags - we will have to manually update them as time goes by but it shouldn't be too hard; it could conceivably be automated using e.g. Github Actions as well.
The current dependencies in Project.toml are not really necessary,
Agreed. We probably only need GeoJSON as a dependency (or GeoJSON2 if it looks like we need speed).
Down the road we can discuss integration with GeoMakie.
This should slot nicely into GeoMakie as a dependency so we can supply coastlines in a more flexible way. No changes will be needed on this end AFAIK.
from naturalearth.jl.
I think I've fixed instantiation errors now, and also moved to installing by tags. Let me know if you still have any issues.
Running build/create_artifacts.jl
takes about a minute or two (which includes temporarily downloading all artifacts), so I don't think download speeds should be an issue. Though I'd be happy to move to shapefile zips if that's significantly faster (we could also provide both sources as prioritized alternatives in Artifacts.toml, with geojson as backup).
In case we use Shapefiles, I think we'd have to implement my functions from MakieOrg/GeoMakie.jl#139 here.
from naturalearth.jl.
I found githack.com which can act as a CDN for github, so we can use that as a secondary download link if the first one fails. With that change, it works on my machine. Will push this up shortly.
GeoJSON is getting some perf improvements now, so I'm not too worried about that.
The issue with zips is that Artifacts
doesn't natively support those, so we'd have to unpack them ourselves, possibly using scratchspaces. This sounds like a hassle, so let's save that until and unless there's no other option?
from naturalearth.jl.
githack looks really neat!
Looks like tar.gz
is supported natively by Artifacts
, but I don't know how much of an advantage they are compared to uncompressed files, especially since we (currently) don't bundle different data files in the same Artifact.
- Though it isn't inconceivable that bundling the same data (e.g. coastlines) at different resolutions, or "similar" data types at the same resolution is a terrible idea... Though it sounds like quite a bit of manual (and subjective) work to make those decisions, which I don't think is advisable now, if ever.
from naturalearth.jl.
Yeah, I like the framework we have going on now. One thing I do want to do is to add a basic dataset search, i.e., I should be able to do some kind of search like datasets(scale = 10, keywords = ["admin0"])
within the package. Otherwise in terms of vector functionality we're pretty much set.
Would you be interested in moving this package to the JuliaGeo org? If so I can reach out to them and ask. It would probably make this package a little more discoverable, as well as having more people around to help with maintenance etc.
Looks like tar.gz is supported natively by Artifacts
Tarballs and gzips are, but .zip
is not, unfortunately :(. That's why we'd have to do this whole scratchspaces/manual unzipping thing, otherwise they'd just be manual additions to the artifact list.
from naturalearth.jl.
Yeah, I totally agree with search. It may also possibly be convenient with some helper functions that guide users through the "hierarchy" of vectors: i.e. category = [admin, bathymetry, coastline, ...]
, resolution=[10, 50, 110]
, etc.
Also, very happy about moving the package to JuliaGeo. I was just invited as collaborator there earlier today, so that wouldn't make any difference to me.
(ps: apologies, I just force pushed to master to fix a stupid bug I introduced in a small change I just made, I hope you didn't notice)
from naturalearth.jl.
Sounds good - looks like that and rasters are our only TODOs before this is ready to go for all available data!
About JuliaGeo: I just realized that I'm a member there already, so I'll transfer the repo now.
No worries - also I'm not reading a force push on my end, so I think it's fine. But yeah, now that it's semi-working we should probably not push directly to master anymore :D
from naturalearth.jl.
Related Issues (5)
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from naturalearth.jl.