GithubHelp home page GithubHelp logo

nationalmap's Introduction

TerriaJS

Build Status Docs

Terria logo

TerriaJS is a library for building rich, web-based geospatial data explorers, used to drive National Map, Digital Earth Australia Map, NSW Spatial Digital Twin and NEII Viewer (and many others). It uses Cesium and WebGL for a full 3D globe in the browser with no plugins. It gracefully falls back to 2D with Leaflet on systems that can't run Cesium. It can handle catalogs of tens of thousands of layers, with dozens of geospatial file and web service types supported. It is almost entirely JavaScript in the browser, meaning it can even be deployed as a static website, making it simple and cheap to host.


We have just released a brand new version of Terria — verson 8!

We've put together a list of things we've removed from version 8 and some steps to help you migrate to the new version in our migration guide

Not ready to move to version 8 yet? You can find terriajs version 7 here: https://github.com/TerriaJS/terriajs/tree/terriajs7


Features

  • Nested catalog of layers which can be independently enabled to create mashups of many layers.
  • Supports GeoJSON, KML, CSV (point and region-mapped), GPX, GeoRSS, CZML and zipped shapefile file types natively.
  • Supports WMS, WFS, WMTS, Esri MapServer, Esri FeatureServer, Bing Maps, Carto Maps, Cesium Ion Imagery, OpenStreetMap-style raster tiles, Mapbox, SDMX, 3D Tiles and GTFS and Sensor Observation Service item types.
  • Supports querying WMS, WFS, Esri MapServer, CSW, CKAN, Socrata, OpenDataSoft and SDMX services for groups of items.
  • 3D globe (Cesium) or 2D mode (Leaflet). 3D objects supported in CZML format.
  • Time dimensions supported for CSV, CZML, WMS. Automatically animate layers, or slide the time control forward and backward.
  • Drag-and-drop files from your desktop to the browser, for instant visualisation (no file upload to server required).
  • All ASGS (Australian Statistical Geographic Standard) region types (LGA, SA2, commonwealth electoral district etc) supported for CSV region mapping, plus several others: Primary Health Networks, Statistical Local Areas, ISO 3 letter country codes, etc.
  • Users can generate a reusable URL link of their current map view, to quickly share mashups of web-hosted data.
  • The interface is internationalised and translations are available for French, Italian and Japanese. Partial translations have also been contributed for other languages (see TerriaJS weblate)

Terria screenshot

Who's using TerriaJS?

Sites developed by Data61

Not Data61

Sites we're aware of that are using TerriaJS. These are not endorsements or testimonials.

Technical

  • NodeJS v16, v18 and v20 are supported
  • Built in TypeScript & ES2020+ JavaScript, compiled with Babel to ES5.
  • Supports modern browsers (recent versions of Microsoft Edge, Mozilla Firefox & Google Chrome).
  • TerriaJS Server component runs in NodeJS and provides proxying for web services that don't support CORS or require authentication. Instead of using TerriaJS-Sever proxy service, an alternative proxying service URL can be specified. See Specify an alternative proxy server URL
  • Dependencies are managed in NPM and assembled using WebPack.

Getting Started

The easiest way to build your own Terria-based map is using the TerriaMap starting point. This gives you the HTML structure, server and build processes you need to get a site up and running immediately.

See Getting Started in the Documentation for all the details.

Components and naming

  • Terria™ is the overall name for the spatial data platform and the team that built TerriaJS.
  • TerriaJS is this TypeScript/JavaScript library consisting of the 2D/3D map, catalog management and many spatial data connectors.
  • Cesium is the 3D WebGL rendering library used by TerriaJS, which provides many low-level functions for loading and displaying imagery and spatial formats such as GeoJSON and KML.
  • TerriaMap is a complete website starting point, using TerriaJS.
  • TerriaJS-Server is a NodeJS-based server that provides proxying and support services for TerriaJS.
  • NationalMap is the flagship Terria deployment, and the origin of the TerriaJS library.

Related components

  • TerriaMapStatic, a pre-built version of TerriaMap, which can be deployed as a static HTML website, such as on Github Pages.

Big Thanks

Hosting and contribution framework for community translations of TerriaJS provided by Weblate under the Libre plan for open source software. See our translation progress for different languages:

Translation status

Join the community

Get in touch!

Publishing TerriaJS

From main using GitHub Actions (preferred method):

  • Checkout and pull main branch of terriajs, run yarn install if necessary.
  • Bump the version number in package.json.
  • Review and edit CHANGES.md.
    • Ensure that entries for new changes are written in the right section by diffing against the last version. Make sure the section name matches the version you set in package.json above.
    • e.g. git diff 8.2.25 HEAD -- CHANGES.md (where 8.2.25 is the previous published version)
    • Or using GitHub web interface https://github.com/TerriaJS/terriajs/compare/8.2.25...main (change the version and master/next to match the previous release and branch)
    • Finalise the heading of the version to be released with version number and date.
    • Add a heading above for upcoming changes to be documented under.
  • Commit and push your changes on a branch and make a PR to main branch.
  • Get someone to review and merge the PR.
  • Wait for slack notification of successful/failed publish.
  • 😄

Publishing to npm without the GitHub Action (not for standard releases):

This method should not be used to publish versions at the tip of main or another branch that is often used for releases. To publish a new version of TerriaJS to npm with this method, you'll need to be listed as a collaborator here. Existing collaborators can add new ones. Then:

  • Checkout and pull the branch of terriajs to be deployed, run yarn install if necessary.
  • Bump the version number in package.json. Follow semver.
  • Appropriately update CHANGES.md, ensuring that changes are listed in their correct sections.
  • Commit and push your changes and make a PR to the branch to be deployed.
  • Get someone to review and merge the PR.
  • Fetch and checkout the merge commit made by merging the PR.
  • rm -rf wwwroot/build
  • Make sure you don't have any changes in your working directory.
  • gulp lint release
  • npm publish --tag your-tag (for a release not at the tip of main a tag must be set, and it must not be latest or next. For releases on old versions you could use e.g. 6-5-x or 6-x-x as a tag, for upcoming features a short name could be used)
  • Ensure that CHANGES.md on main is also updated to list the release and clearly note the nature and reason for release.

The above will publish a new version to npm and also tag that version on GitHub.

nationalmap's People

Contributors

a-stacey avatar alexgilleran avatar anabelgun avatar bsimyo avatar chloeleichen avatar greenkeeper[bot] avatar greenkeeperio-bot avatar keyboardsounds avatar kring avatar layoric avatar maxious avatar meh9 avatar mwu2018 avatar na9da avatar nahidakbar avatar nf-s avatar peterhassall avatar philipgrimmett avatar quadtog avatar racingtadpole avatar rdengate avatar reginapramesti avatar rowanwins avatar snayler0 avatar soyarsauce avatar stevage avatar steve9164 avatar tobybellwood avatar wibily avatar zoran995 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

nationalmap's Issues

Add support for the Atlas of Living Australia

Atlas of Living Australia (ALA) has what/where/when data for 57m animal/plant sightings. They have an API - details at http://api.ala.org.au - this includes the use of Apache Solr to generate WMS layers. In NM, we could allow for users to search for an animal/plant and then show the resulting WMS layer of sightings. They have been a couple of requests from users wanting to see ALA on NM.

IE 9 Layers added by Add Data dont display on the map when selected

Hi Team,

Troy Gill here at GA came across an issue recently when testing the latest release on IE 9. Both Firefox and Chrome work as expected when added an external layer, but can't get data to show up on the map for IE9.

For the test, the following URL was used:

http://www.ga.gov.au/gis/services/topography/Dynamic_National_Map_Culture_and_Infrastructure/MapServer/WMSServer?request=GetCapabilities

No errors are shown and UI behaves the same as other browsers, just missing the data. After using the IE 9 network capture tool, it looks like the requests for layer image data just aren't being made by the application.

image

Steps to reproduce:

  1. Click Add Data
  2. Paste provided URL and click add.
  3. Select one/many layers from the added endpoint to view data.

Expected:
To see associated data on map (works in Chrome/Firefox)
Actual:
Map doesn't change in IE9

GA WMS layers not centring correctly

GA has just enabled their new services (for example). They display in NM, but clicking on a layer name (for example, Elevation > Reefs and Shoals) zooms to 0,0.

The getCapabilities response looks superficially correct:

<EX_GeographicBoundingBox>
<westBoundLongitude>112.92145</westBoundLongitude>
<eastBoundLongitude>153.46622</eastBoundLongitude>
<southBoundLatitude>-43.65649</southBoundLatitude>
<northBoundLatitude>-11.00691</northBoundLatitude>
</EX_GeographicBoundingBox>
<BoundingBox CRS="CRS:84" minx="112.92145" miny="-43.65649" maxx="153.46622" maxy="-11.00691"/>
<BoundingBox CRS="EPSG:4326" minx="-43.65649" miny="112.92145" maxx="-11.00691" maxy="153.46622"/>
<BoundingBox CRS="EPSG:4283" minx="-43.65649" miny="112.92145" maxx="-11.00691" maxy="153.46622"/>

You can see the new list here temporarily.

Move Project Documentation to Repo from Wiki

So external parties can potentially contribute universal gaps/tips... for example

For Testing Purposes Only you can deploy and share your own Version/Demonstration/Test/Beta/Pilot/Etc of The NationalMap with others quickly by remapping Port 80 to Port 3001 with this command:

iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 80 -j REDIRECT --to-port 3001

Web search redirecting/remapping iptable ports for instructions/troubleshooting.
NOTE: this is a temporary change on the operating system - if used with instances that will/may be rebooted - you will need to ensure the proper startup scripts to automatically able these changes on reload. HOWEVER this is meant to be a temporary means to provide world-wide access to prototypes NOT as a permanent solution as production services should be using a dedicated/trusted standard web server/proxy - either using a fully-feature, dedicated, and robust solution like Nginx or a lightweight web-proxy or cache like Varnish is used to current support production - to redirect external traffic to the application. These services are important for security reasons too as they can ensure the application is run as a general user NOT as the root user (Only root users can run services from lower level Port #s) this allows you to ensure that even if the application is exploited ~0.01% risk +1 then the system would not be susceptible (As if that mattered in the age of containerization and isolation :)

Sorry the last Paragraph was a personal rant that would be removed

Make subgroups under data.gov.au

This suggestion came from Peter Leihn. There are getting to be many data providers on data.gov.au and it's getting messy, so we could group:

  • Victorian councils
  • NSW councils
    (etc

Maybe leave the federal departments as they are for now?

Not sure about implementation of this.

Remove GA topography service from data services

GA have said that most or all data under the Topography data provider will be available through new data collection file with the new GA data services (and with better naming). So we should remove the Topography line under "data providers"

Need ability to 'immediately' update cached data for NM

Comms is requesting the ability to 'immediately' update data to NM when it is updated on their site - in this case the wms server behind the ckan server at data.gov.au. The easiest approach is to add purge functionality to our varnish script which will work for sites at 1 ip address or sites we control the AWSinstance for. For the nationalmap.gov.au site which we don't have AWS access to, this will require either a policy update by GA or something more clever by us.

New GA basemaps

From GA email:

The new base map has been published but it has different URL endpoint because we will have both a colour and a grayscale version of the same and wanted to keep consistent naming convention to reflect that difference. GA is also going away from specifying projection in a service name. Here is the link to the new service in our REST catalogue:
http://www.ga.gov.au/gisimg/rest/services/topography/National_Map_Colour_Basemap/MapServer

For completeness, the old service was:
http://www.ga.gov.au/gis/rest/services/topography/National_Map_Basemap_WM/MapServer/WMSServer

It means, the app will need to point to the new base map cache.

The web service for grayscale map is still awaiting to be published.

Geelong data not displaying correctly

Hi,

  1. The "Central Geelong 3D Massing Model" (from Data.gov.au) layer is not displaying. When I click on it it takes me out to the Atlantic Ocean and does not display any models
  2. The Geelong Roofprints layer generates an error message "error accessing catalogue item". But it does display if you zoom in closer and tilt into a oblique view. It seems to have a problem with the full extent and ortho view.
    Regards
    Craig

Change message when zooming in beyond maxscale

The message currently says:


The "Layer Name" dataset will not be shown when zoomed in this close to the map because the data custodian has deemed it to be too inaccurate to show at this scale...


Change to:


The "Layer Name" dataset will not be shown when zoomed in this close to the map because the data custodian has indicated that the data is not intended or suitable for display at this scale. ..


Nationalmap doesn't build with process in wiki

Hi guys,
I follwed the instructions on the wiki. However, when I run
npm install
I get:

npm WARN unmet dependency /home/buggeru/nationalmap/node_modules/gulp-jsdoc requires gulp-util@'~2.2.14' but will load
npm WARN unmet dependency /home/buggeru/nationalmap/node_modules/gulp-util,
npm WARN unmet dependency which is version 3.0.6

Running gulp gives
Error: Cannot find module 'through2'
at Function.Module._resolveFilename (module.js:338:15)
at Function.Module._load (module.js:280:25)
at Module.require (module.js:364:17)
at require (module.js:380:17)
at /home/buggeru/nationalmap/node_modules/gulp-jsdoc/index.js:4:18
at Object. (/home/buggeru/nationalmap/node_modules/gulp-jsdoc/index.js:141:2)
at Module._compile (module.js:456:26)
at Object.Module._extensions..js (module.js:474:10)
at Module.load (module.js:356:32)
at Function.Module._load (module.js:312:12)

For reference uname -a

Linux terminus 3.19.0-28-generic #30-Ubuntu SMP Mon Aug 31 15:52:51 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux

Help please!
Cheers,
B

some incorrect coordinates via Search Locations

Do a search for "Shoalhaven Heads". The result in "Locations" will zoom to the northern hemisphere. Obviously the record has a "positive latitude" data entry mistake. Not sure what the source is for the Locations search. This indicates that its system needs a programmatic scan to validate anything with Australia in the name to be within an Australian bounding rectangle.

(Not sure where to report this, so starting here.)

High-Latitude WMS 'Bug'(?)

I have been working with some data in the National Map application. I encountered one odd error loading WMS data that leads me to wonder if it is a limitation of the design, or a bug.

When I load some of my datasets from a WMS in the Cesium Sandcastle (ie. the ice concentration layer shown in the attached images) it loads completely:

iceconcentration_sandcastle

However, when I load the same WMS into your implementation, a wedge of data is missing, extending from the North Pole out over the Bering Sea. I suspect higher latitudes are not what your application was aimed for, but I found this curious.

iceconcentration_nicta

This same occurs no matter what dataset is loaded. I was wondering if you had any ideas as to why this might occur. I don’t believe it is in the settings of my geoserver data store, but want to be sure. I’d like something like this to work consistently. If it is a part of the design of National Map, perhaps you could point me where in the build this is effected? Thanks!

Left column of help text doesn't scroll

If the window size isn't big enough for display of the full left hand text (ie headings) vertically, it is off the bottom of the screen and can't be reached. The right hand column has a scroll bar so is fine. Tried on Firefox and Chrome and is a problem on both.

ABS Data Services timing out on first open

While testing this I noticed that many ABS Data Services layers time out when they're first loaded, but quickly load on second attempt. I don't know if this is something we can fix - or maybe we can just extend the timeout.

In csv-geo-au spec and code support, add in poa_code as synonym for poa

@stevage I see that the new csv support now doesn't recognise poa_code for ABS postal areas (which it used to support) and supports poa and poa_2011 instead. It seems that ABS themselves often use poa_code (eg as for POAs, they distinguish between poa_name and poa_code). Eg see http://www.abs.gov.au/AUSSTATS/[email protected]/Lookup/1270.0.55.003Explanatory%20Notes10July%202011?OpenDocument and https://censusdata.abs.gov.au/arcgis/rest/services/2011_POA/MapServer/0

Given that we want things to work well for unambiguous cases and that this is one, I think we should add poa_code as a synonym for poa. https://en.wikipedia.org/wiki/Robustness_principle

As well as that, there is also backwards compatibility for people who have already been using CSVs with poa_code (including the standard csv file used in most NM demos).

Misc changes to the NM About page

Here are some changes to the About page - the first ones are requests from DoC and the later ones from me...
Requested by DoC:

  1. Add logos for DoC, GA and NICTA to the left of the ‘About the NationalMap’ text with links to the respective websites. For DoC, see http://www.communications.gov.au/resources/logo (use the ‘stacked regular’ version of the DoC logo). Guidelines are provided at the link under the heading ‘Logo’. For GA, I'll forward the details to the person doing this.
  2. Add CeBIT Awards Finalist logo below these logos
  3. Add a link to the DoC intro video about National Map (URL coming soon) on the right hand side. DoC have asked for a video camera icon - if we want something different, we could do that and have them review,

And here are some other changes from me:
4) Now that we've removed the NM help button (to make room for "related maps"), the reference to the "help button" should be changed to the "about button".
5) In the "NationalMap Software" section, this should now refer to TerriaJS and point to the TerriaJS github site.
6) In the "Additional open source software used", TerriaJS should be added as the first one.(Yes I know it's not necessarily "additional" if describe in the prev section but I think it's worth having in the list to avoid anyone thinking NM is just the main nm js file using cesium and leaflet.).
7) The list of open source software used should be updated or better still, after mentioning TerriaJS, Cesium and Leaflet, just linking to a full list on the NM or TerriaJS github site.
8) In this section, "our github site" should be changed to "the github site" (if that survives the above changes)(Note: this was from DoC)

Change elevation to use AHD

We've had requests to change the elevation data to show Australian Height Datum for NM (Terria should be configurable so that this is an option for developers and NM would use AHD). DoC and GA would prefer that NM uses AHD too.

AREMI NationalMap has complex Share URL

Using the "Share" feature of the main NationalMap provides a very concise URL, and this works beautifully. Also that is fine for the Northern Australia NationalMap.

However the AREMI NationalMap gives a very long and complex URL that would be prone to breaking, and could not be reliably shared.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.