drksephy / noaa-projects

Data Visualization Applications built for use by http://www.noaa.gov/

NOAA-Projects

This repository contains source code of various projects I work on as a Graduate researcher at the City College of New York. My tasks involve creating web applications for NOAA-CREST and subsequently NOAA.

Project Directory

To dump out the contents of a netCDF4 file, run the following command:

ncdump -h <filename>

If you are interested in seeing how to pull out datasets from an .hdf file, please see the following script

• special_regions/grid.html: NOAA Squam page with special regions demo. Clicking on any of the bounding boxes redicts to special_regions/regions.html, which is the NOAA Special Regions page.

• squam_visualization/grid.html: NOAA Squam page with voronoi diagram demo. Hovering over each region displays the granule showing the anomaly of sea surface temperature, and clicking any region redirects to a full image of the granule corresponding to the selected region.

Completed Works

NOAA SQUAM Granule Visualization

One of NOAA's web products is the SST Quality Monitor, which maps sea-surface temperature across the globe through granules (images) provided by two satellites which sweep the globe. From the scale below each satellite image, we can see the agreement/disagreement of data, marked by shades of blue and purple.

My task was to write scripts and provide JavaScript which would allow users to hover over regions of the globe and display the corresponding granule. To do so, I used d3 to build a voronoi diagram. The input to these voronoi diagrams are the computed latitude and longitude centers of each granule for all regions of the Earth, which in turn builds a set of SVG polygons.

Voronoi diagrams allow the user to determine which region they are currently in by computing the centers of all polygons, which are also computed inside of a JSON file containing the image and ID associated with each polygon/region. By attaching event handlers to these polygons through D3, the user can hover over each area and see the corresponding granule. By clicking on the region, the granule opens in a new tab with higher resolution.

LICENSE

The actual images used in any of the demos presented here are not contained within this repository, as they are not part of the public domain. The only source code listed on this page are those that are written myself and are seperate from NOAA and NOAA-CREST.

As such, the JavaScript and conversion files may be useful for those interested in Voronoi diagram generation using D3. All source code is licensed under the MIT License. If any part of this source code is reused, a link back to my github profile would be appreciated.

noaa-projects's Issues

Re-implement IDL findfile in Python

For deeper understanding of the IDL script 'findfiles.pro', it should be re-implemented in Python.

Write full script for building json data

A new script has to be written to build the full json data, containing Latitude, Longitude, ID, Image, External link.

Create demo using static page

A demo using a real image with a voronoi diagram above it needs to be built.

Update voronoi with anomaly images

New images need to be configured inside of granule-json.

Generate full granules

All the .nc files need to generate their corresponding granules using the following algorithm:

1. Open file using netCDF4.
2. Find maximum/minimum of temperature.
3. Run CDAT on file using this computed max/min.

Attach url data to each polygon

Each polygon needs an external link to go to.

Code cleanup

With the completion of the recent Squam Visualization project, any lingering code should be removed or cleaned up for future use.

Link voronoi demo to special regions page

The special regions page: 'http://www.star.nesdis.noaa.gov/sod/sst/cwimages/cwimages.html' needs to be linked from the voronoi demo created earlier.

Investigate generating multi-layer images using CDAT

Given a granule, we would like to save out various layers in the form of variables and render these through CDAT. This requires being able to render different variables as different layers, each with their own data range and color scheme.

Update resampling script to write anomaly to file

Using the provided script, it must be altered to write the anomaly variable to a new netCDF4 file.

Generate thumbnails of all granules

For each global day (144 granules), thumbnails need to be generated using the following steps:

1. Open up file using netCDF4.
2. Copy file to temporary file.
3. Subtract sst_reynolds from sst_regression.
4. Save file out with new data.
5. Run CDAT on these files to generate the thumbnails.

Center voronoi diagram properly

For the demo, the voronoi diagram needs to be centered properly.

Voronai diagrams

The voronai diagram and polygons need to be computed using d3 to map data to each one.

Create demo for IMS page

A static demo of a voronoi diagram for October 22nd needs to be built for the NOAA IMS Site: http://www.natice.noaa.gov/ims/

Build request object to send dummy image back to client

Currently, the server can retrieve the computed JavaScript points via AJAX. Using these points, we can build a query to a dummy image API such as placekitten using the pos_x, pos_y coordinates and return an image to the client.

move to csdirs

I just sent out an invite to the csdirs organization. If you like, move this repo to https://github.com/csdirs
but give it a more specific name than "NOAA-Projects". Maybe "squam-demos".

Also, you probably want to remove references to glasslab. This is more of a CSDIRS project.

Update IMS Demo

The current version of the IMS demo has a voronoi diagram in which its cells don't correspond to the correct images. In order to address this, the demo will be built again from the ground up, starting with getting new granules and recomputing the centers of each one.

Write script for central processing

Currently there exists a multitude of scripts for processing granule data, computing the centers, scaling these coordinates, and building the JSON file. It will be convenient to clean everything up and centralize everything.

Write script for computing center longitude/latitude of granules

Using netCDF4, read longitude / latitude values and get the midpoints for each granule, and store it all inside of a dictionary.

Send image url back to client, display it on hover/click

After sending the coordinates to the client using AJAX, the client computes a url which gets sent back to the server to display an image in a new tab.

Open images on click for Voronoi diagrams

By clicking Polygons, the associated image should pop up in a new tab.

Generate anomaly images

The anomaly for all datasets needs to be computed and generated using CDAT for [-4 4] input range.