gina-alaska / dans-gdal-scripts

A number of utilities for use in conjunction with GDAL.

dans-gdal-scripts

This package consists of a number of utilities for use in conjunction with GDAL. Currently these programs are only supported on Unix systems, but it should be possible to compile them on Windows if you know how to make the corresponding Makefiles.

Programs

gdal_contrast_stretch

Contrast stretch and conversion from 8,16-bit or floating point to 8-bit

gdal_dem2rgb

Generate hillshaded images from DEMs

gdal_get_projected_bounds

Project a polygon and return its bounding rectangle

gdal_landsat_pansharp

Pansharpening - works best for Landsat 7 images

gdal_list_corners

Prints raster geocode information in YAML format (similar to gdalinfo but gives YAML)

gdal_merge_simple

Merge individual bands into a single GeoTIFF image (8-bit only)

gdal_merge_vrt

Merge individual bands into a single VRT image

gdal_raw2geotiff

Convert raw binary files into GeoTIFFs

gdal_trace_outline

Trace the outline of an image and generate WKT or Shapefile (options exist for cleaning up raggedy edges, can also do feature classification)

gdal_wkt_to_mask

Generate a bitmap of the area covered by a polygon

Install

Debian/ubuntu: apt-get install dans-gdal-scripts

from source:

./autogen.sh
./configure  # --prefix=/your/favorite/prefix
make
make install

OSX (homebrew)

To get dans-gdal-scripts working on OSX you'll need to get a few deps:

• Homebrew
• brew install gdal
• brew install boost

Credits

These programs were written by Dan Stahlke of the Geographic Information Network of Alaska. Send questions or comments to [email protected].

dans-gdal-scripts's Issues

gdal_trace_outline producing incorrect results

I have a raster (a_same_value_correct_ndv.tif) in which all values (except no data value) are 1 (Int16). NDV = -32768, i ran gdal_trace_outline but its producing incorrect results. I was expecting the output to exactly match the raster boundary but its not the case. See Incorrect Output . In this , grey area represents the raster and green is the output of gdal_trace_outline

gdal_trace_outline a_same_value_correct_ndv.tif -ndv -32768 -min-ring-area 50 -out-cs en -no-donuts -dp-toler 0.1 -ogr-out a_same_value_area50_donuts_toler0_1.shp

Parameter --quiet

-q or --quiet for be quiet.

calculate alpha channel directly in gdal_trace_outline

Add this function to polygon-rasterizer.cc and we could get the alpah channel tiff file directly . Then we can use gdal_merge_simple to generate the rgba image.

There is no need to generate the temporary pbm and png files .

void mask_alpha_from_mpoly(const Mpoly &mpoly, size_t w, size_t h, const std::string &fn) {
printf("mask draw: begin\n");
printf("%d,%d\n",w,h);
std::vector<row_crossings_t> rows = get_row_crossings(mpoly, 0, h);
GDALAllRegister();
GDALDatasetH hDstDS;
GDALDriverH hDriver = GDALGetDriverByName("GTiff");
char **papszOptions = NULL;
hDstDS = GDALCreate( hDriver, fn.c_str(), w, h, 1, GDT_Byte, papszOptions );
if( hDstDS == NULL )
{
printf("cannot open mask output\n");
}
GDALRasterBandH hBand = GDALGetRasterBand( hDstDS, 1 );
uint8_t *line=(uint8_t *)CPLMalloc(sizeof(uint8_t) * w);

    printf("mask draw: write\n");
    for(size_t y=0; y<h; y++) {
            uint8_t bitp = 128;
            const row_crossings_t &r = rows[y];
            for(size_t i=0; i<w; i++) {
                    uint8_t v = 1;
                    // not the fastest way...
                    for(size_t j=0; j<r.size(); j++) {
                            if(double(i) >= r[j]) v = !v;
                    }
                    if(v) line[i] = 0;
                    else line[i] = 255;
            }
            GDALRasterIO(hBand, GF_Write, 0, y, w, 1, line, w, 1, GDT_Byte, 0, 0 );
    }
    if(line != NULL)
    {
            CPLFree(line);
    }
    GDALClose( hDstDS );

    printf("mask draw: done\n");

}

GDAL v3 breaking changes

How does this library need to change to accommodate GDAL v3's change to axis ordering? I see several places that this code directly uses the "u" and "v" parameters on the transform. Would those areas be affected?

Error during installation

Dan,

I am getting the following error during the make step:

/Library/Developer/CommandLineTools/usr/bin/make all-recursive
Making all in src
g++ -g -O2 -L/usr/local/opt/icu4c/lib -o gdal_raw2geotiff gdal_raw2geotiff.o common.o -framework GDAL
ld: framework not found GDAL
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [gdal_raw2geotiff] Error 1
make[1]: *** [all-recursive] Error 1
make: *** [all] Error 2

It seems that during ./configure step the GDAL is recognized already, as shown below:

checking for gdal-config... /Library/Frameworks/GDAL.framework/Programs/gdal-config

My GDAL is not from brew, rather from QGIS package, as brew version conflicts somehow with QGIS.

Is there any parameter to add to ./configure to fix the problem?

Many thanks
Abdel

full ./configure output is listed below:

checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... ./install-sh -c -d
checking for gawk... no
checking for mawk... no
checking for nawk... no
checking for awk... awk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking for g++... g++
checking whether the C++ compiler works... yes
checking for C++ compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C++ compiler... yes
checking whether g++ accepts -g... yes
checking whether make supports the include directive... yes (GNU style)
checking dependency style of g++... gcc3
checking for gdal-config... /Library/Frameworks/GDAL.framework/Programs/gdal-config
checking how to run the C++ preprocessor... g++ -E
checking for grep that handles long lines and -e... /usr/bin/grep
checking for egrep... /usr/bin/grep -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking cpl_conv.h usability... yes
checking cpl_conv.h presence... yes
checking for cpl_conv.h... yes
checking cpl_port.h usability... yes
checking cpl_port.h presence... yes
checking for cpl_port.h... yes
checking cpl_string.h usability... yes
checking cpl_string.h presence... yes
checking for cpl_string.h... yes
checking gdal.h usability... yes
checking gdal.h presence... yes
checking for gdal.h... yes
checking ogr_api.h usability... yes
checking ogr_api.h presence... yes
checking for ogr_api.h... yes
checking ogr_spatialref.h usability... yes
checking ogr_spatialref.h presence... yes
checking for ogr_spatialref.h... yes
checking ogrsf_frmts.h usability... yes
checking ogrsf_frmts.h presence... yes
checking for ogrsf_frmts.h... yes
checking for Boost headers version >= 1.37.0... yes
checking for Boost's header version... 1_69
checking for ANSI C header files... (cached) yes
checking for an ANSI C-conforming const... yes
checking for stdbool.h that conforms to C99... no
checking for _Bool... no
checking for size_t... yes
checking for stdlib.h... (cached) yes
checking for GNU libc compatible malloc... yes
checking for stdlib.h... (cached) yes
checking for GNU libc compatible realloc... yes
checking for working strtod... yes
checking for memset... yes
checking for sqrt... yes
checking for strtol... yes
checking for pow in -lm... yes
checking that generated files are newer than configure... done
configure: creating ./config.status
config.status: creating Makefile
config.status: creating src/Makefile
config.status: creating config.h
config.status: config.h is unchanged
config.status: executing depfiles commands

Serious bug in gdal_trace_outline (produces broken geometry)

Hi,

I use gdal_trace_outline as part of a workflow to generate national datasets of land use in Norway. This year I took a fresh copy of dan's scripts from master on January 30th, and the resulting map had corrupted polygons all over it.

I went back to the version of dans-gdal-scripts that I previously built from February 2013 and there was no problem.

Then I rebuilt the February 2013 version in case the problem was with some supporting libraries. Again no problems, so it's not the supporting libraries.

Then I went back and rebuilt the 2015/01/30 version from scratch to check the build was correct, but the corrupted polygons showed up in the same places.

So it's an algorithm or raster import bug that has been introduced since February 2013.

You can see the effects below.

This first picture shows the problem is affecting pretty much all of our tiles that contain data. You can see the tiling we're using. gdal_trace_outline is getting one tile at a time.

On the top left, here's a zoomed in version of the buggy output from current stable gdal_trace_outline. Below is the output from gdal_polygonize for a neighbouring tile. To the right is the source tifs for neighbouring tiles.

Top left - original source tile of the tif.

Top left - output from gdal_trace_outline_201302. (february 2013).

Top left - output from gdal_trace_outline_20150130 (rebuilt from scratch).

Flags used are: "POLYGONIZE_FLAGS="-classify -out-cs en -split-polys -dp-toler 0 -bevel-s
ize 0.01 -ogr-fmt 'ESRI Shapefile' -ogr-out ""

Any ideas?

I can give you a diff of the gdal_trace_outline code between the working and broken versions if it's helpful.

(Also, I notice that the older version of the scripts did a ./configure check for proj4-devel which doesn't happen now. Is that deliberate)?

How to install it on a Linux instance of Amazon EC2?

Could it be installed on Amazon EC2 where apt-get is no recognize. Thanks.

Report invalid options in gdal_landsat_pansharp

If you mis-type an option like -o as -out it just reports the usage message. It would be helpful if it said there was an invalid option

Some extra steps to install scripts on MacOS

I just noticed my Mac does not have autoconf and automake installed by default. I thought it is beneficial to list the extra step below for others trying to install the scripts on a Mac system:

foo@Macbar: brew install autoconf automake libtool

gdal_trace_outline ignores alpha layer

I branched the code and modified ndv.cc to handle alpha layers. Mind taking a look? I'm not sure my approach was right, though it does work for me in my situation.

Also in my branch: some changes to use C++11 to avoid Boost references, changes to make the major ring and holes parameters work simultaneously, and an attempt to make erosion parameterizable.

Link: https://github.com/asirobots/dans-gdal-scripts

request for gdal_trace_outline.cc

First thanks for this repository. This is not a bug report but a new feature request. There was some discussion on supporting this capability in released gdal tools which I would love to see. However I would like to see two more options.

(1) an option for a field to be created (e.g. filename) with the image's name stashed in the file. This helps later with merging into one file.
(2) To help with merging, if the shapefile already exists can the new polygon be appended to the file.

thanks,
Trent

Smooth edges on gdal_trace_outline

gdal_trace_outline is a godsend but tends to produce very jagged edges from SRTM data. Applying the Douglas-Peucker simplification results in sharp bends at corners and unnatural-looking shapes. Turning this off produces sawtooth edges.

It would be good to have more 'natural'-looking smoothing, similar to the output produced (in raster form) by gdaldem.

There are some ideas in this thread (which is how I found gdal_trace_outline!). Right now I'm using a combination of ogr2ogr and Mapnik's line-smooth, which produces reasonable results, but it would be better to have it in gdal_trace_outline.

A bug of destructor of BboxBinarySpacePartition class？

In windows 10，the destructor of BboxBinarySpacePartition class can't free the memory as expected.

In the function void fix_topology(const Mpoly &mpoly, std::vector &reduced_rings)

Unknown conversion type character 'z' in format

When compiling on Windows 7 (using mingw32) I get several warnings like this:

gdal_landsat_pansharp.cc:205:65: warning: unknown conversion type character 'z' in format

Once compiled the programs function normally, except where there should be numbers in the output strings, eg:

Output size is zd x zd x zd

Usage info for gdal_contrast_stretch

Contrast stretch and conversion from 8,16-bit or floating point to 8-bit

I would be very grateful if you could add usage instructions in the README?

Can I use this script to stretch the contrast of one TIF file based on another?

Building the tool to work with GDAL 2.1.3

Hello,

Is it possible to build the tool using GDAL 2.1.3 on Mac? I tried it but it didn't work, the build is ok, but the tool doesn't work

thanks
K