GithubHelp home page GithubHelp logo

evanmiller / projcl Goto Github PK

View Code? Open in Web Editor NEW
88.0 8.0 10.0 202 KB

GPU and vector-enabled map projections, geodesic calculations, and image warping ๐ŸŒŽ๐ŸŒ๐ŸŒ

License: MIT License

CMake 9.26% C 90.69% C++ 0.05%
opencl map-projections geo

projcl's Introduction

GitHub Actions build

ProjCL: OpenCL-powered map projection, geodesic, and image-warping library

ProjCL is a C interface to OpenCL routines that perform various geographic computations, including map projection, geodesic (distance) calculations, datum conversion, and basic image warping (reprojection). For projection calculations it is often 4-10X faster than Proj.4 on the CPU, and 15-30X faster on a low-end GPU with large batches of coordinates. For datum shifts ProjCL is smarter than Proj.4 because it does some matrix math in advance, and generally faster because OpenCL can utilize all cores and the CPU's vector capabilities.

Most projection routines were originally adapted from Proj.4 code, with branches replaced with select() statements and various tweaks implemented along the way. The library was developed to support real-time map projections in Magic Maps.

All of the routines are single-precision. In theory, single-precision floats can represent positions on the Earth with about 1m of accuracy. In practice, the test suite guarantees 10 meters of accuracy in projected coordinates, and one arc-second of accuracy in geodetic coordinates (about 30 meters at the Equator). Most routines have been carefully written to avoid round-off error, and have at least twice the accuracy that these guarantees imply.

Double-precision should probably be implemented at some point, but it hasn't been a priority.

The API differs from Proj.4 in that projections are specified using compile-time constants, and parameters are specified using a dedicated data structure. Text-based C APIs like Proj.4's are prone to error in my experience.

A test suite covers the projection routines, but not the geodesic calculations, datum shifts, or image warping. The output is checked for self-consistency, as well as agreement with Proj.4. The code is tested to work on OS X as well as Linux; when making a pull request, please ensure all the tests pass with ./test/projcl_test -CPU.

ProjCL needs more map projections. In fact, the world needs more map projections. If you want to try your hand at one, check out "Adding a Map Projection" below.

Available projections:

  • Albers Equal Area
  • American Polyconic
  • Lambert Azimuthal Equal Area
  • Lambert Conformal Conic
  • Mercator
  • Robinson
  • Transverse Mercator
  • Winkel Tripel

Available datums and spheroids: see include/projcl/projcl_types.h

Available image filters: see include/projcl/projcl_warp.h

Building

ProjCL requires CMake build system. To build the library do:

$ cmake CMakeLists.txt
$ make	

Setup

#include <projcl/projcl.h>

cl_int error = CL_SUCCESS;

PLContext *ctx = pl_context_init(CL_DEVICE_TYPE_CPU, &error);

PLCode *code = pl_compile_code(ctx, "/path/to/ProjCL/kernel", 
        PL_MODULE_DATUM 
        | PL_MODULE_GEODESIC 
        | PL_MODULE_PROJECTION
        | PL_MODULE_WARP 
        | PL_MODULE_FILTER);

error = pl_load_code(ctx, code);

Teardown

pl_unload_code(ctx);
pl_release_code(code);
pl_context_free(ctx);

Forward projection

/* get the input data from somewhere */
/* latitude-longitude pairs */
int count = ...;
float *lat_lon_data = ...;

/* load point data */
PLProjectionBuffer *proj_buffer = pl_load_projection_data(ctx, lat_lon_data, count, 1, &error);

/* allocate output buffer */
float *xy_data = malloc(2 * count * sizeof(float));

/* Set some params */
PLProjectionParams *params = pl_params_init();
pl_params_set_scale(1.0);
pl_params_set_spheroid(PL_SPHEROID_WGS_84);
pl_params_set_false_northing(0.0);
pl_params_set_false_easting(0.0);

/* project forwards */
error = pl_project_points_forward(ctx, PL_PROJECT_MERCATOR, params, proj_buffer, xy_data);

/* unload */
pl_unload_projection_data(proj_buffer);
pl_params_free(params);

Inverse projection

/* get the input data from somewhere */
/* X-Y pairs */
int count = ...;
float *xy_data = ...;

PLProjectionBuffer *cartesian_buffer = pl_load_projection_data(ctx, xy_data, count, 1, &error);

float *lat_lon_data = malloc(2 * count * sizeof(float));

PLProjectionParams *params = pl_params_init();
pl_params_set_scale(1.0);
pl_params_set_spheroid(PL_SPHEROID_WGS_84);
pl_params_set_false_northing(0.0);
pl_params_set_false_easting(0.0);

error = pl_project_points_reverse(ctx, PL_PROJECT_MERCATOR, params, cartesian_buffer, lat_lon_data);

pl_unload_projection_data(cartesian_buffer);

Datum shift

/* load lon-lat coordinates */
int count = ...;
float *xy_in = ...;
PLDatumShiftBuffer *buf = pl_load_datum_shift_data(ctx, PL_SPHEROID_WGS_84,
    xy_in, count, &error);

/* allocate space for result */
float *xy_out = malloc(2 * count * sizeof(float));

/* perform the shift */
error = pl_shift_datum(ctx, 
        PL_DATUM_NAD_83,  /* source */
        PL_DATUM_NAD_27,  /* destination */
        PL_SPHEROID_CLARKE_1866, /* destination spheroid */
        buf, xy_out);

pl_unload_datum_shift_data(buf);

Image warping

#include <projcl/projcl.h>
#include <projcl/projcl_warp.h>

void *src_image_data = ...;
void *dst_image_data = malloc(...);

PLProjectionParams *src_params = pl_params_init();
PLProjectionParams *dst_params = pl_params_init();
/* set projection parameters here... */


/* load the source image (raw pixel data) */
PLImageBuffer *image = pl_load_image(pl_ctx,
    CL_RGBA,            /* channel order */
    CL_UNSIGNED_INT8,   /* channel type */
    1024,               /* image width */
    1024,               /* image height */
    0,                  /* row pitch (0 to calculate automatically) */
    src_image_data,
    1,                  /* copy data? */
    &error);

/* load a grid of destination points to back-project to the source */
PLPointGridBuffer *xy_grid = pl_load_grid(ctx,
    0.0,   /* X-origin */
    100.0, /* width in projected coordinate space */
    1024,  /* width in pixels */
    0.0,   /* Y-origin */
    100.0, /* height in projected coordinate space */
    1024,  /* height in pixels */
    &error);

PLPointGridBuffer *geo_grid = pl_load_empty_grid(pl_ctx, 1024, 1024, &error);

/* project destination XY coordinates back to geo */
pl_project_grid_reverse(pl_ctx, PL_PROJECT_WINKEL_TRIPEL, dst_params, xy_grid, geo_grid);

/* project geo to source XY coordinates */
pl_project_grid_forward(pl_ctx, PL_PROJECT_MERCATOR, src_params, geo_grid, xy_grid);

/* now sample the points using your favorite filter */
pl_sample_image(pl_ctx, xy_grid, image, PL_IMAGE_FILTER_BICUBIC, dst_image_data);

See include/projcl/projcl_warp.h for a more complete discussion, including how to perform datum conversion on the point grid.

Forward geodesic: Fixed distance, multiple points, multiple angles (blast radii)

/* get the point data from somewhere */
float *xy_in = ...;
int xy_count = ...;

/* get the angle (azimuth) data from somewhere */
float *az_in = ...;
int az_count = ...;

/* load it up */
PLForwardGeodesicFixedDistanceBuffer *buf = pl_load_forward_geodesic_fixed_distance_data(ctx,
    xy_in, xy_count, az_in, az_count, &error);

/* allocate output buffer */
float *xy_out = malloc(2 * xy_count * az_count * sizeof(float));

/* compute */
error = pl_forward_geodesic_fixed_distance(ctx, buf, xy_out, PL_SPHEROID_SPHERE,
        1000.0 /* distance in meters */
        );

/* unload */
pl_unload_forward_geodesic_fixed_distance_data(buf);

Forward geodesic: Fixed angle, single point, multiple distances (great circle)

int count = ...;
float *dist_in = ...;

PLForwardGeodesicFixedAngleBuffer *buf = pl_load_forward_geodesic_fixed_angle_data(ctx,
    dist_in, count, &error);

float *xy_out = malloc(2 * count * sizeof(float));
float xy_in[2] = ...;

error = pl_forward_geodesic_fixed_angle(ctx, buf, xy_in, xy_out, PL_SPHEROID_SPHERE, 
        M_PI_2 /* angle in radians */
        );

pl_unload_forward_geodesic_fixed_angle_data(buf);

Inverse geodesic: Many-to-many (distance table)

int count1 = ...;
float *xy1_in = ...;

int count2 = ...;
float *xy2_in = ...;

float *dist_out = malloc(count1 * count2 * sizeof(float));

PLInverseGeodesicBuffer *buf = pl_load_inverse_geodesic_data(ctx, 
        xy1_in, count1, 1, xy2_in, count2, &error);

error = pl_inverse_geodesic(ctx, buf, dist_out, PL_SPHEROID_SPHERE, 
        1.0 /* scale */);

pl_unload_inverse_geodesic_data(buf);

Adding a Map Projection

It's relatively straightforward to add a map projection to ProjCL. You just need to...

  1. Create a file kernel/pl_project_<name>.opencl in kernel/ with the projection routines

  2. Create a pl_enqueue_kernel_<name> function in src/projcl_run.c

  3. Add an entry to the PLProjection enum in include/projcl/projcl_types.h

  4. Add an entry to the _pl_projection_info array in src/projcl_run.c

  5. Add tests to test/projcl_test.c

Some tips on writing OpenCL routines:

  • Use float16 arrays of 8 points for input and output
  • Use any and all for break conditions
  • Use select or the ternary operator for conditional assignments
  • Use sincos if you need the sine and cosine of the same angle
  • Double-angle and half-angle formulas are your friend
  • log(tan(M_PI_4 + 0.5*x)) == asinh(tan(x))
  • 2*atan(exp(x)) - M_PI_2 == asin(tanh(x))

projcl's People

Contributors

bobbane avatar evanmiller avatar hobu avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

projcl's Issues

Lambert Azimuthal Equal Area test failures

Consistency is OK but the Proj.4 comparison is failing

Testing consistency of Lambert Azimuthal Equal Area
-- Spherical, centered... ok
-- ...same as Proj.4... 1000 failures
**** Max longitudinal error: 1246421.500000 at (-4876150.000000, -0.000000)
**** Max latitudinal error: 1798909.500000 at (-3671586.500000, -3035005.000000)
-- Spherical, off-center... ok
-- ...same as Proj.4... 1000 failures
**** Max longitudinal error: 285615.750000 at (-3425188.000000, -5102897.000000)
**** Max latitudinal error: 900460.250000 at (-5062127.500000, -3530770.750000)
-- Ellipsoidal, centered... ok
-- ...same as Proj.4... 1000 failures
**** Max longitudinal error: 1240957.000000 at (-4881614.500000, -0.000000)
**** Max latitudinal error: 1813808.750000 at (-3656572.000000, -3044595.250000)
-- Ellipsoidal, off-center... ok
-- ...same as Proj.4... 1000 failures
**** Max longitudinal error: 292755.000000 at (-3458744.250000, -5063430.000000)
**** Max latitudinal error: 918456.500000 at (-5049309.000000, -3541705.000000)

What's up with spherical inverse Lambert Azimuthal Equal Area?

All of the ProjCL routines seem to be 4X - 20X faster than Proj.4, with the exception of the spherical inverse routine for Lambert Azimuthal Equal Area.

On OS X:

Testing consistency of Lambert Azimuthal Equal Area
-- Spherical, centered... ok
-- ...forward same as Proj.4... ok (8.9X faster)
-- ...inverse same as Proj.4... ok (81.1X faster)
-- Spherical, off-center... ok
-- ...forward same as Proj.4... ok (7.9X faster)
-- ...inverse same as Proj.4... ok (75.1X faster)

On Linux:

Testing consistency of Lambert Azimuthal Equal Area
-- Spherical, centered... ok
-- ...forward same as Proj.4... ok (6.1X faster)
-- ...inverse same as Proj.4... ok (0.9X faster)
-- Spherical, off-center... ok
-- ...forward same as Proj.4... ok (8.3X faster)
-- ...inverse same as Proj.4... ok (1.0X faster)

I.e. it's reported as 80X faster on OS X, and about the same speed as Proj.4 (1.0X) on Linux. It looks like a bug, but I haven't been able to figure it out. The Proj.4 code doesn't seem to do anything radically different from the ProjCL code.

Create command-line tools

Create (limited) drop-in replacements for Proj.4โ€˜s proj and geod command-line tools. It will need additional options for configuring the OpenCL device (and might need to drop the kernel files into /usr/share).

Support for double-precision calculations

(Following up on the discussion in #4, cc @BobBane)

I avoided double-precision in the past because Apple's OpenCL compiler was very buggy with doubles (crashing etc). The compiler quality has improved considerably since 2011, and my other OpenCL project (not open source, and not geo-related) uses double-precision exclusively now.

I see a few paths forward here:

  1. Migrate everything to double-precision

  2. Maintain separate kernels for single-precision and double-precision

  3. Use typedefs / #defines with a single set of kernels

I'm hesitant to adopt the #define KFLOAT float approach because the algorithms themselves may need to differ between single and double-precision. I.e. I use a lot of tricks to avoid round-off in the single-precision world that wouldn't be necessary with double-precision. Then there's the various tolerance levels and iteration counts that will need to differ between the two precisions, as well as the annoyance that literals must have an "f" specifier in the single-precision world (i.e. 0.5 has to be written 0.5f).

The only real reason to maintain single-precision is for applications that (strongly) prefer speed over accuracy, or to support GPUs that lack double-precision support. With Magic Maps, the slowdown might be an acceptable trade-off, but I won't really know until I try it out. So I'm hesitant to rip out the single-precision code willy-nilly.

For now I'm leaning toward a two-kernel world, implementing (porting) double-precision versions of projections as needed. I imagine an extra argument to pl_context_init would specify the desired computation precision โ€” which would later be passed to pl_find_kernel โ€” and I think that a wrapper function (or several) around clSetKernelArg should mean we won't have to duplicate many host-side functions. I'm envisioning two separate folders kernel/float/ and kernel/double/, with all OpenCL functions prefixed with either plf_ or pld_ depending on the precision. Generally only one set of kernels would be compiled for a given PLContext.

To keep a unified C API, I am fine with requiring all input/output buffers to be double-precision; for my application, copying matters much less than raw computation speed, so I'm okay with sending in and reading out only double and double *. So the user would only need to think about the precision choice when initializing the context, and thus could easily switch between computation precisions without having to rework all the client code.

What do you think?

Transverse Mercator failures

Testing consistency of Transverse Mercator
-- Spherical, centered... 4 failures
**** Max longitudinal error: 0.000004 at (44.999111, 0.282741)
**** Max latitudinal error: 0.000531 at (-44.999111, -0.282741)
-- Spherical, off-center... 50 failures
**** Max longitudinal error: 0.000011 at (-44.849968, 3.671577)
**** Max latitudinal error: 16.308537 at (44.255035, 8.154260)
-- Ellipsoidal, centered... 674 failures
**** Max longitudinal error: 0.087635 at (43.515068, 11.464683)
**** Max latitudinal error: 0.025311 at (43.133480, 12.825867)
-- Ellipsoidal, off-center... 652 failures
**** Max longitudinal error: 1.193104 at (-41.943199, -16.302391)
**** Max latitudinal error: 0.494332 at (-42.884071, -13.636587)

We likely need a better algorithm here. Charles Karney has published one as part of GeographicLib and described in this paper:

https://arxiv.org/pdf/1002.1417.pdf

From an email correspondence with Charles:

The Krueger series method is straightforward and presumably would be
easy to adapt to different platforms.  I don't know how OpenCL handles
branching, loops with a unknown number of iterations, etc.  Depending on
how this is done, you might, for example, want to replace the Newton's
method solution of the geographic latitude in terms of the conformal
latitude with a series example.

Ubuntu: Travis can't find CBLAS

CMake Error at /usr/share/cmake-3.2/Modules/FindPackageHandleStandardArgs.cmake:138 (message):
  Could NOT find CBLAS (missing: CBLAS_LIBRARIES)
Call Stack (most recent call first):
  /usr/share/cmake-3.2/Modules/FindPackageHandleStandardArgs.cmake:374 (_FPHSA_FAILURE_MESSAGE)
  cmake/modules/FindCBLAS.cmake:18 (find_package_handle_standard_args)
  CMakeLists.txt:180 (find_package)

Robinson test failures

Likely a scaling issue?

Testing consistency of Robinson
-- Spherical... ok
-- ...same as Proj.4... 1000 failures
**** Max longitudinal error: 4246032.500000 at (4246699.000000, 0.000000)
**** Max latitudinal error: 4798941.000000 at (0.000000, 4799694.500000)

Compilation warnings

/Users/emiller/Code/ProjCL/src/projcl.c:58:69: warning: incompatible pointer to integer
      conversion initializing 'cl_context_properties' (aka 'long') with an expression of
      type 'cl_platform_id' (aka 'struct _cl_platform_id *') [-Wint-conversion]
  ...context_properties[] = {CL_CONTEXT_PLATFORM, platform_id[PLATFORM_INDEX], NULL};
                                                  ^~~~~~~~~~~~~~~~~~~~~~~~~~~
/Users/emiller/Code/ProjCL/src/projcl.c:58:98: warning: incompatible pointer to integer
      conversion initializing 'cl_context_properties' (aka 'long') with an expression of
      type 'void *' [-Wint-conversion]
  ...context_properties[] = {CL_CONTEXT_PLATFORM, platform_id[PLATFORM_INDEX], NULL};
                                                                               ^~~~
/Users/emiller/Code/ProjCL/src/projcl.c:61:60: warning: incompatible pointer to integer
      conversion passing 'void *' to parameter of type 'cl_uint' (aka 'unsigned int')
      [-Wint-conversion]
        error = clGetDeviceIDs(platform_id[PLATFORM_INDEX], type, NULL, NULL, &num...
                                                                  ^~~~

American Polyconic test failures

For some reason, this error appears on Linux w/ Intel drivers, but not on Apple platforms.

Testing consistency of American Polyconic
-- Spherical, centered... 4 failures
**** Max longitudinal error: 0.000000 at (0.000000, 0.000000)
**** Max latitudinal error: 0.000527 at (44.999111, 0.282741)
-- Spherical, off-center... ok
-- Ellipsoidal, centered... 4 failures
**** Max longitudinal error: 0.000004 at (44.999111, 0.282741)
**** Max latitudinal error: 0.000530 at (44.999111, 0.282741)
-- Ellipsoidal, off-center... ok

Project status

I no longer need this library, and am putting it up for adoption. Please leave a comment here if you have any interest in maintaining it. Thanks!

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.