ossama-othman / marc Goto Github PK

The idea is to append something like the following to the end of each of C++ source file (*.{h,cpp}):

/*
  Local Variables:
  mode: c++
  c-basic-offset: 4
  indent-tabs-mode: nil
  End:
*/

MaRC's flex generated text scanner retains leading whitespace from the following input file entries:

• AUTHOR
• ORIGIN
• COMMENT
• XCOMMENT

The Flex rules for these MaRC input file keywords all leverage the same underlying rule, comment_init, with the goal of retaining whitespace between letters. However, that also includes leading and trailing whitespace, e.g. the space between the colon and the comment in "COMMENT: Foo". Leading and trailing whitespace should not be included in the matched text. In particular, leading and trailing whitespace should not be written to the map FITS files generated by MaRC.

MaRC's map grid plotting routines are fine for map projections where the grid lines are strictly horizontal or vertical but not very good otherwise. Leverage algorithms like Bresenham's Line Algorithm and the Midpoint Circle Algorithm, as well as Bezier curve algorithms to improve the quality of the map grids.

Where applicable, automatically configure the FITS World Coordinate System parameters in the generated map based on existing WCS parameters in the source photo FITS file(s). At the very least, MaRC should handle automatic use of the EQUINOX FITS WCS keyword in the generated map if it exists in the source photo FITS file. Ideally, other FITS WCS keywords such as RADESYS should also be handled.

Special handling may be needed for mosaic images since each photo in the mosaic could potentially have different FITS WCS keywords.

The automatic data scaling feature (#37) introduced in v0.9.9 incorrectly calculates the offset for data with a positive range of values.

For example, MaRC uses a longitude data range of [0, 360]. The scale for LONGITUDE maps is calculated correctly but the offset is not. Assuming a map with 16 bit signed integer data is being plotted, the maximum value is 32767. The LONGITUDE map data will be scaled by 100 with an offset of 0, making the maximum plotted data value 36000, which is larger than the largest 16 bit signed integer 32767. The result is longitude values greater than 327 degrees end up being dropped from the map. The offset value should have been -18000, for example, to shift the map value into the chosen integer range of values, i.e. [0, 360] scaled by 100 and offset -18000 results in map data in the range [-18000, 18000] which is well within the 16 bit signed integer data range [-32768, 32767]. In this case, the corresponding FITS BSCALE and BZERO values needed to convert from the map value to the physical value would be 0.01 and 180, respectively.

This problem only occurs when creating a LONGITUDE integer typed map plane containing scaled longitude values larger than a certain value (e.g. larger than 32767 in 16 bit signed integer maps). Floating point maps don't exhibit this problem.

MaRC::root_find() as implemented is not suitable for all cases. It suffers from a few deficiencies:

• The step value h used when numerically computing the first derivative of the supplied function may not be suitable for all cases. An attempt should be made to automate selection of that step value. See Section 5.7 in the book "Numerical Recipes in C".
• There is a potential division by zero in the Newton-Raphson implementation if the computed derivative is zero.
• The search bracket used after the first attempt at finding the root fails seems rather large. Leverage something like a hybrid bisection / Newton-Raphson approach to address this problem. The hybrid approach would also address the fixed step value used during the search. See Section 9.4 of "Numerical Recipes in C".

These issues should be addressed so that other code in MaRC that call MaRC::root_find(), such as the conformal projections, work reliably in all cases.

The MaRC shared library (libmarc.so*) should enable and support Libtool based versioning to prevent incompatible versions of the library from being dynamically linked with binaries that use it.

The goal is to support proper library versioning, which means that the "version" in the library filename (e.g. libMaRC.so.1.0.1) may not match MaRC package version. The two versions should not be forced to match (e.g. via Libtool release numbers).

The FITS standard supports 64 bit signed integers (BITPIX value of 64) but MaRC does not yet support that data type. It should be straightforward to implement support for 64 bit signed integers. CFITSIO, the underlying FITS library MaRC currently uses, already supports 64 bit signed integers.

Use of unsigned integers such as std::size_t for array indices is error prone. Consistently use signed integers instead and assert() if the array index is invalid instead.

See Eric Neibler's comment on array indices for a simple explanation.

Calculations in MaRC's Transverse Mercator projection assume that the poles (+/- 90 degrees) may be mapped. That is incorrect since the distortion in that projection approaches infinity at the poles. A user-configurable maximum latitude should be supported for the Mercator projection in a way similar to the Polar Stereographic projection does that.

Furthermore, the xmax calculation in the MaRC::Mercator::plot_map() and plot_grid() methods seem to assume that xmax is a latitude. It is not. The MaRC::Mercator::mercator_x() should be used to determine xmax at the maximum configured latitude range. Determine whether or not the code needs to be corrected.

Create a set of MaRC packages for Debian based distributions.

Add support for generation of Doxygen based documentation for MaRC.

MaRC ends up leaving behind an incomplete map output file when an error occurs. Consider preventing the file from being created, or at least deleting it, if an error occurs.

The simple approach to comparing floating point numbers using the standard comparison operators (e.g, <, ==, etc) generally isn't the best way to compare floating point numbers. MaRC should leverage some of the more robust floating point comparison techniques described by Knuth, for example. See the blog post "Comparing Floating Point Numbers, 2012 Edition" to get additional ideas of how to best address potential floating point issues in MaRC.

When generating a FLOAT data type map with a latitude data plane, the latitude data plane ends up full of NaN values. The longitude plane is fine, however.

The MaRC library should not dump progress to the console. Rather, the MaRC program should be responsible for that. The library should instead provide a means for the program to track progress.

The MaRC::PhotoImageFactory::invert_h() and MaRC::PhotoImageFactory::invert_v() methods do not function as expected. Horizontal (left to right) inversion is essentially a no-op, and vertical (top-to-bottom) inversion incorrectly shuffles some of the photo image array elements. In either case, no inversion doesn't occur as expected.

This bug is triggered when using the INVERT MaRC input file keyword.

The Orthographic map outputs the body center in the map after each map plane completes plotting. It should only be output once per map rather than map plane.

Performance between v0.9.8 and v0.9.9 dropped by at least 15% in some cases. It's likely that the floating point robustness improvements may be at fault. If that is the case, determine if it is possible to improve performance without sacrificing the improvements.

MaRC's existing unit tests follow a simple pattern where the boolean result of all test cases in a given unit test program are combined into a single overall pass/fail result. That is fine for automated test runs, but it isn't ideal for debugging purposes since more work is needed to pinpoint exactly which test case in a given unit test program has failed.

Leverage the Boost.Test framework to improve MaRC's unit tests.

MaRC should support GCC symbol visibility to gain the accompanying size and performance benefits, as well to start explicitly marking what's part of the public/exported MaRC library API.

When mapping photos with known resolution (e.g. kilometers per pixel) MaRC should be able to automatically choose the appropriate map dimensions that retain or do not decrease the source photo resolution.

Special care must be taken when mapping a mosaic since the source photos that make up the mosaic may each potentially have different resolutions (not ideal!).

Formalize logging in MaRC. MaRC currently has various std::cout and std::cerr statements peppered throughout itself. Use a logging framework like Boost.Log to making logging in MaRC much more robust and configurable.

On platforms where concurrent execution of tasks makes sense, such as such multi-core CPU or hyper-threaded platforms, mapping operations in MaRC can be partitioned and executed concurrently to improve performance. For example, one thread could map the top half of map plane, while another could map the bottom half.

MaRC's input file parser and scanner are generated by GNU Bison and Flex, respectively. Look into replacing MaRC's parser and scanner with a Boost.Spirit based one instead.

If the source photo FITS file contains a BLANK card/value use that same value for the generated map if the map data type is also a comparable integer. Special handling may be needed for the mosaic case since multiple potentially different BLANK values could exist.

If the BLANK keyword doesn't exist, a value chosen using a procedure that prioritizes speed like the following:

1. Choose a value outside of the [DATAMIN, DATAMAX] range in the FITS file, ideally less than DATAMIN.
2. If neither of those values exist, scan the image array for minimum and maximum array values, and choose a BLANK outside of that range.
3. If the array scan doesn't provide a suitable result, e.g. if the scanned minimum and maximum array values are the same as the image data type minimum and maximum, fall back on a scan with the sky masked out of the image. This step can be time intensive.

Automatically determine best map plane and data scale and offset based on the map data type and plane type. To reduce confusion, the scale should probably be a power of 10 (e.g. 10, 100, 1000, etc), and the offset should be zero for all map data types except BYTE since an offset may be necessary to support negative source image data values.

The goal is to automatically determine the best source image data scale and offset parameters to retain as many significant digits as possible when storing that data in integer typed maps. It is is currently intended for use with MaRC virtual images, such as cosine of emission angle images (MaRC::MuImage), latitude images (MaRC::LatitudeImage), etc. This would allow for the removal of the hard-coded scale and offset values in MuImage, Mu0Image and CosPhaseImage implementations.

The computed scale and offset should not override MaRC DATA_SCALE and DATA_OFFSET input file value, and should instead be adjusted to work with those values.

The map FITS header should also document the computed scale and offset. If virtual images with the same data range (e.g. [-1,1]) are the only the ones mapped, the FITS BSCALE and BZERO keyword values should be set. The FITS BUNIT keyword value may also be set in that case. Otherwise, document the scale and offset needed to convert the virtual image map values in a set of FITS HISTORY entries.

MaRC implements the well known form of Quadratic Formula in several places in the trivial way that most would expect. However, that implementation is subject to loss of significant digits, resulting in at least one potentially inaccurate root.

MaRC should instead implement the form of the Quadratic Formula described in Section 5.6: "Quadratic and Cubic Equations" in "Numerical Recipes in C", 1992, by Press, Teukolsky, Veterrling and Flannery. It isn't subject to the catastrophic cancellation that could occur when calculating the root that requires a subtraction in the Quadratic Formula.

Create a set of MaRC Homebrew packages, e.g. for MacOS.

Automatically calculate the optimal latitude / longitude range for a given map to display the most amount of data while retaining the properties of the map.

The MaRC::Matrix and MaRC::Vector equality operator, i.e. operator==() for both classes does not correctly support floating point types. They both default to the trivial exact check for equality between each matrix/vector element, instead of checking if the value of each floating point element is almost equal. Rather than use std::equal_to(), MaRC::almost_equal() and MaRC::almost_zero() should be use for floating point typed matrices and vectors. Overloads of the latter two for MaRC::Matrix and MaRC::Vector would be appropriate.

The angle units in MaRC::ViewingGeometry methods that accept an angle, such as a latitude, are not consistent. For example, the sub_observ() method expects degrees but the is_visible() method expects radians. This error prone. Angle units should be all be consistent, preferrably in radians, to minimize the need to convert while performing mapping operations.

A number of conventions are used that allow the coordinate system used in FITS files to be well defined. MaRC should also support those conventions. See the FITS WCS page for additional information.

The MaRC input file grammar currently requires unnecessary information for the MU and MU0 map planes. For example, the sub-observation latitude and longitude, range, and sub-solar latitude and longitude are all required by the input file grammar for the MU map plane but only the sub-observation latitude and longitude, and range are required. Similarly, only the sub-solar latitude and longitude are required for the MU0 map plane. MaRC should not require the user to add unnecessary information to the input file.

Support for specifying the latitude and longitude at the center of a photo exists in MaRC but has been disabled for years due to a bug that was not resolved. Recent fixes to the MaRC::Vector<> may have resolved the problem. Verify that the MaRC::PhotoImage::rot_matrices() method used when the latitude and longitude at the optical axis was given works with the MaRC::Vector<> fix in place. Enable the code if it does.

Currently the math behind MaRC is a bit of a black box. All math used by MaRC, such as coordinate transformations, map equations, etc, should be documented.

Maps are implemented as class templates to support different map data types. However, the bulk of the map operations and calculations are type independent. Those operations should be moved to a base class or implementation class member to substantially reduce the size of the marc program.

The MaRC configure script currently checks for the CFITSIO library through a simple autoconf library search (e.g. AC_SEARCH_LIBS([ffinit],[cfitsio], ...). It may be more suitable to leverage pkg-config to pull in additional build flags CFITSIO may need. One problem with this proposed change is that it adds a dependency on pkg-config.

See the attached patch to get an idea of the changes that would be necessary to support.
pkg-config.diff.txt

An unexpected bright band of data was reported to exist in a map generated by MaRC in a MacOS build. Look for potential memory corruption.

Leverage Coverity integration with Travis CI to run Coverity static analysis builds of MaRC.

MaRC has enough information to automatically add COMMENT fields in the map FITS header that describes each map plane, such as type of plane, plane data scaling/offset factors, etc. Similar things can be done with the map grid extension XCOMMENT fields, including latitude/longitude grid line intervals, bounds, etc. Generate such COMMENT and XCOMMENT fields automatically.

Leverage GitHub's Travis CI integration to automatically build each MaRC push to GitHub.

Leverage font rasterization libraries, like FreeType, to add text to MaRC grid images so that at each grid line may be identified, e.g. 45 N.

Bear in mind license compatibility of such 3rd party libraries with MaRC's current LGPLv2.1 and GPLv2 licenses. For example, FreeType's license is not compatible with GPLv2.

The MaRC::OblateSpheroid constructor currently accepts three floating point parameters:

• equatorial radius
• polar radius
• flattening

where any two of the three are required to be valid (positive). The third will be computed from the two that are supplied. This can be confusing and error prone. Look into altering the constructor so that only two radii/flattening parameters are required.

An "Invalid read of size 8" is triggered when running valgrind on MaRC when generating maps of type LONG. To reproduce, running the following In the MaRC source tree:

$ cd tests
$ ../libtool --mode=execute valgrind --leak-check=full ../src/marc test_map.inp
...
Creating map: test_map.fits
Plane 1 / 9 : .........20.........40.........60.........80.........100%
==31501== Invalid read of size 8
==31501==    at 0x51059B0: ffi4fi4 (in /usr/lib/x86_64-linux-gnu/libcfitsio.so.5.3.41)
==31501==    by 0x51061B9: ??? (in /usr/lib/x86_64-linux-gnu/libcfitsio.so.5.3.41)
==31501==    by 0x5106422: ffpprj (in /usr/lib/x86_64-linux-gnu/libcfitsio.so.5.3.41)
==31501==    by 0x50FC499: ffppr (in /usr/lib/x86_64-linux-gnu/libcfitsio.so.5.3.41)
==31501==    by 0x11EC10: MaRC::MapCommand_T<int>::make_map_planes(fitsfile*, int&) (MapCommand_T.cpp:139)
==31501==    by 0x113019: MaRC::MapCommand::execute() (MapCommand.cpp:195)
==31501==    by 0x10FB21: main (marc.cpp:147)
==31501==  Address 0x683d888 is 0 bytes after a block of size 5,000 alloc'd
==31501==    at 0x4C2E19F: operator new(unsigned long) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31501==    by 0x11EB91: allocate (new_allocator.h:104)
==31501==    by 0x11EB91: allocate (alloc_traits.h:436)
==31501==    by 0x11EB91: _M_allocate (stl_vector.h:170)
==31501==    by 0x11EB91: _M_create_storage (stl_vector.h:185)
==31501==    by 0x11EB91: _Vector_base (stl_vector.h:136)
==31501==    by 0x11EB91: vector (stl_vector.h:293)
==31501==    by 0x11EB91: make_map (MapFactory.cpp:51)
==31501==    by 0x11EB91: MaRC::MapCommand_T<int>::make_map_planes(fitsfile*, int&) (MapCommand_T.cpp:129)
==31501==    by 0x113019: MaRC::MapCommand::execute() (MapCommand.cpp:195)
==31501==    by 0x10FB21: main (marc.cpp:147)
==31501== 

The invalid read error occurs during the fits_write_image() call, and only occurs when writing FITS images of type LONG. No such error occurs when changing the DATA_TYPE in the test MaRC input file to any of the other types.

Many of the source files in MaRC are missing a license header. Clarify which files are under the GPL, and which are under the LGPL by using the appropriate license header.

The MaRC::ViewingGeometry class (formerly part of MaRC::PhotoImage) expects all supplied longitudes, such as the sub-observation and sub-solar longitudes, to be West longitudes, regardless of the body rotation direction. This is confusing since the typically followed convention is to use West longitudes for bodies with a prograde rotation, and East longitudes for those with retrograde rotation.
Ideally, the supplied longitudes should follow that convention. Doing so would allow for removal of the body rotation checks in MaRC::ViewingGeometry implementation.

Convenience wrappers, such as MaRC::EastLongitude and MaRC::WestLongitude, could be provided to prevent the user from inadvertently supplying an East longitude when a West longitude was expected, or vice versa.

MaRC should automatically choose the map data type based on the data type in the source photo FITS file if a type is not explicitly chosen by the user. For example, if a source photo is comprised of 16 bit signed integers (FITS "short"), the generated map should also contain 16 bit signed integers.

When converting between body and observer (camera) coordinates in viewing geometry related code, longitudes are shifted to take into account the sub-observation longitude. For example, the following code exists in MaRC::ViewingGeometry::latlon2pix():

    if (this->body_->prograde())
        lon  = this->sub_observ_lon_ - lon;
    else
        lon -= this->sub_observ_lon_;

Since this is basically just a rotation about the Z-axis, it should be possible incorporate this rotation in to the rotation matrices used when converting between body and observer coordinates. Doing so would further simplify such lat/lon-pixel conversion code.