GithubHelp home page GithubHelp logo

futurefire's People

Contributors

smnorris avatar

Stargazers

avatar

Watchers

avatar avatar avatar

futurefire's Issues

parallel processing

Processing is relatively slow - ~40min to run through all years for a single region/draw/scenario on a 2.9/4.8GHZ laptop.
Process draw/scenario combinations in parallel - years need to be done sequentially to handle regen.

endless loop

A fire of only 1513 ha should be easy to place. Is the fetch of a new ignition point not working correctly, or is something unexpected happening in the data?

2019-03-29 16:08:32,338 futurefire.main INFO     Processing - runid:58 year:2068 regionid:3 burnid:362369 target_area:1513
2019-03-29 16:08:32,388 futurefire.main INFO     N expansions = 101 - runid:58 year:2068 regionid:3 burnid:362369 target_area:1513
2019-03-29 16:08:33,265 futurefire.main INFO     N expansions = 501 - runid:58 year:2068 regionid:3 burnid:362369 target_area:1513
2019-03-29 16:08:37,321 futurefire.main INFO     More than 1000 expansions - runid:58 year:2068 regionid:3 burnid:362369 target_area:1513
2019-03-29 16:08:37,375 futurefire.main INFO     N expansions = 101 - runid:58 year:2068 regionid:3 burnid:362369 target_area:1513
2019-03-29 16:08:38,257 futurefire.main INFO     N expansions = 501 - runid:58 year:2068 regionid:3 burnid:362369 target_area:1513
2019-03-29 16:08:43,260 futurefire.main INFO     More than 1000 expansions - runid:58 year:2068 regionid:3 burnid:362369 target_area:1513
2019-03-29 16:08:43,332 futurefire.main INFO     N expansions = 101 - runid:58 year:2068 regionid:3 burnid:362369 target_area:1513
2019-03-29 16:08:44,230 futurefire.main INFO     N expansions = 501 - runid:58 year:2068 regionid:3 burnid:362369 target_area:1513
2019-03-29 16:08:50,161 futurefire.main INFO     More than 1000 expansions - runid:58 year:2068 regionid:3 burnid:362369 target_area:1513
2019-03-29 16:08:50,233 futurefire.main INFO     N expansions = 101 - runid:58 year:2068 regionid:3 burnid:362369 target_area:1513
2019-03-29 16:08:51,343 futurefire.main INFO     N expansions = 501 - runid:58 year:2068 regionid:3 burnid:362369 target_area:1513
2019-03-29 16:08:58,468 futurefire.main INFO     More than 1000 expansions - runid:58 year:2068 regionid:3 burnid:362369 target_area:1513
2019-03-29 16:08:58,612 futurefire.main INFO     N expansions = 101 - runid:58 year:2068 regionid:3 burnid:362369 target_area:1513
2019-03-29 16:08:59,495 futurefire.main INFO     N expansions = 501 - runid:58 year:2068 regionid:3 burnid:362369 target_area:1513
2019-03-29 16:09:04,562 futurefire.main INFO     More than 1000 expansions - runid:58 year:2068 regionid:3 burnid:362369 target_area:1513
2019-03-29 16:09:04,635 futurefire.main INFO     N expansions = 101 - runid:58 year:2068 regionid:3 burnid:362369 target_area:1513
2019-03-29 16:09:05,821 futurefire.main INFO     N expansions = 501 - runid:58 year:2068 regionid:3 burnid:362369 target_area:1513
2019-03-29 16:09:09,799 futurefire.main INFO     More than 1000 expansions - runid:58 year:2068 regionid:3 burnid:362369 target_area:1513
2019-03-29 16:09:09,852 futurefire.main INFO     N expansions = 101 - runid:58 year:2068 regionid:3 burnid:362369 target_area:1513
2019-03-29 16:09:10,937 futurefire.main INFO     N expansions = 501 - runid:58 year:2068 regionid:3 burnid:362369 target_area:1513
2019-03-29 16:09:14,428 futurefire.main INFO     More than 1000 expansions - runid:58 year:2068 regionid:3 burnid:362369 target_area:1513
2019-03-29 16:09:14,482 futurefire.main INFO     N expansions = 101 - runid:58 year:2068 regionid:3 burnid:362369 target_area:1513
2019-03-29 16:09:15,353 futurefire.main INFO     N expansions = 501 - runid:58 year:2068 regionid:3 burnid:362369 target_area:1513
2019-03-29 16:09:18,931 futurefire.main INFO     More than 1000 expansions - runid:58 year:2068 regionid:3 burnid:362369 target_area:1513
2019-03-29 16:09:18,986 futurefire.main INFO     N expansions = 101 - runid:58 year:2068 regionid:3 burnid:362369 target_area:1513
2019-03-29 16:09:19,863 futurefire.main INFO     N expansions = 501 - runid:58 year:2068 regionid:3 burnid:362369 target_area:1513
2019-03-29 16:09:23,297 futurefire.main INFO     More than 1000 expansions - runid:58 year:2068 regionid:3 burnid:362369 target_area:1513
2019-03-29 16:09:23,349 futurefire.main INFO     N expansions = 101 - runid:58 year:2068 regionid:3 burnid:362369 target_area:1513
2019-03-29 16:09:24,263 futurefire.main INFO     N expansions = 501 - runid:58 year:2068 regionid:3 burnid:362369 target_area:1513
2019-03-29 16:09:28,641 futurefire.main INFO     More than 1000 expansions - runid:58 year:2068 regionid:3 burnid:362369 target_area:1513
2019-03-29 16:09:28,695 futurefire.main INFO     N expansions = 101 - runid:58 year:2068 regionid:3 burnid:362369 target_area:1513
2019-03-29 16:09:29,781 futurefire.main INFO     N expansions = 501 - runid:58 year:2068 regionid:3 burnid:362369 target_area:1513
2019-03-29 16:09:33,824 futurefire.main INFO     More than 1000 expansions - runid:58 year:2068 regionid:3 burnid:362369 target_area:1513
2019-03-29 16:09:33,878 futurefire.main INFO     N expansions = 101 - runid:58 year:2068 regionid:3 burnid:362369 target_area:1513
2019-03-29 16:09:34,784 futurefire.main INFO     N expansions = 501 - runid:58 year:2068 regionid:3 burnid:362369 target_area:1513

convert outputs to gcbm standard

Warp output rasters to EPSG:4326, add template raster or its profile/bounds etc to config. Consider adding template tiff to repository.

retry fires that don't have space to burn

If a fire cannot meet target area within 1000 expansions, don't just bail - try a new spot to burn.

2019-03-26 07:08:26,480 futurefire.main INFO     Processing fires for region Coast, year 2025
Traceback (most recent call last):
  File "/Users/snorris/miniconda3/envs/futurefire/bin/futurefire", line 11, in <module>
    load_entry_point('futurefire', 'console_scripts', 'futurefire')()
  File "/Users/snorris/miniconda3/envs/futurefire/lib/python3.7/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/Users/snorris/miniconda3/envs/futurefire/lib/python3.7/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/Users/snorris/miniconda3/envs/futurefire/lib/python3.7/site-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/snorris/miniconda3/envs/futurefire/lib/python3.7/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/snorris/miniconda3/envs/futurefire/lib/python3.7/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/Users/snorris/Projects/cat/2019_wildfire/futurefire/futurefire/cli.py", line 268, in burn
    fires, forest_reg, forest_prov, burn_image, runid, region, year, n=n
  File "/Users/snorris/Projects/cat/2019_wildfire/futurefire/futurefire/main.py", line 158, in apply_fires
    raise RuntimeError("Cannot meet target area")
RuntimeError: Cannot meet target area

process all regions for a given run/year

Modify the loop to process and combine all regions for a given timestep.
Post processing may be easy but this will keep things simpler and make for less disk space overhead.

logfile naming

For easier management when logs are coming from multiple instances, name the file based on time started or something else distinct

supplying config makes fire_ellipse_pct_growth fail

The command works fine when no config file provided.
It might be simpler to abandon the default python config and require the ini file - maintaining parameters in only 1 spot.

$ time futurefire burn inputs/lowscenario.csv --runid 1 --region Coast -c sample_config.cfg
2019-03-19 21:39:43,543 futurefire.util INFO     Loading config from file: sample_config.cfg
2019-03-19 21:39:48,237 futurefire.cli INFO     Processing region Coast
2019-03-19 21:39:49,585 futurefire.cli INFO     Processing runid 1
2019-03-19 21:39:53,209 futurefire.main INFO     Processing fires for year 2020
Traceback (most recent call last):
  File "/Users/snorris/miniconda3/envs/futurefire/bin/futurefire", line 11, in <module>
    load_entry_point('futurefire', 'console_scripts', 'futurefire')()
  File "/Users/snorris/miniconda3/envs/futurefire/lib/python3.7/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/Users/snorris/miniconda3/envs/futurefire/lib/python3.7/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/Users/snorris/miniconda3/envs/futurefire/lib/python3.7/site-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/snorris/miniconda3/envs/futurefire/lib/python3.7/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/snorris/miniconda3/envs/futurefire/lib/python3.7/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/Users/snorris/Projects/cat/2019_wildfire/futurefire/futurefire/cli.py", line 231, in burn
    forest_current, burn_image, burn_list = futurefire.apply_fires(fires, forest_current, burn_image, year, n=n)
  File "/Users/snorris/Projects/cat/2019_wildfire/futurefire/futurefire/main.py", line 114, in apply_fires
    increment = (config["fire_ellipse_pct_growth"] * .01) * target_area
TypeError: can't multiply sequence by non-int of type 'float'

config file

[CONFIG]

wksp                    = wksp
inputs_gdb              = inputs/future_fire_2019_01_24.gdb
roads                   = roads
inventory               = inventory
regions                 = inputs/regions.tif

# bounds are  "xmin ymin xmax ymax" in target area preserving projection
# (EPSG:3005, BC Albers)
bounds                  = 159587.5 173787.5 1881187.5 1748187.5

buffer                  = 500
cell_size               = 100
fire_axis_ratio_min     = 20
fire_axis_ratio_max     = 80

# increasing the pct growth reduces iterations and processing time
fire_ellipse_pct_growth = 2

regen                   = 10
outputs                 = outputs
log_file                = futurefire.log


# match the region names in scenario file to regions.tif raster values
[REGION_LOOKUP]
Coast = 1
Northern Interior = 2
Southern Interior = 3

performance

The job takes a long time to run - there are likely substantial speedups possible by optimizing code.

Expanding the ellipse to meet a fixed area takes a long time. Looping over arrays is a known bottleneck. It may be possible to adapt the ellipse function to handle expansion rather than looping until target met? If so, perhaps all ignition points could be generated in one step, then expansion applied in a single step. This requires some better familiarity with numpy.

regen

Burns don't seem to be burning forest that has had regen years to recover. Add a test to check that this is happening

csv creation

csv is being overwritten for each draw, place it in the output for the given draw

salvage

Generate and export salvage data

input rasters are not quite aligned 

IndexError: boolean index did not match indexed array along dimension 0; dimension is 15745 but corresponding boolean dimension is 15744
(futurefire)

>>> regions.shape
(15744, 17216)
>>> forest.shape
(15745, 17217)

allow for supplying probability raster

Supplying a fire ignition probability raster is straightforward, but it looks like we have to modify the randomization to use a random choice for every fire rather than shuffling forest and picking from the top of the 'deck'.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.