smnorris / futurefire Goto Github PK
View Code? Open in Web Editor NEWRandomly generate burned areas for future fire scenarios
License: Apache License 2.0
Randomly generate burned areas for future fire scenarios
License: Apache License 2.0
Processing is relatively slow - ~40min to run through all years for a single region/draw/scenario on a 2.9/4.8GHZ laptop.
Process draw/scenario combinations in parallel - years need to be done sequentially to handle regen.
A fire of only 1513 ha should be easy to place. Is the fetch of a new ignition point not working correctly, or is something unexpected happening in the data?
2019-03-29 16:08:32,338 futurefire.main INFO Processing - runid:58 year:2068 regionid:3 burnid:362369 target_area:1513
2019-03-29 16:08:32,388 futurefire.main INFO N expansions = 101 - runid:58 year:2068 regionid:3 burnid:362369 target_area:1513
2019-03-29 16:08:33,265 futurefire.main INFO N expansions = 501 - runid:58 year:2068 regionid:3 burnid:362369 target_area:1513
2019-03-29 16:08:37,321 futurefire.main INFO More than 1000 expansions - runid:58 year:2068 regionid:3 burnid:362369 target_area:1513
2019-03-29 16:08:37,375 futurefire.main INFO N expansions = 101 - runid:58 year:2068 regionid:3 burnid:362369 target_area:1513
2019-03-29 16:08:38,257 futurefire.main INFO N expansions = 501 - runid:58 year:2068 regionid:3 burnid:362369 target_area:1513
2019-03-29 16:08:43,260 futurefire.main INFO More than 1000 expansions - runid:58 year:2068 regionid:3 burnid:362369 target_area:1513
2019-03-29 16:08:43,332 futurefire.main INFO N expansions = 101 - runid:58 year:2068 regionid:3 burnid:362369 target_area:1513
2019-03-29 16:08:44,230 futurefire.main INFO N expansions = 501 - runid:58 year:2068 regionid:3 burnid:362369 target_area:1513
2019-03-29 16:08:50,161 futurefire.main INFO More than 1000 expansions - runid:58 year:2068 regionid:3 burnid:362369 target_area:1513
2019-03-29 16:08:50,233 futurefire.main INFO N expansions = 101 - runid:58 year:2068 regionid:3 burnid:362369 target_area:1513
2019-03-29 16:08:51,343 futurefire.main INFO N expansions = 501 - runid:58 year:2068 regionid:3 burnid:362369 target_area:1513
2019-03-29 16:08:58,468 futurefire.main INFO More than 1000 expansions - runid:58 year:2068 regionid:3 burnid:362369 target_area:1513
2019-03-29 16:08:58,612 futurefire.main INFO N expansions = 101 - runid:58 year:2068 regionid:3 burnid:362369 target_area:1513
2019-03-29 16:08:59,495 futurefire.main INFO N expansions = 501 - runid:58 year:2068 regionid:3 burnid:362369 target_area:1513
2019-03-29 16:09:04,562 futurefire.main INFO More than 1000 expansions - runid:58 year:2068 regionid:3 burnid:362369 target_area:1513
2019-03-29 16:09:04,635 futurefire.main INFO N expansions = 101 - runid:58 year:2068 regionid:3 burnid:362369 target_area:1513
2019-03-29 16:09:05,821 futurefire.main INFO N expansions = 501 - runid:58 year:2068 regionid:3 burnid:362369 target_area:1513
2019-03-29 16:09:09,799 futurefire.main INFO More than 1000 expansions - runid:58 year:2068 regionid:3 burnid:362369 target_area:1513
2019-03-29 16:09:09,852 futurefire.main INFO N expansions = 101 - runid:58 year:2068 regionid:3 burnid:362369 target_area:1513
2019-03-29 16:09:10,937 futurefire.main INFO N expansions = 501 - runid:58 year:2068 regionid:3 burnid:362369 target_area:1513
2019-03-29 16:09:14,428 futurefire.main INFO More than 1000 expansions - runid:58 year:2068 regionid:3 burnid:362369 target_area:1513
2019-03-29 16:09:14,482 futurefire.main INFO N expansions = 101 - runid:58 year:2068 regionid:3 burnid:362369 target_area:1513
2019-03-29 16:09:15,353 futurefire.main INFO N expansions = 501 - runid:58 year:2068 regionid:3 burnid:362369 target_area:1513
2019-03-29 16:09:18,931 futurefire.main INFO More than 1000 expansions - runid:58 year:2068 regionid:3 burnid:362369 target_area:1513
2019-03-29 16:09:18,986 futurefire.main INFO N expansions = 101 - runid:58 year:2068 regionid:3 burnid:362369 target_area:1513
2019-03-29 16:09:19,863 futurefire.main INFO N expansions = 501 - runid:58 year:2068 regionid:3 burnid:362369 target_area:1513
2019-03-29 16:09:23,297 futurefire.main INFO More than 1000 expansions - runid:58 year:2068 regionid:3 burnid:362369 target_area:1513
2019-03-29 16:09:23,349 futurefire.main INFO N expansions = 101 - runid:58 year:2068 regionid:3 burnid:362369 target_area:1513
2019-03-29 16:09:24,263 futurefire.main INFO N expansions = 501 - runid:58 year:2068 regionid:3 burnid:362369 target_area:1513
2019-03-29 16:09:28,641 futurefire.main INFO More than 1000 expansions - runid:58 year:2068 regionid:3 burnid:362369 target_area:1513
2019-03-29 16:09:28,695 futurefire.main INFO N expansions = 101 - runid:58 year:2068 regionid:3 burnid:362369 target_area:1513
2019-03-29 16:09:29,781 futurefire.main INFO N expansions = 501 - runid:58 year:2068 regionid:3 burnid:362369 target_area:1513
2019-03-29 16:09:33,824 futurefire.main INFO More than 1000 expansions - runid:58 year:2068 regionid:3 burnid:362369 target_area:1513
2019-03-29 16:09:33,878 futurefire.main INFO N expansions = 101 - runid:58 year:2068 regionid:3 burnid:362369 target_area:1513
2019-03-29 16:09:34,784 futurefire.main INFO N expansions = 501 - runid:58 year:2068 regionid:3 burnid:362369 target_area:1513
Warp output rasters to EPSG:4326, add template raster or its profile/bounds etc to config. Consider adding template tiff to repository.
If a fire cannot meet target area within 1000 expansions, don't just bail - try a new spot to burn.
2019-03-26 07:08:26,480 futurefire.main INFO Processing fires for region Coast, year 2025
Traceback (most recent call last):
File "/Users/snorris/miniconda3/envs/futurefire/bin/futurefire", line 11, in <module>
load_entry_point('futurefire', 'console_scripts', 'futurefire')()
File "/Users/snorris/miniconda3/envs/futurefire/lib/python3.7/site-packages/click/core.py", line 764, in __call__
return self.main(*args, **kwargs)
File "/Users/snorris/miniconda3/envs/futurefire/lib/python3.7/site-packages/click/core.py", line 717, in main
rv = self.invoke(ctx)
File "/Users/snorris/miniconda3/envs/futurefire/lib/python3.7/site-packages/click/core.py", line 1137, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/Users/snorris/miniconda3/envs/futurefire/lib/python3.7/site-packages/click/core.py", line 956, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/Users/snorris/miniconda3/envs/futurefire/lib/python3.7/site-packages/click/core.py", line 555, in invoke
return callback(*args, **kwargs)
File "/Users/snorris/Projects/cat/2019_wildfire/futurefire/futurefire/cli.py", line 268, in burn
fires, forest_reg, forest_prov, burn_image, runid, region, year, n=n
File "/Users/snorris/Projects/cat/2019_wildfire/futurefire/futurefire/main.py", line 158, in apply_fires
raise RuntimeError("Cannot meet target area")
RuntimeError: Cannot meet target area
Modify the loop to process and combine all regions for a given timestep.
Post processing may be easy but this will keep things simpler and make for less disk space overhead.
For easier management when logs are coming from multiple instances, name the file based on time started or something else distinct
The command works fine when no config file provided.
It might be simpler to abandon the default python config and require the ini file - maintaining parameters in only 1 spot.
$ time futurefire burn inputs/lowscenario.csv --runid 1 --region Coast -c sample_config.cfg
2019-03-19 21:39:43,543 futurefire.util INFO Loading config from file: sample_config.cfg
2019-03-19 21:39:48,237 futurefire.cli INFO Processing region Coast
2019-03-19 21:39:49,585 futurefire.cli INFO Processing runid 1
2019-03-19 21:39:53,209 futurefire.main INFO Processing fires for year 2020
Traceback (most recent call last):
File "/Users/snorris/miniconda3/envs/futurefire/bin/futurefire", line 11, in <module>
load_entry_point('futurefire', 'console_scripts', 'futurefire')()
File "/Users/snorris/miniconda3/envs/futurefire/lib/python3.7/site-packages/click/core.py", line 764, in __call__
return self.main(*args, **kwargs)
File "/Users/snorris/miniconda3/envs/futurefire/lib/python3.7/site-packages/click/core.py", line 717, in main
rv = self.invoke(ctx)
File "/Users/snorris/miniconda3/envs/futurefire/lib/python3.7/site-packages/click/core.py", line 1137, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/Users/snorris/miniconda3/envs/futurefire/lib/python3.7/site-packages/click/core.py", line 956, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/Users/snorris/miniconda3/envs/futurefire/lib/python3.7/site-packages/click/core.py", line 555, in invoke
return callback(*args, **kwargs)
File "/Users/snorris/Projects/cat/2019_wildfire/futurefire/futurefire/cli.py", line 231, in burn
forest_current, burn_image, burn_list = futurefire.apply_fires(fires, forest_current, burn_image, year, n=n)
File "/Users/snorris/Projects/cat/2019_wildfire/futurefire/futurefire/main.py", line 114, in apply_fires
increment = (config["fire_ellipse_pct_growth"] * .01) * target_area
TypeError: can't multiply sequence by non-int of type 'float'
config file
[CONFIG]
wksp = wksp
inputs_gdb = inputs/future_fire_2019_01_24.gdb
roads = roads
inventory = inventory
regions = inputs/regions.tif
# bounds are "xmin ymin xmax ymax" in target area preserving projection
# (EPSG:3005, BC Albers)
bounds = 159587.5 173787.5 1881187.5 1748187.5
buffer = 500
cell_size = 100
fire_axis_ratio_min = 20
fire_axis_ratio_max = 80
# increasing the pct growth reduces iterations and processing time
fire_ellipse_pct_growth = 2
regen = 10
outputs = outputs
log_file = futurefire.log
# match the region names in scenario file to regions.tif raster values
[REGION_LOOKUP]
Coast = 1
Northern Interior = 2
Southern Interior = 3
The job takes a long time to run - there are likely substantial speedups possible by optimizing code.
Expanding the ellipse to meet a fixed area takes a long time. Looping over arrays is a known bottleneck. It may be possible to adapt the ellipse function to handle expansion rather than looping until target met? If so, perhaps all ignition points could be generated in one step, then expansion applied in a single step. This requires some better familiarity with numpy
.
Burns don't seem to be burning forest that has had regen
years to recover. Add a test to check that this is happening
csv is being overwritten for each draw, place it in the output for the given draw
Generate and export salvage data
Some runs are bailing. Likely due to the new logic for finding a new ignition cell - this should be tested.
Line 169 in dd9af1d
IndexError: boolean index did not match indexed array along dimension 0; dimension is 15745 but corresponding boolean dimension is 15744
(futurefire)
>>> regions.shape
(15744, 17216)
>>> forest.shape
(15745, 17217)
Supplying a fire ignition probability raster is straightforward, but it looks like we have to modify the randomization to use a random choice for every fire rather than shuffling forest and picking from the top of the 'deck'.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.