oliver-be / ring-of-fire Goto Github PK
View Code? Open in Web Editor NEWA parallelized genetic program for evolving rules that describe a cellular automaton-based wildfire simulation.
License: Eclipse Public License 2.0
A parallelized genetic program for evolving rules that describe a cellular automaton-based wildfire simulation.
License: Eclipse Public License 2.0
Thanks for that.
FYI I'm including below a reply from Andy, which adds a bunch of additional info.
Some of the formatting may not be great, but Moodle does unfortunate things (html escaping) with some characters in markdown literal mode, so I'm just formatting this as plain text... if you want some of the output to line up better, paste it into something that uses fixed-width fonts.
Issac, your runs get singled out here because those are the ones that are currently in the pipeline; don't take it personally!
-Lee
--- from Andy:
Hold usually means it can’t start the job for some reason. Idle may means it’s waiting for nodes, and that can be because it’s been running for a while and has been “evicted”. At the moment I see that almost all of the nodes are busy:
$ condor_status -sched
TotalRunningJobs TotalIdleJobs TotalHeldJobs
Total 86 317 1
So there’s a lot of competition for nodes. It may also take a while to get running because the available nodes don’t have the necessary resources, e.g. not enough memory is available. Many of Isaac’s jobs are 10–12 GB in size, when on average there is only 8 GB per job available (nodes 1-3) or 4 GB per job available (nodes 4–6).
He currently has one held job, sitting there for a day and a half, the reason is:
$ condor_q -held 725.3
-- Schedd: csc3-desktop.amherst.edu : <148.85.78.197:9618?... @ 04/30/20 02:26:42
ID OWNER HELD_SINCE HOLD_REASON
725.3 icaruso21 4/28 18:31 Failed to initialize user log to /home/icaruso21/ring-of-fire/prod-results-v2s/timestep1/3/log
but the actual directory name is /home/icaruso21/ring-of-fire/prod-results-v2/timestep1/3 . (“v2” not “v2s”).
Important: please remind the students to put their projects into cluster-archive. Their home directory has relatively limited space available.
— Andy
ring-of-fire/src/ring_of_fire/propel_helper.clj
Lines 491 to 503 in ca0e0d4
ring-of-fire/prod-results-v2/1fire/7/out
Line 99 in e7175f6
We need to figure out where the bulk of our runtime is occurring. It seems that it is likely in run-fire when we have a small time-step (meaning we only increase time by a small amount for each loop), but we need to explore other parts of our program to narrow this down so we can run more generations at a higher time step
So one of our lightweight jobs discovered an error... could be what @Oliver-BE mentioned in the other error issue.
ring-of-fire/production-results/lightweight/2/err
Lines 1 to 64 in 7f0a648
And this is the output data that causes the error:
ring-of-fire/production-results/lightweight/2/out
Lines 1 to 15 in 7f0a648
Since different fires with different dimensions are being tested on each generation, we should both print out that total number of cells for the fire(s) chosen and a simple percent error print statement to better help sift through our results
ring-of-fire/prod-results-v2/1fire/5/out
Lines 19 to 24 in 220e342
run-fire always updates-cells 1 more time than it needs to
This code would allow us to watch as the fire gets updated
ring-of-fire/iPhoneXT/4firest/1/out
Lines 12 to 20 in e1e189e
Not sure this is working / this is the best way to implement this
Not sure what changed here, but we can't read it in
our time step is still too low and runs too slowly
fix the circular dependencies at the top of our files
dont forget to squash it
set aside 3 fires that are to be tested on (and never trained on in the error function)
implement the "pick 1 per error" function that professor Spector was talking about to speed up runtime
turn map of neighbors into a vector where direction is net direction of burning cells and magnitude is the avg time all neighbors have been burning
Can we leave the output of compare grids (which determines our error vector for lexicase selection) as a lazy sequence or does it need to be a vector? Casting an extremely large lazy sequence as a vector means that we are potentially wasting lots of time each time we call our fire-error-function
We need to aggregate and/or select 6-12 best instructions to give our program
ring-of-fire/prod-results-v2/lightweight/4/out
Lines 75 to 80 in 1b8c936
Oliver can do this, see write-csv method in data.clj
This would reduce bias in fires with a larger number of non-fuel cells, and allow us to better compare the work our actual program is doing
Tidy up the code, provide working examples for each function
In update-cell we might save a bunch of time by creating a weather dictionary of all of our individual value instead of calling the lookup function we created on 8 different variables
We need to look at our instructions to make sure all of them make sense and perhaps remove some that logically would not affect fire spread. Specifically, we should look at the weather data and decide whether or not each column has relevant data (if there is data that doesn't change much across 24 hours then we shouldn't waste time by having it be an instruction). In short, we should cut down on the number of weather variables we are looking at, if possible.
We should also consider consolidating weather variables into more useful data points such as wind direction that our program could then use in lieu of many different weather variables and having to figure it out itself.
Do we want to use slope data? If so, we would need to get rid of arrowhead 2 from the training set (and potentially altogether) as it contains missing slope data. (We could only test on it if the program does not include slope)
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.