nrel / dynamo

A modular MATLAB toolkit for Dynamic programming (DP) and Approximate Dynamic Programming (ADP) for Adaptive Modeling and Optimization

dynamo

dynamo - Dynamic programming for Adaptive Modeling and Optimization

Install

• Install MATLAB (R2017a or latter preferred)
• Clone this repository
• Open the Home>Set Path dialog and click on Add Folder to add the following folders to the PATH:
  • $DYNAMO_Root/src
  • $DYNAMO_Root/extern (Add all subfolders for this one)

Getting Started

Explore the example directory. The multi_inventory and Storage_Size_for_PV examples are best well tested.

Tests

1. Add: $DYNAMO_Root/example/multi_inventory and $DYNAMO_Root/example/Storage_Size_for_PV to your MATLAB path

2. Change to the $DYNAMO_Root/test folder

3. Run the following command to run all unit and doctest-based tests

   dynamo_test_all

dynamo's Issues

Automatically extend state_set to cover all time periods

Assume extended states match the last one provided by the user.

ADP NaN bug!

Hi @bpalmintier
At this point, we have our HEMS example running and working properly for DP ( using dpBI) and we are simulating some cases. For the ADP( using adpSBI) however, we are having this NaN issue that looks to be a toolbox bug based on the history of faLocalAvg.m below, if we have not misunderstood.

%% HISTORY
% ver     date    time       who     changes made
% ---  ---------- -----  ----------- ---------------------------------------
%  14  2017-04-26 22:55  BryanP      BUGFIX: fix to avoid excess NaN

When we run adpSBI for our HEMS problem, most of the time we get the following error.

>> HEMS_Main
Elapsed time is 0.000109 seconds.
Sampled Backward Induction ([0.1] ksamples/period)
    Creating empty post-decision value functions (LocalAvg)
    T=8 (terminal period): Done
    T=7:S........................................100
    T=6:S........................................**Error using faLocalAvg/build_func (line 255)
input data contains NAN values.

Error in FuncApprox/approx (line 201)
                obj.build_func();

Error in faLocalAvg/approx (line 207)
            out_vals = approx@FuncApprox(obj, out_pts);

Error in FindOptDecFromVfun (line 43)
    future_val_list = approx(vfun, vfun_state_list, vfun_approx_params{:});

Error in adpSBI (line 403)
    parfor next_pre_idx = 1:n_next_pre

Error in HEMS_Main (line 34)
results = adpSBI(HEMS_problem, sbi_opt);**

The sampling parameters we used for this particular case is below:

sbi_opt = {     'sbi_state_samples_per_time'            100    % Number of state samples per time period
                'sbi_decisions_per_sample'                      20    % Number of decision samples per state
                'sbi_uncertain_samples_per_post'            20   % Number of random/uncertainty samples per time, used for all decisions
                'vfun_approx'                           'LocalAvg'
                };

We would like your inputs on this issue.

Instructions in Readme file outdated

It seems that the instructions to set-up dynamo are not updated. There is no file setup_dynamo.

I ran dynamo_test_all and it gave me some errors in Matlab R2016b. Here is the output

------ UNIT TESTS -------
Running testrpLattice
..........
.
Done testrpLattice
__________

testrpLattice.m: PASS: 11/11 tests pass (100%)


------ DOCTESTS -------

Testing setSingleItem
TAP version 13
1..8
ok 1 - "format shortG"
ok 2 - "my_set = setSingleItem([1 2 3 4 5.6]);"
ok 3 - "my_set.N_dim"
not ok 4 - "my_set.sample()"
    in setSingleItem
    expected: ans = 1 2 3 4 5.6
    got     : ans = Columns 1 through 4 1 2 3 4 Column 5 5.6
not ok 5 - "my_set.sample(4)"
    in setSingleItem
    expected: ans = 1 2 3 4 5.6 1 2 3 4 5.6 1 2 3 4 5.6 1 2 3 4 5.6
    got     : ans = Columns 1 through 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 Column 5 5.6 5.6 5.6 5.6
not ok 6 - "my_set.as_array()"
    in setSingleItem
    expected: ans = 1 2 3 4 5.6
    got     : ans = Columns 1 through 4 1 2 3 4 Column 5 5.6
ok 7 - "my_set = setSingleItem([10; 20; 30]);"
ok 8 - "my_set.Value"
FAIL: 5/8 tests pass (62.5%)

Testing setList
TAP version 13
1..5
ok 1 - "rng('default'); my_set = setList([1 2; 11 12; 21 22]);"
ok 2 - "my_set.N_dim"
ok 3 - "my_set.sample()"
ok 4 - "my_set.sample(4)"
ok 5 - "my_set.as_array()"
PASS: 5/5 tests pass (100%)

Testing setBasic
TAP version 13
1..6
ok 1 - "format shortG"
ok 2 - "steps = [0.05 25 600 0 0];"
ok 3 - "ranges = [-0.15 0 200 0 20; 0.2 100 800 0 10]; % corresponds to [min min min; max max max]"
ok 4 - "states = {setBasic(ranges, '',  steps)};"
ok 5 - "state_list = states{1}.as_array;"
not ok 6 - "state_list_top15 = state_list(1:15, :)"
    in setBasic
    expected: state_list_top15 = -0.15 0 200 0 20 -0.15 0 800 0 20 -0.15 25 200 0 20 -0.15 25 800 0 20 -0.15 50 200 0 20 -0.15 50 800 0 20 -0.15 75 200 0 20 -0.15 75 800 0 20 -0.15 100 200 0 20 -0.15 100 800 0 20 -0.1 0 200 0 20 -0.1 0 800 0 20 -0.1 25 200 0 20 -0.1 25 800 0 20 -0.1 50 200 0 20
    got     : state_list_top15 = Columns 1 through 4 -0.15 0 200 0 -0.15 0 800 0 -0.15 25 200 0 -0.15 25 800 0 -0.15 50 200 0 -0.15 50 800 0 -0.15 75 200 0 -0.15 75 800 0 -0.15 100 200 0 -0.15 100 800 0 -0.1 0 200 0 -0.1 0 800 0 -0.1 25 200 0 -0.1 25 800 0 -0.1 50 200 0 Column 5 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20
FAIL: 5/6 tests pass (83.3%)

Testing setCombinWithLimits
TAP version 13
1..7
ok 1 - "rng('default')  %Reset random number stream"
ok 2 - "my_set = setCombinWithLimits('from_list', false, [2 3], 6)"
ok 3 - "my_set.as_array()"
ok 4 - "my_set.sample()"
ok 5 - "my_set.sample(5)"
ok 6 - "my_set = setCombinWithLimits('from_list', true, [2 3], 6)"
ok 7 - "my_set.as_array()"
PASS: 7/7 tests pass (100%)

Testing rpDiscreteSample
TAP version 13
1..15
ok 1 - "rng(0); s = rpDiscreteSample({[0, 1, 2, 3]'}, {[0.1, 0.5, 0.35, 0.05]'});"
ok 2 - "s.sample()"
ok 3 - "s = rpDiscreteSample({[0 1; 1 2; 3 4; 5 6], [10 11; 20 21; 30 31]});"
ok 4 - "[t1_vals, t1_prob] = s.dlistnext()"
ok 5 - "s.cur_state()"
ok 6 - "s.t"
ok 7 - "s.step()"
ok 8 - "s.t"
ok 9 - "s.step()"
ok 10 - "s.step()"
ok 11 - "s.t"
ok 12 - "s = rpDiscreteSample({[0 1; 1 2; 3 4; 5 6], [10 11; 20 21; 30 31]});"
ok 13 - "s.range()"
ok 14 - "s.range(2)"
ok 15 - "s.range('all')"
PASS: 15/15 tests pass (100%)

Testing rpLattice
TAP version 13
1..2
ok 1 - "format shortG"
ok 2 - "lattice_object = rpLattice(5, [0.5  1.2 1.5 ]', [0.15 0.5 0.35]', 3)"
PASS: 2/2 tests pass (100%)

Testing rpTransMatrix
TAP version 13
1..11
ok 1 - "format shortG"
ok 2 - "rng(0);"
ok 3 - "states = {0.76, [0.76+0.33; 0.76+0.1], [0.76+0.33+0.42; .76+0.1+0.4; 0.76+0.1+0.25]};"
ok 4 - "prob = {[1 1]/2, [0.80 0.20  0   ; 0   0.2 0.8]};"
ok 5 - "rtm_obj = rpTransMatrix(states, prob)"
ok 6 - "[s_list, p_list] = rtm_obj.as_array(1)"
ok 7 - "[s_list, p_list] = rtm_obj.as_array(3)"
ok 8 - "rtm_obj.sample(3)"
ok 9 - "rtm_obj.sample(5, 3)"
ok 10 - "rtm_obj.dlistnext(2,1.09)"
ok 11 - "rtm_obj.dlistnext(2,0.86)"
PASS: 11/11 tests pass (100%)

Testing utilRandSetCurState
TAP version 13
1..4
ok 1 - "rp_set = {rpDiscreteSample({1}), rpDiscreteSample({[10 20; 30 40]})};"
ok 2 - "utilRandSetCurState(rp_set)"
ok 3 - "utilRandSetCurState(rp_set, {1, [100, 200]})"
ok 4 - "utilRandSetCurState(rp_set, [1 100 200])"
PASS: 4/4 tests pass (100%)

Testing utilRandStatefromState
TAP version 13
1..4
ok 1 - "full_state = [1 20 3 44];"
ok 2 - "utilRandStatefromState(full_state, {[4] [2,1] []})"
ok 3 - "ans{2}"
ok 4 - "utilRandStatefromState(full_state, {[] [] []})"
PASS: 4/4 tests pass (100%)

Testing IntegerRangeFromReal
TAP version 13
1..5
ok 1 - "format shortG"
not ok 2 - "[integer_max, offset, step_size] = IntegerRangeFromReal([0 1 1.5 3 3.2 20; 2 2 3 4.2 4 80], [.5 .5 .3 .2 0 60])"
    in IntegerRangeFromReal
    expected: integer_max = 5 3 6 7 11 2 offset = 0 1 1.5 3 3.2 20 step_size = 0.5 0.5 0.3 0.2 0.08 60
    got     : integer_max = 5 3 6 7 11 2 offset = Columns 1 through 4 0 1 1.5 3 Columns 5 through 6 3.2 20 step_size = Columns 1 through 4 0.5 0.5 0.3 0.2 Columns 5 through 6 0.08 60
ok 3 - "min_max = zeros(2, length(integer_max));"
ok 4 - "min_max(2,:) = (integer_max - 1) .* step_size;"
not ok 5 - "min_max = bsxfun(@plus, min_max, offset)"
    in IntegerRangeFromReal
    expected: min_max = 0 1 1.5 3 3.2 20 2 2 3 4.2 4 80
    got     : min_max = Columns 1 through 4 0 1 1.5 3 2 2 3 4.2 Columns 5 through 6 3.2 20 4 80
FAIL: 3/5 tests pass (60%)

Testing doctest_MultiInv
TAP version 13
1..28
ok 1 - "rng('default')"
ok 2 - "multi_inv_problem = MultiInv_demo('med');   %Runs MultiInvParamsSetup"
ok 3 - "t = 1;"
ok 4 - "test_pre_state_list = multi_inv_problem.state_set{t}.sample(6)"
ok 5 - "test_pre_state = multi_inv_problem.state_set{t}.sample()"
ok 6 - "dec_set_test = multi_inv_problem.fDecisionSet(multi_inv_problem.params, t, test_pre_state)"
ok 7 - "dec_list = dec_set_test.as_array;"
ok 8 - "size(dec_list)"
ok 9 - "dec_subset = dec_list(1:10,:)"
ok 10 - "multi_inv_problem.fDecisionCost(multi_inv_problem.params, t, test_pre_state, dec_subset)"
ok 11 - "post_state_list = multi_inv_problem.fDecisionApply(multi_inv_problem.params, t, test_pre_state, dec_subset)"
ok 12 - "sample_list = RandSetSample(multi_inv_problem.random_items, 10, t)"
ok 13 - "rand_contrib = multi_inv_problem.fRandomCost(multi_inv_problem.params, t, post_state_list, sample_list)"
ok 14 - "next_pre_list = multi_inv_problem.fRandomApply(multi_inv_problem.params, t, post_state_list, sample_list)"
ok 15 - "isempty(multi_inv_problem.fOpsBeforeDecision)"
not ok 16 - "func2str(multi_inv_problem.fOpsAfterDecision)"
    in doctest_MultiInv
    expected: ans = 'MultiInvOps'
    got     : ans = MultiInvOps
ok 17 - "isempty(multi_inv_problem.fOpsAfterRandom)"
ok 18 - "MultiInvOps(multi_inv_problem.params, t, test_pre_state)"
ok 19 - "multi_inv_problem.fOpsAfterDecision(multi_inv_problem.params, t, post_state_list, dec_subset)"
ok 20 - "MultiInvOps(multi_inv_problem.params, t, post_state_list, dec_subset, 'junk')"
ok 21 - "[~, full_list] = MultiInvOps(multi_inv_problem.params, t, test_pre_state, [], [], 'old_full')"
ok 22 - "[~, full_list] = multi_inv_problem.fOpsAfterDecision(multi_inv_problem.params, t, post_state_list, dec_subset, [], 'old_full')"
ok 23 - "[~, full_list] = MultiInvOps(multi_inv_problem.params, t, post_state_list, dec_subset, 'junk', 'old_full')"
ok 24 - "final_states = multi_inv_problem.state_set{multi_inv_problem.n_periods}.sample(10)"
ok 25 - "cache_term_val = multi_inv_problem.params.term_unit_val;"
ok 26 - "multi_inv_problem.params.term_unit_val = 1:multi_inv_problem.params.n_products;"
ok 27 - "MultiInvTerminalValue(multi_inv_problem.params, multi_inv_problem.n_periods, final_states)"
ok 28 - "multi_inv_problem.params.term_unit_val = cache_term_val;"
FAIL: 27/28 tests pass (96.4%)

Testing MultiInv_demo
TAP version 13
1..11
ok 1 - "format shortG"
ok 2 - "small_opt_policy_cell = {[0,0]		[2,4]	[2,4]	[1,2]; [0,1]		[2,3]	[2,3]	[0,0]; [0,2]		[2,2]	[2,2]	[0,0];  [0,3]		[2,1]	[0,0]	[0,0] ; [0,4]		[2,0]	[2,0]	[0,0] ; [0,5]		[2,0]	[2,0]	[0,0] ; [0,6]		[0,0]	[0,0]	[0,0];  [1,0]		[1,4]	[1,4]	[0,2] ; [1,1]		[1,3]	[1,3]	[0,0]; [1,2]		[0,0]	[0,0]	[0,0] }; "
ok 3 - "small_opt_value = 'TODO';"
ok 4 - "[small_prob, small_dp_result] = MultiInv_demo('small', 'dp');"
ok 5 - "small_dp_policy = cell2mat(small_dp_result.dpbi_policy);"
ok 6 - "small_opt_policy = cell2mat(small_opt_policy_cell);"
ok 7 - "isequal(small_opt_policy(:, 3:end), small_dp_policy(1:10,:))"
ok 8 - "rng('default')"
not ok 9 - "[~, small_sbi_result] = MultiInv_demo('small', 'sbi');"
    in MultiInv_demo
    expected: Sampled Backward Induction ([0.1] ksamples/period) Creating empty post-decision value functions (LocalRegr) T=4 (terminal period): Done T=3:S........................................100 T=2:S........................................100 T=1:S........................................100 Warning: Multiple initial states defined, using first in list > In adpSBI (line ***)
    got     : ??? Error using faLocalAvg/build_func (line 255) Attempt to execute SCRIPT kdtree_build as a function: /Users/jdlara/Dropbox/Remote Code Rep/Matlab/Dynamo/extern/kdtree/kdtree_build.m
not ok 10 - "small_sbi_result"
    in MultiInv_demo
    expected: small_sbi_result = struct with fields: log: [1×1 struct] post_vfun: [1×4 faLocalRegr] adp_opt: [1×1 struct] first_decision: [2 4] objective: 16.322
    got     : ??? Undefined function or variable 'small_sbi_result'.
not ok 11 - "isequal(small_sbi_result.first_decision, small_opt_policy_cell{1,2})"
    in MultiInv_demo
    expected: ans = logical 1
    got     : ??? Undefined variable "small_sbi_result" or class "small_sbi_result.first_decision".
FAIL: 8/11 tests pass (72.7%)

Testing MultiInvSetupProblem
TAP version 13
1..10
ok 1 - "format shortG"
ok 2 - "params = {'total_space', 3; 'unit_space', 1; 'p_demand', [0.25; 0.5; 0.25] };"
ok 3 - "params = MultiInvSetupParams(params);"
not ok 4 - "multi_inv_problem = MultiInvSetupProblem(params)"
    in MultiInvSetupProblem
    expected: multi_inv_problem = struct with fields: params: [1×1 struct] discount_rate: 0.1 n_periods: 3 state_set: {[1×1 setCombinWithLimits] [1×1 setCombinWithLimits] [1×1 setCombinWithLimits] [1×1 setCombinWithLimits]} fTerminalValue: @MultiInvTerminalValue fDecisionSet: @MultiInvDecisionSet fDecisionCost: @MultiInvDecisionCost fDecisionApply: @MultiInvDecisionApply decision_vfun_map: [] random_items: {[1×1 rpDiscreteSample]} fRandomCost: @MultiInvRandomCost fRandomApply: @MultiInvRandomApply random_state_map: {[]} fOpsBeforeDecision: [] fOpsAfterDecision: @MultiInvOps fOpsAfterRandom: [] fCompareDecisionPolicy: [] fCompareValue: [] fMapState2Vfun: [] fOptimalDecision: [] fRandomSample: [] fRandomJoint: []
    got     : multi_inv_problem = struct with fields: params: [1×1 struct] discount_rate: 0.1 n_periods: 3 state_set: {1×4 cell} fTerminalValue: @MultiInvTerminalValue fDecisionSet: @MultiInvDecisionSet fDecisionCost: @MultiInvDecisionCost fDecisionApply: @MultiInvDecisionApply decision_vfun_map: [] random_items: {[1×1 rpDiscreteSample]} fRandomCost: @MultiInvRandomCost fRandomApply: @MultiInvRandomApply random_state_map: {[]} fOpsBeforeDecision: [] fOpsAfterDecision: @MultiInvOps fOpsAfterRandom: [] fCompareDecisionPolicy: [] fCompareValue: [] fMapState2Vfun: [] fOptimalDecision: [] fRandomSample: [] fRandomJoint: []
ok 5 - "multi_inv_problem = MultiInvSetupProblem(params, 'discount_rate', 0)"
ok 6 - "alt_as_cell = {'discount_rate', 0.2; 'n_periods', 5};"
ok 7 - "multi_inv_problem = MultiInvSetupProblem(params, alt_as_cell)"
ok 8 - "alt_as_struct.discount_rate = 0.03;"
ok 9 - "alt_as_struct.n_periods = 4;"
ok 10 - "multi_inv_problem = MultiInvSetupProblem(params, alt_as_struct)"
FAIL: 9/10 tests pass (90%)

========= dynamo testing summary ========
 UNIT TEST ( 1 files): PASS,  11/ 11 suites pass (100%) in 7.52417 sec
 DOCTEST   (13 files): FAIL, 105/116 lines  pass (90.5%) in 7.17235 sec

Inventory example return error

None of the DP examples are running on the develop branch

ADP gives '0' decision when it has small state space.

I run DP and ADP to solve the HEMS_HVAC_only case. It has 151 states. DP gives me reasonable decisions, but ADP always gives me '0'. I used exactly same thing other than call dpBI or adpSBI. The number of samples is enough for ADP to make decision.
Also, ADP is working when state space is large.

I just uploaded the HEMS HVAC only code to here. You can run 'plotSchedule.m' to get the ADP results. In order to run DP, just Comment line 46 and 47, Uncomment line 51 and 52 in 'plotSchedule.m'.

Also, Another question.
When I doing the water heater case, I found out the ADP always consider next hour random variable to make current hour decision. And I double checked my code and looked into the source code. I found something strange. The default random sample function is ‘RandSetSample.m’ which we are using. In line 43, the function call will be conditional sample and return sample of next hour. I think it should consider current hour sample, not next hour. Also, I looked into ‘RandProcess.m’ and ‘rpDiscreteSample.m’ and found that the overloaded parameters ‘cur_rand_state’ is not used. For now, I just simply changed the function call in .RandSetSample.m’ to be a normal sample, and it works fine.

Thank you.

Storage example

@bpalmintier Could you please remember to include this repo the Matlab code for the example you developed and presented in ISGT of storage investment?

Merge in @jbukenbe updates from Gen_Cap_Develop

@jbukenbe made a number of important bug fixes and extensions that should be part of the core code but require a bit of vetting. Notably:

• Fixing bugs associated with merging data points in kdtree-based function approximations
• Supporting Path-dependant random processes in adpSBI by passing the current random state to the sampling function
• Others?

ADP bug

If in the state set the final state is a set with one possible state (setSingleItem) the ADPSBI algorithm returns an error. If however I use (setBasic) and add a dummy state the code runs fine. It might be something you might want to look into.

Sampled Backward Induction ([0.002] ksamples/period)
Creating empty post-decision value functions (LocalAvg)
T=25 (terminal period): Done
T=24: SError using StorageSetDecisionApply (line 14)
Index exceeds matrix dimensions.

Error in adpSBI (line 223)
parfor samp = 1:samp_per_time

Error in Storage_Demo (line 38)
results_SBI = adpSBI(Storage_problem,adp_opt);