sliders are visible, can move them around, but they don't update anything

missing DOCUMENTER_KEY?

I'm trying to do a solve for a problem with units, and the output has dimensions, but not units. To be able to multiply this by elements with units, I've been assigning it NoUnit units. However, this is throwing an error related to the getproperty
Here's a MWE

using BLUEs, Unitful, DimensionalData, UnitfulLinearAlgebra, LinearAlgebra
using Revise
K = u"K"

#unitless DE works 
de1 = DimEstimate(rand(25), rand(25, 25), (X(1:5), Y(1:5)))

#unitful DE works 
urange2 = fill(K, 25)
y2 = UnitfulMatrix(rand(25), urange2)
Cyy2 = UnitfulMatrix(rand(25,25), urange2, urange2 .^ -1)
de2 = DimEstimate(y2, Cyy2, (X(1:5), Y(1:5)))

#what if the type is a Quantity but no units
urange3 = fill(NoUnits, 25)
y3 = UnitfulMatrix(rand(25), urange3)
Cyy3 = UnitfulMatrix(rand(25, 25), urange3, urange3 .^ -1)
#next line technically runs, but fails the show
de3 = DimEstimate(y3, Cyy3, (X(1:5), Y(1:5)))

The error looks like

julia> de3 = DimEstimate(y3, Cyy3, (X(1:5), Y(1:5)))
DimEstimate{Float64, Float64}
Estimate and 1σ uncertainty
Error showing value of type DimEstimate{Float64, Float64}:
ERROR: MethodError: no method matching measurement(::Quantity{Float64, NoDims, Unitful.FreeUnits{(), NoDims, nothing}}, ::Float64)

I think I've tracked the problem to the fact that de3.v and de2.v both return UnitfulMatrix, but de3.\sigma returns Vector{Float} while de2.v returns a Vector{Quantity}. This causes the above error at Line 154
https://github.com/ggebbie/BLUEs.jl/blob/d94ed006eb5580e44715c843b69ab9a55bda2b3b/src/BLUEs.jl#L154C22-L154C22
When measurement. is called in getproperty, it's receiving a Quantity for the v value and a Float for x.\sigma, hence the above error.

After a little perusing, I've come up with two solutions

1. Hacky solution in getproperty: just check the actual units on x.v and if they're NoUnits, call parent. This would just look like changing the following line
   

   BLUEs.jl/src/BLUEs.jl

   Line 150 in d94ed00

   v = Matrix(x.v)

   
   to

v = x.v.dims[1][1] isa Unitful.FreeUnits{(), NoDims, nothing} ? parent(x.v) : Matrix(x.v)

2. Less hacky, more likely to break something else: the real root of the problem is that x.\sigma should not return Vector{Float}, it should return Vector{Quantity}. This type gets decided by diag(x.C) in getproperty. diag first checks dimensionless(x.C) which returns true for the case where the dimensions are NoUnits. I guess this is technically correct, but ultimately causes diag to return the Vector{Float}. Should there be a third case in diag for the UnitfulMatrix with NoUnits? Or should dimensionless return false for this case? (I guess I don't really have a solution here, but more questions)

• Start from test "source water inversion: obs at one time, many surface regions, with circulation lag" or the case with an observational timeseries
• construct a random M (water-mass matrix) like before, but construct two "x" variables
• decide how to return the x solution: 1) 3D DimArray with time, surface region, variable type, 2) NamedTuple with 2 fields that each have time, surface region, 3) Two or more DimArrays
• update convolve (i.e., the forward function that does x -> y) to handle the output from the previous step.
• Store convolve output as the synthetic data. Ultimately the data should be corrupted according to Cnn.
• Given knowledge of y and dims(x), produce a first guess x0 that has the same form as step 3 above
• Run x0 through the forward function to make a first prediction of the observation(s) y0
• See if datacost still works to compute half of the cost function
• Update construction of Cxx to handle two variables, assume diagonal in first step, relax this later, probably best to use vec to change x0::Union{DimArray,NamedTuple,etc.} into a vector, Cxx requires the same type of bookkeeping, perhaps a new function to do this automatically would be useful
• Modify controlcost to handle two state variables, must take the Cxx output from the previous step
• If using a timeseries of observations, modify impulseresponse to take M water-mass matrix and return E the matrix used in BLUEs.solve
• Check that E*vec(x0) gives the same thing as convolve(x0,M)
• Pass new problem to BLUEs.solve
• Modify BLUEs.getproperty to handle cases where there are two state variables, if the state variables are in a DimArray then ndims>2 could be checked to see if there are multiple state variables
• Check that output and covariance is appropriately displayed in the REPL in the same form as x and x0, and that x.x contains this information

I can multiply a matrix by an Estimate

julia> e = Estimate(randn(5), abs.(randn((5,5))))

julia> randn((1,5)) * e
Estimate{Float64, Float64}
Estimate and 1σ uncertainty
1-element Vector{Measurements.Measurement{Float64}}:
 1.6 ± 2.0

But not an estimate by a matrix

julia> e * randn((1,1))
ERROR: MethodError: no method matching *(::Estimate{Float64, Float64}, ::Matrix{Float64})

Multiplication does not work for a DimEstimate times a UnitfulDimMatrix, either way

julia> de = DimEstimate(randn(10), abs.(randn((10,10))), (X(1:5), Y(1:2)))
julia> udm = UnitfulDimMatrix(randn(1,10), fill(unit(1.0), 1), fill(unit(1.0), 10), dims = (Ti = [1], X = 1:10))
julia> udm * de
ERROR: MethodError: no method matching Estimate

Estimate tracks best estimate and uncertainty

When there are multiple bundled Estimates, like an Estimate for temperature and salinity in one problem, their uncertainty can be correlated through off-diagonal elements in the joint Estimate uncertainty matrix. Therefore some information is missing if two independent Estimates are used for temperature and salinity separately.

One could expand the Estimate struct to include a covariance matrix. Depending on the number of variables/sub-types in the Estimate, it would get messy fast. BlockFactorizations could probably handle these blocks in the Estimate uncertainty matrix.

Alternatively one could use a Matrix that has indexing to locate various subtypes. This approach would just store one vector and one matrix in Estimate, as it was originally constructed. We expect the uncertainty matrix to generally be dense and full, and thus this approach would not be missing out on memory savings by storing one big matrix.

For approach #2, there are a number of community packages. AxisArrays.jl is less maintained.

AxisArrays 35, AxisArraysFuture 30, NamedDims.jl 40, DimensionalData.jl 37
https://github.com/Tokazama/AxisIndices.jl

discussion here:
JuliaCollections/AxisArraysFuture#1
https://discourse.julialang.org/t/the-fate-of-dimensionalarrays-axisarrays-in-julia-and-which-to-actually-use/33253

Can we have an M with no time lag? This is how I've been handling M in all OPT2k code up to now. Two options for this

• M follows same structure (DimArray with 2 dimensions, Ti and SurfaceRegion) as existing test, is just one row instead of 4, with zero lag. I bet this works out of the box
• M is a DimArray with 1 dimension, SurfaceRegion. Don't need to convolve anymore, can just calculate Mx (I think I would have to define DimArray * DimArray multiplication?). Would everything after this just work?

should add test for DimEstimate

I'm trying to figure out why my solutions have significantly larger values at the first time step than other time steps. I'm not sure if this is an issue, or if it would solve my problem, but I'm wondering if it's appropriate to use Near while time indexing in convolve

• error propagation a la Distributions.jl for vectors with correlation
• Problem struct a la LinearSolve.jl
• make struct types more concrete to help performance
• Underdetermined problems
• Distributions not keeping solution/error significant digits consistent
• permit weighting matrices to be vectors that describe diagonal
• optional calculate cost in solve
• check performance of inner products, allow quick inner product when weighted by a diagonal matrix
• replace weighting matrices with Cholesky factorizations
• permit NamedTuple Estimate, pass this Estimate to solve
• permit NamedTuple Estimate with covariance between fields, explore BlockFactorizations for operations/(*)
• permit scalar (not vector) Estimate
• Multiple observational matrices in a NamedTuple
• do tests for dimensionless problems

NOTE: unitrange, unitdomain have different units. requires kludge below.

Cnn⁻¹ = Diagonal(fill(σₓ^-1,M),unitrange(E).^-1,unitrange(E).^1,exact=true)

failed at creating a notebook environment

may require registering UnitfulLinearAlgebra

I have an error being thrown on #47 right now that also occurs is if I clone a fresh version of BLUEs, and run runtests.jl again. The error is that for any \, a DimensionMismatch occurs. Not quite sure why this is happening...

Example error

julia> include("test/runtests.jl")
left-uniform problem with prior info: Error During Test at /home/brynn/Downloads/BLUEs.jl/test/runtests.jl:89
  Got exception outside of a @test
  DimensionMismatch: Dimension index [‰] and Unitful.FreeUnits[‰] do not match
  Stacktrace:

in runtests, the left-uniform test with prior information (and other tests) will not run unless y=UnitfulMatrix([-1.9permil]) but it would be more convenient to write y=[-1.9permil].

The error is

julia> @test cost(x̃1,oproblem) < 1 # rough guide, coul
Error During Test at REPL[71]:1
  Test threw exception
  Expression: cost(x̃1, oproblem) < 1
  MethodError: no method matching rebuild(::UnitfulMatrix{Float64, 1, Tuple{UnitfulLinearAlgebra.Units{DimensionalData.Dimensions.LookupArrays.Sampled{Unitful.FreeUnits{(‰,), NoDims, nothing}, Vector{Unitful.FreeUnits{(‰,), NoDims, nothing}}, DimensionalData.Dimensions.LookupArrays.Unordered, DimensionalData.Dimensions.LookupArrays.Irregular{Tuple{Nothing, Nothing}}, DimensionalData.Dimensions.LookupArrays.Points, DimensionalData.Dimensions.LookupArrays.NoMetadata}}}, Vector{Float64}}, ::Vector{Float64}, ::Tuple{UnitfulLinearAlgebra.Units{DimensionalData.Dimensions.LookupArrays.Sampled{Unitful.FreeUnits{(‰,), NoDims, nothing}, Vector{Unitful.FreeUnits{(‰,), NoDims, nothing}}, DimensionalData.Dimensions.LookupArrays.Unordered, DimensionalData.Dimensions.LookupArrays.Irregular{Tuple{Nothing, Nothing}}, DimensionalData.Dimensions.LookupArrays.Points, DimensionalData.Dimensions.LookupArrays.NoMetadata}}}, ::Tuple{}, ::Symbol, ::DimensionalData.Dimensions.LookupArrays.NoMetadata)
  
  Closest candidates are:
    rebuild(::AbstractDimStack, ::Any, ::Any, ::Any, ::Any, ::Any)
     @ DimensionalData ~/.julia/packages/DimensionalData/4TpBG/src/stack/stack.jl:135
    rebuild(::AbstractDimStack, ::Any, ::Any, ::Any, ::Any, ::Any, ::Any)
     @ DimensionalData ~/.julia/packages/DimensionalData/4TpBG/src/stack/stack.jl:135
    rebuild(::UnitfulDimMatrix, ::Any, ::Tuple, ::Tuple, ::Any, ::Any)
     @ UnitfulLinearAlgebra ~/.julia/packages/UnitfulLinearAlgebra/CiPHW/src/UnitfulDimMatrix.jl:76
    ...
  
  Stacktrace:
    [1] rebuild
      @ ~/.julia/packages/DimensionalData/4TpBG/src/array/array.jl:54 [inlined]
    [2] copy(bc::Base.Broadcast.Broadcasted{DimensionalData.DimensionalStyle{Base.Broadcast.DefaultArrayStyle{1}}, Tuple{UnitRange{Int64}}, typeof(-), Tuple{Vector{Quantity{Float64, NoDims, Unitful.FreeUnits{(‰,), NoDims, nothing}}}, UnitfulMatrix{Float64, 1, Tuple{UnitfulLinearAlgebra.Units{DimensionalData.Dimensions.LookupArrays.Sampled{Unitful.FreeUnits{(‰,), NoDims, nothing}, Vector{Unitful.FreeUnits{(‰,), NoDims, nothing}}, DimensionalData.Dimensions.LookupArrays.Unordered, DimensionalData.Dimensions.LookupArrays.Irregular{Tuple{Nothing, Nothing}}, DimensionalData.Dimensions.LookupArrays.Points, DimensionalData.Dimensions.LookupArrays.NoMetadata}}}, Vector{Float64}}}})
      @ DimensionalData ~/.julia/packages/DimensionalData/4TpBG/src/array/broadcast.jl:43
    [3] materialize
      @ ./broadcast.jl:873 [inlined]
    [4] broadcast_preserving_zero_d
      @ ./broadcast.jl:862 [inlined]
    [5] -(A::Vector{Quantity{Float64, NoDims, Unitful.FreeUnits{(‰,), NoDims, nothing}}}, B::UnitfulMatrix{Float64, 1, Tuple{UnitfulLinearAlgebra.Units{DimensionalData.Dimensions.LookupArrays.Sampled{Unitful.FreeUnits{(‰,), NoDims, nothing}, Vector{Unitful.FreeUnits{(‰,), NoDims, nothing}}, DimensionalData.Dimensions.LookupArrays.Unordered, DimensionalData.Dimensions.LookupArrays.Irregular{Tuple{Nothing, Nothing}}, DimensionalData.Dimensions.LookupArrays.Points, DimensionalData.Dimensions.LookupArrays.NoMetadata}}}, Vector{Float64}})
      @ Base ./arraymath.jl:8
    [6] datacost(x̃::Estimate{Float64, Float64}, p::OverdeterminedProblem)
      @ BLUEs ~/projects/BLUEs/src/BLUEs.jl:390
    [7] cost(x̃::Estimate{Float64, Float64}, op::OverdeterminedProblem)
      @ BLUEs ~/projects/BLUEs/src/BLUEs.jl:369
    [8] top-level scope
      @ ~/.julia/juliaup/julia-1.9.3+0.x64.linux.gnu/share/julia/stdlib/v1.9/Test/src/Test.jl:478
    [9] eval
      @ ./boot.jl:370 [inlined]
   [10] eval_user_input(ast::Any, backend::REPL.REPLBackend, mod::Module)
      @ REPL ~/.julia/juliaup/julia-1.9.3+0.x64.linux.gnu/share/julia/stdlib/v1.9/REPL/src/REPL.jl:153
   [11] repl_backend_loop(backend::REPL.REPLBackend, get_module::Function)
      @ REPL ~/.julia/juliaup/julia-1.9.3+0.x64.linux.gnu/share/julia/stdlib/v1.9/REPL/src/REPL.jl:249
   [12] start_repl_backend(backend::REPL.REPLBackend, consumer::Any; get_module::Function)
      @ REPL ~/.julia/juliaup/julia-1.9.3+0.x64.linux.gnu/share/julia/stdlib/v1.9/REPL/src/REPL.jl:234
   [13] run_repl(repl::REPL.AbstractREPL, consumer::Any; backend_on_current_task::Bool, backend::Any)
      @ REPL ~/.julia/juliaup/julia-1.9.3+0.x64.linux.gnu/share/julia/stdlib/v1.9/REPL/src/REPL.jl:379
   [14] run_repl(repl::REPL.AbstractREPL, consumer::Any)
      @ REPL ~/.julia/juliaup/julia-1.9.3+0.x64.linux.gnu/share/julia/stdlib/v1.9/REPL/src/REPL.jl:365
   [15] (::Base.var"#1017#1019"{Bool, Bool, Bool})(REPL::Module)
      @ Base ./client.jl:421
   [16] #invokelatest#2
      @ ./essentials.jl:819 [inlined]
   [17] invokelatest
      @ ./essentials.jl:816 [inlined]
   [18] run_main_repl(interactive::Bool, quiet::Bool, banner::Bool, history_file::Bool, color_set::Bool)
      @ Base ./client.jl:405
   [19] exec_options(opts::Base.JLOptions)
      @ Base ./client.jl:322
   [20] _start()
      @ Base ./client.jl:522
ERROR: There was an error during testing