matbesancon / mathoptsetdistances.jl

Distances to sets for MathOptInterface

MathOptSetDistances

Set of functions to compute distances and projections to sets defined in MathOptInterface.jl.

Distance to set

set_distance(d::D, v::V, s::S) defines the distance of a point v to a set s. The distance is always 0 if v ∈ s. S is a MOI.AbstractSet, v is a scalar or vector value and d a type of distance that is a subtype of AbstractDistance.

New sets should implement at least set_distance(::DefaultDistance, v::V, s::MySet).

Projection on set

projection_on_set(d::D, v, s::S) returns the point on S that is closest to v with respect to the distance d. projection_gradient_on_set returns the gradient of this projection, i.e. the transpose of the Jacobian.

Gradients as ChainRules

Gradients projection_gradient_on_set eagerly computes the full derivative matrix. This is often simpler to test and implement, but leads to unnecessary allocations and expensive operations. They are also implemented using ChainRulesCore.jl methods rrule and frule. Both methods should be implemented for each derivative and tested against projection_gradient_on_set and FiniteDifferences.jl.

Special matrix types

When some gradients or projections have structural zeros (sparsity patterns), they can and should return non-standard matrices including FillArrays Zeros, Eyes, Ones, Fill, sparse arrays and LinearAlgebra.Diagonal.

mathoptsetdistances.jl's Issues

Testing with complex-valued sets

I assume lots of assumptions here are relying on real values v and real-valued sets s. Solvers are starting to support complex, so should we

Trait for sets with an interior

This makes sense only for continuous sets I think: adding a trait for sets that have an interior

Leverage special arrays when relevant

Lots of returned vectors and matrices are all zeros, identities and so on.
We should use FillArrays, SparseArrays and matrices from LinearAlgebra instead of dense arrays

Domains and inequality consistency

Suppose that you have a set defined as:
x | x >= 0 & f(x) <= 0

with dom(f) = {x | x >= 0}, this implies f(x) cannot be computed if x >= 0.
A first idea was to compute a distance as

if x < 0
    abs(x)
else
    max(f(x), 0)
end

This can result in consistency issues (still haven't formally written down how it appears)

frule from ChainRules

Implement frule and rrule from https://github.com/JuliaDiff/ChainRulesCore.jl for the gradient of projection

Switch Random to test-only dependency

I saw that Random was already in the package dependencies, but I don't think it's used outside of tests. I can move it to extras, unless there's something I'm missing.

Originally posted by @tjdiamandis in #29 (comment)

TagBot trigger issue

This issue is used to trigger TagBot; feel free to unsubscribe.

If you haven't already, you should update your TagBot.yml to include issue comment triggers.
Please see this post on Discourse for instructions and more details.

If you'd like for me to do this for you, comment TagBot fix on this issue.
I'll open a PR within a few hours, please be patient!

Test projections against AD

AD should work on these functions and let us check automatically if the automatic gradient is equal to the analytical form we provide

Exponential cone projection gradient

given v = [0.04, -3, 11]

the heuristic in _exp_cone_proj_case_4 returns [0, 0, 11]
then NaN is returned in projection_gradient_on_set because there is a division by the second element of such array in the sequence....

cc @matbesancon @tjdiamandis

Reduce usage of Random in tests

Reduce duplications

Lots of distances can be deduced from the corresponding projection, when computing the distance only is not more efficient, fallback to computing distance to the projection

Add norm ball projections

Add closed-form norm ball projections

Factor of sqrt(2) in unvec_symm and vec_symm?

I think the unvec_symm and vec_symm functions should have a sqrt(2) factor on the off-diagonal entires to preserve the inner product, i.e., Tr(CX) = vec(C)' * vec(X) (see SCS README).

X = [ X11 X12 ... X1k
      X21 X22 ... X2k
      ...
      Xk1 Xk2 ... Xkk ]

becomes

vec(X) = (X11, sqrt(2)*X21, ..., sqrt(2)*Xk1, X22, sqrt(2)*X32, ..., Xkk).

Register package

@JuliaRegistrator register()

Add projection api

Suggested by @mlubin

Add a projection API, which instead of returning the distance, returns the projected point on the set, i.e.

arg min_y ||v - y||
y in S

ChainRules for MOI functions

We defined derivatives of projections, it might be interesting to have derivatives of MOI functions, especially
ScalarAffineFunction, VectorAffineFunction

Distances and membership for exponential cone

Motivated by #29

From @joaquimg:
This might be the first version of having two distances.
Because a very common distance is the euclidean.
But euclidean requires the projection first.
Our current distance is some epigraphical distance which is good for boolean checking if the point is the the cone.
I don't we should remove the current distance, I would just rename them and have both.
One could be used inside the the other, but I would use euclidean by default. Actually, whenever I possible I think we should use euclidean to have is as uniform as possible.

Originally posted by @joaquimg in #29 (comment)

Should all projection gradient be square matrices?

The projection gradient is d proj[i] / dv[j]. Given that the projection is of identical dimension as the initial point v, the Jacobian / gradient should be a square matrix (preferrably sparse).
ping @Aks1996

Add inplace versions

Inplace versions for projection_on_set, projection_gradient_on_set

Checking if a point is a feasible solution to a JuMP model

I don't know that this actually belongs here.
The is_feasible(constraint_set::MOI.AbstractSet, point) probably does.
but all the stuff that wrangles through a JuMP.Model probably doesn't.

But here is the code that can detect if a set of values meets the constraints.

using JuMP
using MathOptSetDistances
using MathOptSetDistances: DefaultDistance
const MOI = JuMP.MOI
using JuMP.MOI.Utilities



"""
    is_feasible(model, variable_values::AbstractDict)

Detects is `variable_values` reperesents a feasible solution to the constraints of the
`model`.
Any variables present in the model that values are not provided for are assume to take
allowable values.

 - model: a `JuMP.Model`
 - `variable_values` a dictionary mapping `variable_ref => value` where each
   `variable_ref` is a variable that is part of the `model`, and `value` is the value it is
   set to in this candidate solution point.
"""
function is_feasible(model::Model, variable_values::AbstractDict{VariableRef})
    # Check inputs are sensible
    for (variable_ref, variable_val) in variable_values
        check_belongs_to_model(variable_ref, model)
    end

    var_map = construct_varmap(variable_values)

    # now we check all the constraints
    for (f, s) in list_of_constraint_types(model)
        for constraint in constraint_object.(all_constraints(model, f, s))
            constraint_moi_func = moi_function(constraint)
            point = substitute(var_map, constraint_moi_func)
            # We have not provided a value for one of the terms, thus this is not binding
            point === nothing && continue
            constraint_set = moi_set(constraint)
            is_feasible(constraint_set, point) || return false
        end
    end
    return true
end

"""
    is_feasible(constraint_set::MOI.AbstractSet, point)

Determines if the `point` is within the `constraint_set`.
"""
function is_feasible(constraint_set::MOI.AbstractSet, point)
    return distance_to_set(DefaultDistance(), point, constraint_set) <= 0
end


"""
    substitute(varmap, moi_func::MOI.AbstractFunction)

Given a funtion `varmap` that maps from some `MOI.VariableIndex` to a value,
and a MOI Function `moi_func`:
evalutes the `moi_func`, substituting in variable values according to `varmap`/
Returns `nothing` if a `KeyError` is thrown by `varmap` indicating that no value was
provided for one ot the variables that was used.
"""
function substitute(varmap, moi_func::MOI.AbstractFunction)
    try
        return MOI.Utilities.eval_variables(varmap, moi_func)
    catch err
        err isa KeyError || rethrow()
        return nothing
    end
end

"""
    construct_varmap(variable_values::AbstractDict{VariableRef})

Given a `variable_values` a dictionary mapping `variable_ref => value` where each
`variable_ref` is a JuMP variable that is part of the `model`, and `value` being it's value,
this constructs `varmap` a function mapping from `MOI.VariableIndex` to the value.
"""
function construct_varmap(variable_values::AbstractDict{VariableRef})
    moi_variable_values = Dict(JuMP.index(k) => v for (k, v) in variable_values)
    return k->getindex(moi_variable_values, k)
end

Demo

using Test

model = Model()
@variable(model, 0 <= y <= 30)
@variable(model, x <= 6)
@constraint(model, x <= y)

@variable(model, z)
m = @constraint(model, z <= x^2+y^2)

@test is_feasible(model, Dict(x=>1, y=>2))  # Good
@test !is_feasible(model, Dict(x=>2, y=>1)) # x>y
@test !is_feasible(model, Dict(x=>2, y=>100)) # y>30
@test !is_feasible(model, Dict(x=>7, y=>1)) # x > 6
@test !is_feasible(model, Dict(x=>-2, y=>-1)) # y < 0

@test is_feasible(model, Dict(x=>-2)) # y not given.
@test !is_feasible(model, Dict(z=>10, y=>2, x=>1))
@test is_feasible(model, Dict(z=>10, y=>5, x=>1))

PSD gradient wrong?

The gradient of projection onto PSD cone seems to fail for most random negative definite matrices. This had not been tested by previous tests

Error in tests when changing bissection lines

In the bisection function in utils,

mid = (left + right) / 2
if left == mid || right == mid
    return mid
end

We should be able to use isapprox(mid, left) instead of equality since it should be enough as convergence criterion, but the change yields errors in the tests, for example:

Exponential Cone Projections: Test Failed at /home/mbesancon/Documents/projects_julia/MathOptSetDistances.jl/test/projections.jl:126
  Expression: _test_proj_exp_cone_help(x, atol; dual = false)

and

Exponential Cone Projections: Test Failed at /home/mbesancon/Documents/projects_julia/MathOptSetDistances.jl/test/projections.jl:129
  Expression: _test_proj_exp_cone_help(x, atol; dual = true)
Stacktrace:
 [1] top-level scope at /home/mbesancon/Documents/projects_julia/MathOptSetDistances.jl/test/projections.jl:129
 [2] top-level scope at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.5/Test/src/Test.jl:1115
 [3] top-level scope at /home/mbesancon/Documents/projects_julia/MathOptSetDistances.jl/test/projections.jl:83

I left it as-is but might be missing something here.