udellgroup / nys-ip-pmm

Randomized Nyström Preconditioned Interior Point-Proximal Method of Multipliers (Nys-IP-PMM)

This code base uses the Julia Language and DrWatson to make a reproducible scientific project named

Nys-IP-PMM

The project is a companion code for the article "Randomized Nyström Preconditioned Interior Point-Proximal Method of Multipliers", authored by Ya-Chi Chu, Luiz-Rafael Santos, Madeleine Udell. We provide the instructions to reproduce the experiments in the paper.

Preliminary steps

0. Open a Julia console and run the following commands:

   julia> using Pkg
   julia> Pkg.add("DrWatson") # install globally, for using `quickactivate`
   julia> Pkg.activate("path/to/this/project")
   julia> Pkg.instantiate()
   

Large-Scale portfolio optimization experiment (Section 5.1)

1. Run the script scripts/Portfolio/run_portfolio.jl. It will generate the data, run the experiments for the synthetic portfolio optimization problem, and save the results in the folder Portfolio/results/risk_model.
2. Run the script notebooks/Portfolio_analysis/get_demonstrate_results.jl. It will copy the latest 6 results in Portfolio/results/risk_model to notebooks/Portfolio_analysis/demonstrate_results folder for plotting.
3. Run the notebook notebooks/Portfolio_analysis/plotting.ipynb to generate the plot in the paper.

SVM experiments (Section 5.2)

0. Download the required support vector machine (SVM) datasets to a new folder data by running the file scripts/SVM/SVM_data_download.jl.

SVM on all datasets (Section 5.2.1)

1. Run the script scripts/SVM/SVM_run_tests.jl. It will preprocess datasets, run the experiments, and save the results as .csv files under the folder SVM/results/[dataset_name]/IPPMM.
2. Run the script scripts/SVM/SVM_collect_results.jl to collect all the results in the folder SVM/results and save them as a summary .csv file under the folder SVM/results/summary.

Condition numbers at different IP-PMM stages (Section 5.2.2)

0. The condition numbers are saved under notebooks/SVM_analysis/condnum_rank.
1. Run the section "Condition Number v.s. Rank" in the notebook notebooks/SVM_analysis/plotting.ipynb to generate the plot in the paper.

Running time experiments (Section 5.2.3)

1. Run the section "Time v.s. Rank" in the notebook notebooks/SVM_analysis/plotting.ipynb to generate the plot in the paper.

nys-ip-pmm's Issues

IPM fails during initialization possibly due to non-symmetric operator

I've been playing around with this code, and for the most part it runs great. Thanks for all the work.

One issue I have encountered, however, is the code hits a bug during the IPM initialization phase.

Here is how you can reproduce it. Within scripts/Portfolio/run_portfolio.jl I just changed the problem dimension to something smaller:

m, n, k = 50, 80, 10

Here is the REPL error (the line number for file IP-PMM_helper-func.jl may be off since I was using a debugger)

julia> include("scripts/Portfolio/run_portfolio.jl")
Loading the matrices from the file (m = 50, n = 80, k = 10)...
Matrices are loaded from the file.

ERROR: LoadError: DomainError with -3.3660456462482996e9:
sqrt will only return a complex result if called with a complex argument. Try sqrt(Complex(x)).
Stacktrace:
 [1] throw_complex_domainerror(f::Symbol, x::Float64)
   @ Base.Math ./math.jl:33
 [2] sqrt
   @ ./math.jl:677 [inlined]
 [3] cg!(solver::CgSolver{...omitting a lot of text that pertains to the opAAT operator...} , b::Vector{Float64}; M::NystromPreconditionerInverse{Float64}, ldiv::Bool, radius::Float64, linesearch::Bool, atol::Float64, rtol::Float64, itmax::Int64, timemax::Float64, verbose::Int64, history::Bool, callback::Krylov.var"#48#54", iostream::Core.CoreSTDOUT)
   @ Krylov ~/.julia/packages/Krylov/fwLT6/src/cg.jl:231
 [4] obtain_initial_point!(initial_point::IPMVariables{Float64}, input::IPMInput{Float64}; cg_solver::CgSolver{Float64, Float64, Vector{Float64}}, sketchsize::Int64, printlevel::Int64, krylov_tol::Float64)
   @ Main /global/u1/c/cju33/Code/Nys-IP-PMM/src/IP-PMM_helper-func.jl:89
 [5] test_IPPMM(problem_type::portfolio_risk_model{Float64}, problem_name::String, method_P_list::Vector{Any}, tol::Float64; krylov_tol::Float64, maxit::Int64, timed::Bool, saved::Bool)
   @ Main /global/u1/c/cju33/Code/Nys-IP-PMM/src/run_test_utils.jl:41
 [6] top-level scope
   @ /global/u1/c/cju33/Code/Nys-IP-PMM/scripts/Portfolio/run_portfolio.jl:29
 [7] include(fname::String)
   @ Base.MainInclude ./client.jl:478
 [8] top-level scope
   @ REPL[2]:1
in expression starting at /global/u1/c/cju33/Code/Nys-IP-PMM/scripts/Portfolio/run_portfolio.jl:29

The error arises when cg! is called. I suspect it's because the matrix opAAT from the function obtain_initial_point! is not numerically symmetric. In particular, I found the following results:

minimum(real.(eigvals(Matrix(opAAT)))) = 0.823149988532435
minimum(imag.(eigvals(Matrix(opAAT)))) = -3.422629183189052e-5
minimum(imag.(eigvals(Matrix(opAAT)))) = 3.422629183189052e-5
norm(Matrix(opAAT) - Matrix(opAAT)') = 0.00018408569892437092
norm(Matrix(opAAT) - Matrix(opAAT)')/norm(Matrix(opAAT)) = 1.8164265147413125e-20

That imaginary eigenvalue could be the cause of the error within CG, since CG requires a symmetric matrix (hence, real eigenvalues). So I was wondering if you encountered this error while developing the code.