Failed to generate a custom sysimage for JuMP about jump.jl

[joaquimg]

Can you add more detail of what you are trying to do?
Clarabel is another julia package that is not mentioned in your post.

[hlyang1992]

Can you add more detail of what you are trying to do? Clarabel is another julia package that is not mentioned in your post.

I followed the documentation but got the error mentioned above.

Content of model.jl:

using JuMP, HiGHS
model = Model(HiGHS.Optimizer)
set_silent(model)
@variable(model, x >= 0)
@variable(model, 0 <= y <= 3)
@objective(model, Min, 12x + 20y)
@constraint(model, c1, 6x + 8y >= 100)
@constraint(model, c2, 7x + 12y >= 120)
optimize!(model)
open("model.log", "w") do io
    print(io, solution_summary(model; verbose = true))
    return
end

I encountered the error trying to run the following command to generate generate a custom sysimage

using PackageCompiler, Libdl
PackageCompiler.create_sysimage(
    ["JuMP", "HiGHS"],
    sysimage_path = "customimage." * Libdl.dlext,
    precompile_execution_file = "model.jl",
)

[odow]

Are you sure? That's the exact code you ran and you're not using Clarabel anywhere?

Can you provide the full log of every command that you ran?

I cannot reproduce your error:

(pc) pkg> st
Status `/private/tmp/pc/Project.toml`
  [87dc4568] HiGHS v1.7.5
  [4076af6c] JuMP v1.16.0
  [9b87118b] PackageCompiler v2.1.15
  [8f399da3] Libdl

julia> using PackageCompiler, Libdl

julia> PackageCompiler.create_sysimage(
           ["JuMP", "HiGHS"],
           sysimage_path = "customimage." * Libdl.dlext,
           precompile_execution_file = "model.jl",
       )
Precompiling project...
  44 dependencies successfully precompiled in 85 seconds
[ Info: PackageCompiler: Executing /private/tmp/pc/model.jl => /var/folders/bg/dzq_hhvx1dxgy6gb5510pxj80000gn/T/jl_packagecompiler_YrqHLh/jl_rjQ25W
[ Info: PackageCompiler: Done
✔ [04m:57s] PackageCompiler: compiling incremental system image

[hlyang1992]

Are you sure? That's the exact code you ran and you're not using Clarabel anywhere?

Can you provide the full log of every command that you ran?

I cannot reproduce your error:

(pc) pkg> st
Status `/private/tmp/pc/Project.toml`
  [87dc4568] HiGHS v1.7.5
  [4076af6c] JuMP v1.16.0
  [9b87118b] PackageCompiler v2.1.15
  [8f399da3] Libdl

julia> using PackageCompiler, Libdl

julia> PackageCompiler.create_sysimage(
           ["JuMP", "HiGHS"],
           sysimage_path = "customimage." * Libdl.dlext,
           precompile_execution_file = "model.jl",
       )
Precompiling project...
  44 dependencies successfully precompiled in 85 seconds
[ Info: PackageCompiler: Executing /private/tmp/pc/model.jl => /var/folders/bg/dzq_hhvx1dxgy6gb5510pxj80000gn/T/jl_packagecompiler_YrqHLh/jl_rjQ25W
[ Info: PackageCompiler: Done
✔ [04m:57s] PackageCompiler: compiling incremental system image

Thank you for your reply, here is the full log

(base) ➜  tmp julia
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.9.4 (2023-11-14)
 _/ |\__'_|_|_|\__'_|  |  Built by Homebrew (v1.9.4)
|__/                   |

shell> cat model.jl
using JuMP, HiGHS
model = Model(HiGHS.Optimizer)
set_silent(model)
@variable(model, x >= 0)
@variable(model, 0 <= y <= 3)
@objective(model, Min, 12x + 20y)
@constraint(model, c1, 6x + 8y >= 100)
@constraint(model, c2, 7x + 12y >= 120)
optimize!(model)
open("model.log", "w") do io
    print(io, solution_summary(model; verbose = true))
    return
end


shell> ^C

(@v1.9) pkg> st
Status `~/.julia/environments/v1.9/Project.toml`
  [1e616198] COSMO v0.8.8
  [a076750e] CPLEX v1.0.1
  [336ed68f] CSV v0.10.11
  [9961bab8] Cbc v1.2.0
  [61c947e1] Clarabel v0.6.0
  [e2554f3b] Clp v1.0.3
  [a93c6f00] DataFrames v1.6.1
  [87dc4568] HiGHS v1.7.5
  [7073ff75] IJulia v1.24.2
  [4076af6c] JuMP v1.16.0
  [0327d340] MultiObjectiveAlgorithms v1.3.1
  [9b87118b] PackageCompiler v2.1.15
  [91a5bcdd] Plots v1.39.0
  [c3e4b0f8] Pluto v0.19.32
  [438e738f] PyCall v1.96.2
  [1fd47b50] QuadGK v2.9.1
⌃ [82193955] SCIP v0.11.6
  [276daf66] SpecialFunctions v2.3.1
  [f3b207a7] StatsPlots v0.15.6
  [8f399da3] Libdl
Info Packages marked with ⌃ have new versions available and may be upgradable.

(@v1.9) pkg> ^C

julia> using PackageCompiler, Libdl

julia> PackageCompiler.create_sysimage(
           ["JuMP", "HiGHS"],
           sysimage_path = "customimage." * Libdl.dlext,
           precompile_execution_file = "model.jl",
       )
Precompiling project...
  ✗ Clarabel
  0 dependencies successfully precompiled in 9 seconds. 292 already precompiled.

ERROR: The following 1 direct dependency failed to precompile:

Clarabel [61c947e1-3e6d-4ee4-985a-eec8c727bd6e]

Failed to precompile Clarabel [61c947e1-3e6d-4ee4-985a-eec8c727bd6e] to "/Users/hlyang/.julia/compiled/v1.9/Clarabel/jl_VWueix".
ERROR: LoadError: MethodError: no method matching Clarabel.GenPowerConeT(::Vector{Float64}, ::Int64)

Closest candidates are:
  Clarabel.GenPowerConeT(::Vector{Float64}, ::Int32)
   @ Clarabel ~/.julia/packages/Clarabel/YHU2j/src/cones/cone_api.jl:40

Stacktrace:
 [1] __precompile_native()
   @ Clarabel ~/.julia/packages/Clarabel/YHU2j/src/precompile.jl:46
 [2] macro expansion
   @ ~/.julia/packages/Clarabel/YHU2j/src/Clarabel.jl:99 [inlined]
 [3] macro expansion
   @ ~/.julia/packages/SnoopPrecompile/1XXT1/src/SnoopPrecompile.jl:62 [inlined]
 [4] (::Clarabel.var"#65#66")()
   @ Clarabel ~/.julia/packages/Clarabel/YHU2j/src/Clarabel.jl:97
 [5] (::Base.RedirectStdStream)(thunk::Clarabel.var"#65#66", stream::Base.DevNull)
   @ Base ./stream.jl:1429
 [6] top-level scope
   @ ~/.julia/packages/Clarabel/YHU2j/src/Clarabel.jl:97
 [7] include
   @ ./Base.jl:457 [inlined]
 [8] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt128}}, source::Nothing)
   @ Base ./loading.jl:2049
 [9] top-level scope
   @ stdin:3
in expression starting at /Users/hlyang/.julia/packages/Clarabel/YHU2j/src/Clarabel.jl:2
in expression starting at stdin:3
Stacktrace:
 [1] pkgerror(msg::String)
   @ Pkg.Types /usr/local/Cellar/julia/1.9.4/share/julia/stdlib/v1.9/Pkg/src/Types.jl:69
 [2] precompile(ctx::Pkg.Types.Context, pkgs::Vector{Pkg.Types.PackageSpec}; internal_call::Bool, strict::Bool, warn_loaded::Bool, already_instantiated::Bool, timing::Bool, kwargs::Base.Pairs{Symbol, Base.TTY, Tuple{Symbol}, NamedTuple{(:io,), Tuple{Base.TTY}}})
   @ Pkg.API /usr/local/Cellar/julia/1.9.4/share/julia/stdlib/v1.9/Pkg/src/API.jl:1619
 [3] precompile(pkgs::Vector{Pkg.Types.PackageSpec}; io::Base.TTY, kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
   @ Pkg.API /usr/local/Cellar/julia/1.9.4/share/julia/stdlib/v1.9/Pkg/src/API.jl:156
 [4] precompile(pkgs::Vector{Pkg.Types.PackageSpec})
   @ Pkg.API /usr/local/Cellar/julia/1.9.4/share/julia/stdlib/v1.9/Pkg/src/API.jl:145
 [5] precompile(; name::Nothing, uuid::Nothing, version::Nothing, url::Nothing, rev::Nothing, path::Nothing, mode::Pkg.Types.PackageMode, subdir::Nothing, kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
   @ Pkg.API /usr/local/Cellar/julia/1.9.4/share/julia/stdlib/v1.9/Pkg/src/API.jl:171
 [6] precompile()
   @ Pkg.API /usr/local/Cellar/julia/1.9.4/share/julia/stdlib/v1.9/Pkg/src/API.jl:162
 [7] top-level scope
   @ none:1
ERROR: failed process: Process(setenv(`/usr/local/Cellar/julia/1.9.4/bin/julia --color=yes --startup-file=no --pkgimages=no --sysimage=/usr/local/Cellar/julia/1.9.4/lib/julia/sys.dylib -e 'using Pkg; Pkg.precompile()'`,["XPC_FLAGS=0x0", "LSCOLORS=Gxfxcxdxbxegedabagacad", "PATH=/Applications/CPLEX_Studio2211/cplex/bin/x86-64_osx:/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin:/Users/hlyang/opt/miniconda3/bin:/Users/hlyang/opt/miniconda3/condabin", "_CE_M=", "all_proxy=socks5://127.0.0.1:7890", "XPC_SERVICE_NAME=0", "CONDA_PYTHON_EXE=/Users/hlyang/opt/miniconda3/bin/python", "__CF_USER_TEXT_ENCODING=0x1F5:0:0", "TMUX=/private/tmp/tmux-501/default,27087,0", "__CFBundleIdentifier=com.googlecode.iterm2"  …  "_CE_CONDA=", "CONDA_SHLVL=1", "ITERM_SESSION_ID=w0t0p0:5455346F-23A3-41C3-955E-B582A31AA81B", "CONDA_EXE=/Users/hlyang/opt/miniconda3/bin/conda", "HOME=/Users/hlyang", "TERM=screen-256color", "TMUX_PANE=%9", "LS_COLORS=di=1;36:ln=35:so=32:pi=33:ex=31:bd=34;46:cd=34;43:su=30;41:sg=30;46:tw=30;42:ow=30;43", "COLORTERM=truecolor", "OPENBLAS_MAIN_FREE=1"]), ProcessExited(1)) [1]

Stacktrace:
 [1] pipeline_error
   @ ./process.jl:565 [inlined]
 [2] run(::Cmd; wait::Bool)
   @ Base ./process.jl:480
 [3] run
   @ ./process.jl:477 [inlined]
 [4] ensurecompiled(project::String, packages::Vector{String}, sysimage::String)
   @ PackageCompiler ~/.julia/packages/PackageCompiler/Na90o/src/PackageCompiler.jl:278
 [5] create_sysimage(packages::Vector{String}; sysimage_path::String, project::String, precompile_execution_file::String, precompile_statements_file::Vector{String}, incremental::Bool, filter_stdlibs::Bool, cpu_target::String, script::Nothing, sysimage_build_args::Cmd, include_transitive_dependencies::Bool, base_sysimage::Nothing, julia_init_c_file::Nothing, julia_init_h_file::Nothing, version::Nothing, soname::Nothing, compat_level::String, extra_precompiles::String)
   @ PackageCompiler ~/.julia/packages/PackageCompiler/Na90o/src/PackageCompiler.jl:554
 [6] top-level scope
   @ REPL[4]:1

julia>

[odow]

This seems like an issue with Clarabel. I don't know if we can fix in JuMP.

You should instead use a Pkg environment that has the minimal set of packages you need and doesn't include Clarabel.

[hlyang1992]

This seems like an issue with Clarabel. I don't know if we can fix in JuMP.

You should instead use a Pkg environment that has the minimal set of packages you need and doesn't include Clarabel.

Deleting Clarabel worked, thanks.

[odow]

I looked into the details. The Int64/Int32 issue is because of
https://github.com/oxfordcontrol/Clarabel.jl/blob/1f654b0b9a0450673fda7e17f95af5fa9798c1c1/src/Clarabel.jl#L6

Built by Homebrew (v1.9.4)

You're not using the official Julia binaries. Please download the official version of Julia from https://julialang.org/downloads/

Other binaries sources are not supported, because they often compile Julia in a way that causes multiple issues with third-party libraries like BLAS.

[hlyang1992]

I looked into the details. The Int64/Int32 issue is because of https://github.com/oxfordcontrol/Clarabel.jl/blob/1f654b0b9a0450673fda7e17f95af5fa9798c1c1/src/Clarabel.jl#L6

Built by Homebrew (v1.9.4)

You're not using the official Julia binaries. Please download the official version of Julia from https://julialang.org/downloads/

Other binaries sources are not supported, because they often compile Julia in a way that causes multiple issues with third-party libraries like BLAS.

I download the official version of Julia from https://julialang.org/downloads/. The following code works fine. This issue should be caused by the Julia version of Homebrew. Thank you.

PackageCompiler.create_sysimage(
           ["JuMP", "HiGHS", "Clarabel"],
           sysimage_path = "customimage." * Libdl.dlext,
           precompile_execution_file = "model.jl",
       )