stanjulia / stan.jl Goto Github PK
View Code? Open in Web Editor NEWStan.jl illustrates the usage of the 'single method' packages, e.g. StanSample, StanOptimize, etc.
License: MIT License
Stan.jl illustrates the usage of the 'single method' packages, e.g. StanSample, StanOptimize, etc.
License: MIT License
The URL of this package does not match that stored in METADATA.jl.
cc: @goedman
Hi Rob-
I encountered a strange situation in which CMDSTAN_HOME cannot be found on my system. I am using Ubuntu 16.04 and Julia .7. (@info Stan does not provide the version of Stan). First, I tried adding the path to my bashcr file to no avail. Next, I tried to hard code it into Stan.jl also to no avail. I also tried using the set function, but it did not work either.
I believe it is pointing to the correct location:
dfish@dfish-MS-7850:~/cmdstan-2.17.1$ ls
examples LICENSE makefile README.md src test-all.sh
Jenkinsfile make Projects runCmdStanTests.py stan
dfish@dfish-MS-7850:~/cmdstan-2.17.1$ pwd
/home/dfish/cmdstan-2.17.1
Error:
julia> using Pkg
julia> using Stan
[ Info: Recompiling stale cache file /home/dfish/.julia/compiled/v0.7/Stan/zPI1R.ji for Stan [682df890-35be-576f-97d0-3d8c8b33a550]
WARNING: eval from module Base to Stan:
Expr(:->, :x, Expr(:block, #= Symbol("/home/dfish/.julia/packages/Documenter/sN5Qn/src/DocChecks.jl"):650 =#, :y = :have_color, #= Symbol("/home/dfish/.julia/packages/Documenter/sN5Qn/src/DocChecks.jl"):650 =#, Expr(:global, :have_color = :x), #= Symbol("/home/dfish/.julia/packages/Documenter/sN5Qn/src/DocChecks.jl"):650 =#, :y))
** incremental compilation may be broken for this module **
┌ Warning: `warn()` is deprecated, use `@warn` instead.
│ caller = #warn#781(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::Function, ::String) at deprecated.jl:1032
└ @ Base ./deprecated.jl:1032
WARNING: Environment variable CMDSTAN_HOME not found. Use set_cmdstan_home!.
julia> CMDSTAN_HOME
""
julia> set_cmdstan_home!("/home/dfish/cmdstan-2.17.1")
"/home/dfish/cmdstan-2.17.1"
julia> CMDSTAN_HOME
"/home/dfish/cmdstan-2.17.1"
julia> using Stan
julia> Pkg.test("Stan")
Testing Stan
Status `/tmp/tmppcKWvH/Manifest.toml`
[9e28174c] BinDeps v0.8.9
[34da2185] Compat v1.0.1
[ffbed154] DocStringExtensions v0.4.5
[e30172f5] Documenter v0.19.3
[d9be37ee] Homebrew v0.6.4
[682c06a0] JSON v0.19.0
[682df890] Stan v3.0.1
[30578b45] URIParser v0.4.0
[2a0f44e3] Base64 [`~/julia.7/bin/../share/julia/stdlib/v0.7/Base64`]
[ade2ca70] Dates [`~/julia.7/bin/../share/julia/stdlib/v0.7/Dates`]
[8bb1440f] DelimitedFiles [`~/julia.7/bin/../share/julia/stdlib/v0.7/DelimitedFiles`]
[8ba89e20] Distributed [`~/julia.7/bin/../share/julia/stdlib/v0.7/Distributed`]
[b77e0a4c] InteractiveUtils [`~/julia.7/bin/../share/julia/stdlib/v0.7/InteractiveUtils`]
[76f85450] LibGit2 [`~/julia.7/bin/../share/julia/stdlib/v0.7/LibGit2`]
[8f399da3] Libdl [`~/julia.7/bin/../share/julia/stdlib/v0.7/Libdl`]
[37e2e46d] LinearAlgebra [`~/julia.7/bin/../share/julia/stdlib/v0.7/LinearAlgebra`]
[56ddb016] Logging [`~/julia.7/bin/../share/julia/stdlib/v0.7/Logging`]
[d6f4376e] Markdown [`~/julia.7/bin/../share/julia/stdlib/v0.7/Markdown`]
[a63ad114] Mmap [`~/julia.7/bin/../share/julia/stdlib/v0.7/Mmap`]
[44cfe95a] Pkg [`~/julia.7/bin/../share/julia/stdlib/v0.7/Pkg`]
[de0858da] Printf [`~/julia.7/bin/../share/julia/stdlib/v0.7/Printf`]
[3fa0cd96] REPL [`~/julia.7/bin/../share/julia/stdlib/v0.7/REPL`]
[9a3f8284] Random [`~/julia.7/bin/../share/julia/stdlib/v0.7/Random`]
[ea8e919c] SHA [`~/julia.7/bin/../share/julia/stdlib/v0.7/SHA`]
[9e88b42a] Serialization [`~/julia.7/bin/../share/julia/stdlib/v0.7/Serialization`]
[1a1011a3] SharedArrays [`~/julia.7/bin/../share/julia/stdlib/v0.7/SharedArrays`]
[6462fe0b] Sockets [`~/julia.7/bin/../share/julia/stdlib/v0.7/Sockets`]
[2f01184e] SparseArrays [`~/julia.7/bin/../share/julia/stdlib/v0.7/SparseArrays`]
[10745b16] Statistics [`~/julia.7/bin/../share/julia/stdlib/v0.7/Statistics`]
[8dfed614] Test [`~/julia.7/bin/../share/julia/stdlib/v0.7/Test`]
[cf7118a7] UUIDs [`~/julia.7/bin/../share/julia/stdlib/v0.7/UUIDs`]
[4ec0a83e] Unicode [`~/julia.7/bin/../share/julia/stdlib/v0.7/Unicode`]
┌ Warning: `warn()` is deprecated, use `@warn` instead.
│ caller = #warn#781(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::Function, ::String) at deprecated.jl:1032
└ @ Base ./deprecated.jl:1032
WARNING: Environment variable CMDSTAN_HOME not found. Use set_cmdstan_home!.
Running tests for Stan-j0.7-v3.0.0:
CMDSTAN_HOME not set or found.
Skipping all tests that depend on CmdStan!
Testing Stan tests passed
julia> CMDSTAN_HOME
"/home/dfish/cmdstan-2.17.1"
The docstring for stan()
says that the data
and ProjDir
arguments are required, but in the source they're defined as optional positional arguments. Also, the way they're described is confusing since you don't call the method with the argument name (while you do for the keyword arguments, which are described similarly).
I think it would help a lot to have example syntax for all of them. If I get time I'll open a PR but wanted to point this out here since it came up on slack.
@JuliaRegistrator register()
My apologies in advance if these features currently exist. I was wondering if it would be possible to add the ability to modify default settings, such as number of samples per chain and warm up samples. Currently, my workaround is to modify line 58 in sampletype.jl and change the size of the output (e.g. sim1[1:N] where N is the number of samples per chain)
Also, is there a simple method of allocating the chains to each processor? This might be useful for complex models.
Thank you in advance
I'm not sure if this is a problem, but when I did a package update this morning I saw that Stan was updated to 5.0.2. Then a precompile resulted in the following warnings. Just thought I'd mention it.
Michael
(v1.3) pkg> up
Updating registry at ~/.julia/registries/General
Updating git-repo https://github.com/JuliaRegistries/General.git
Resolving package versions...
Installed OpenBLAS_jll ───── v0.3.7+1
Installed CSTParser ──────── v2.0.0
Installed Arpack ─────────── v0.4.0
Installed StatsFuns ──────── v0.9.2
Installed JuliaInterpreter ─ v0.7.5
Installed Stan ───────────── v5.0.2
Installed Arpack_jll ─────── v3.5.0+2
Installed Optim ──────────── v0.19.6
Updating ~/.julia/environments/v1.3/Project.toml
[682df890] ↑ Stan v5.0.1 ⇒ v5.0.2
Updating ~/.julia/environments/v1.3/Manifest.toml
[7d9fca2a] ↑ Arpack v0.3.1 ⇒ v0.4.0
[68821587] + Arpack_jll v3.5.0+2
[00ebfdb7] ↑ CSTParser v1.1.0 ⇒ v2.0.0
[aa1ae85d] ↑ JuliaInterpreter v0.7.4 ⇒ v0.7.5
[4536629a] + OpenBLAS_jll v0.3.7+1
[429524aa] ↑ Optim v0.19.5 ⇒ v0.19.6
[682df890] ↑ Stan v5.0.1 ⇒ v5.0.2
[4c63d2b9] ↑ StatsFuns v0.8.0 ⇒ v0.9.2
(v1.3) pkg> precompile
Precompiling project...
Precompiling Stan
[ Info: Precompiling Stan [682df890-35be-576f-97d0-3d8c8b33a550]
WARNING: Method definition findall(Union{Base.Regex, AbstractString}, AbstractString) in module Base at regex.jl:345 overwritten in module StanBase at /Users/prange/.julia/packages/StanBase/uoK8W/src/utils/findall.jl:20.
** incremental compilation may be fatally broken for this module **
WARNING: Method definition #findall(Any, typeof(Base.findall), Union{Base.Regex, AbstractString}, AbstractString) in module Base overwritten in module StanBase.
** incremental compilation may be fatally broken for this module **
WARNING: Method definition findall(Union{Base.Regex, AbstractString}, AbstractString) in module Base at regex.jl:345 overwritten in module StanBase at /Users/prange/.julia/packages/StanBase/uoK8W/src/utils/findall.jl:20.
** incremental compilation may be fatally broken for this module **
WARNING: Method definition #findall(Any, typeof(Base.findall), Union{Base.Regex, AbstractString}, AbstractString) in module Base overwritten in module StanBase.
** incremental compilation may be fatally broken for this module **
WARNING: Method definition findall(Union{Base.Regex, AbstractString}, AbstractString) in module Base at regex.jl:345 overwritten in module StanBase at /Users/prange/.julia/packages/StanBase/uoK8W/src/utils/findall.jl:20.
** incremental compilation may be fatally broken for this module **
WARNING: Method definition #findall(Any, typeof(Base.findall), Union{Base.Regex, AbstractString}, AbstractString) in module Base overwritten in module StanBase.
** incremental compilation may be fatally broken for this module **
WARNING: Method definition findall(Union{Base.Regex, AbstractString}, AbstractString) in module Base at regex.jl:345 overwritten in module StanBase at /Users/prange/.julia/packages/StanBase/uoK8W/src/utils/findall.jl:20.
** incremental compilation may be fatally broken for this module **
WARNING: Method definition #findall(Any, typeof(Base.findall), Union{Base.Regex, AbstractString}, AbstractString) in module Base overwritten in module StanBase.
** incremental compilation may be fatally broken for this module **
WARNING: Method definition findall(Union{Base.Regex, AbstractString}, AbstractString) in module Base at regex.jl:345 overwritten in module StanBase at /Users/prange/.julia/packages/StanBase/uoK8W/src/utils/findall.jl:20.
** incremental compilation may be fatally broken for this module **
WARNING: Method definition #findall(Any, typeof(Base.findall), Union{Base.Regex, AbstractString}, AbstractString) in module Base overwritten in module StanBase.
** incremental compilation may be fatally broken for this module **
WARNING: Method definition findall(Union{Base.Regex, AbstractString}, AbstractString) in module Base at regex.jl:345 overwritten in module StanBase at /Users/prange/.julia/packages/StanBase/uoK8W/src/utils/findall.jl:20.
** incremental compilation may be fatally broken for this module **
WARNING: Method definition #findall(Any, typeof(Base.findall), Union{Base.Regex, AbstractString}, AbstractString) in module Base overwritten in module StanBase.
** incremental compilation may be fatally broken for this module **
(v1.3) pkg>
Hi Rob-
Perhaps there is already a solution, but I could not figure out how to save either the Stan or Mamba posterior summaries. In either case, I was thinking it might be convenient to automatically save that along with the data and posterior sample files.
All of the cmdstan
tests passed (e.g., make src/test/interface
).
$ ./julia -E 'versioninfo();println(Pkg.installed("Stan"));CMDSTAN_HOME="/usr/local/src/cmdstan";Pkg.test("Stan")'
Julia Version 0.4.0-dev+2177
Commit a2ff1ea* (2014-12-17 01:05 UTC)
Platform Info:
System: Linux (i686-redhat-linux)
CPU: Genuine Intel(R) CPU T2250 @ 1.73GHz
WORD_SIZE: 32
BLAS: libopenblas (DYNAMIC_ARCH NO_AFFINITY Banias)
LAPACK: libopenblas
LIBM: libopenlibm
LLVM: libLLVM-3.3
0.2.0
INFO: Testing Stan
Running tests for Stan-j0.3-v0.2.0:
* test_utilities.jl *
[Lots & lots of ---------> ]WARNING: deprecated syntax "{}"
Environment variable JULIA_SVG_BROWSER not found.
`echo 2`
`echo 3 4`
`echo 5`
1
2
3
4
5
hello
hello
hello
hello
hello
hello
* test_cmdtype.jl *
WARNING: deprecated syntax "(ASCIIString=>Any)[a=>b, ...]" at /home/rick/.julia/v0.4/Stan/test/test_cmdtype.jl:23.
Use "Dict{ASCIIString,Any}(a=>b, ...)" instead.
WARNING: deprecated syntax "(ASCIIString=>Any)[a=>b, ...]" at /home/rick/.julia/v0.4/Stan/test/test_cmdtype.jl:24.
Use "Dict{ASCIIString,Any}(a=>b, ...)" instead.
WARNING: deprecated syntax "(ASCIIString=>Any)[a=>b, ...]" at /home/rick/.julia/v0.4/Stan/test/test_cmdtype.jl:25.
Use "Dict{ASCIIString,Any}(a=>b, ...)" instead.
WARNING: deprecated syntax "(ASCIIString=>Any)[a=>b, ...]" at /home/rick/.julia/v0.4/Stan/test/test_cmdtype.jl:26.
Use "Dict{ASCIIString,Any}(a=>b, ...)" instead.
File /home/rick/.julia/v0.4/Stan/test/tmp/bernoulli.stan will be updated.
name = "bernoulli"
nchains = 4
update = 1000
adapt = 1000
thin = 1
monitors = ASCIIString[]
model_file = "bernoulli.stan"
data_file = ""
output = Output()
file = ""
diagnostics_file = ""
refresh = 100
Optimize(Stan.Lbfgs(0.001,1.0e-8,1.0e-8,1.0e-8,5), 2000, false)
name = "bernoulli"
nchains = 4
update = 1000
adapt = 1000
thin = 1
monitors = ASCIIString[]
model_file = "bernoulli.stan"
data_file = ""
output = Output()
file = ""
diagnostics_file = ""
refresh = 100
method = Optimize()
algorithm = Lbfgs()
init_alpha = 0.001
tol_obj = 1.0e-8
tol_grad = 1.0e-8
tol_param = 1.0e-8
history_size = 5
iterations = 2000
save_iterations = false
Sample(1000, 1000, false, 1, Stan.Adapt(true,0.05,0.8,0.75,10.0,75,50,25), Stan.Hmc(Stan.Nuts(10),Stan.diag_e(),1.0,1.0))
method = Sample()
num_samples = 1000
num_warmup = 1000
save_warmup = false
thin = 1
algorithm = HMC()
engine = NUTS()
max_depth = 10
metric = Stan.diag_e
stepsize = 1.0
stepsize_jitter = 1.0
adapt = Adapt()
gamma = 0.05
delta = 0.8
kappa = 0.75
t0 = 10.0
init_buffer = 75
term_buffer = 50
window = 25
Optimize(Stan.Bfgs(0.001,1.0e-9,1.0e-8,1.0e-8), 2000, true)
method = Optimize()
algorithm = Bfgs()
init_alpha = 0.001
tol_obj = 1.0e-9
tol_grad = 1.0e-8
tol_param = 1.0e-8
iterations = 2000
save_iterations = true
Diagnose(Stan.Gradient(1.0e-6,1.0e-7))
method = Diagnose()
diagnostic = Gradient()
epsilon = 1.0e-6
error = 1.0e-7
CMDSTAN_HOME found! Try to run bernoulli.
* test_bernoulli.jl *
Moving to directory: /home/rick/.julia/v0.4/Stan/Examples/Bernoulli
File /home/rick/.julia/v0.4/Stan/Examples/Bernoulli/tmp/bernoulli.stan will be updated.
Stanmodel that will be used:
name = "bernoulli"
nchains = 4
update = 1200
adapt = 1000
thin = 2
monitors = ASCIIString[]
model_file = "bernoulli.stan"
data_file = ""
output = Output()
file = ""
diagnostics_file = ""
refresh = 100
method = Sample()
num_samples = 1200
num_warmup = 1000
save_warmup = false
thin = 1
algorithm = HMC()
engine = NUTS()
max_depth = 10
metric = Stan.diag_e
stepsize = 1.0
stepsize_jitter = 1.0
adapt = Adapt()
gamma = 0.05
delta = 0.8
kappa = 0.75
t0 = 10.0
init_buffer = 75
term_buffer = 50
window = 25
Input observed data dictionary:
4-element Array{Dict{ASCIIString,Any},1}:
Dict{ASCIIString,Any}("N"=>10,"y"=>[0,1,0,1,0,0,0,0,0,1])
Dict{ASCIIString,Any}("N"=>10,"y"=>[0,1,0,0,0,0,1,0,0,1])
Dict{ASCIIString,Any}("N"=>10,"y"=>[0,0,0,0,0,0,1,0,1,1])
Dict{ASCIIString,Any}("N"=>10,"y"=>[0,0,0,1,0,0,0,1,0,1])
--- Translating Stan model to C++ code ---
bin/stanc /home/rick/.julia/v0.4/Stan/Examples/Bernoulli/tmp/bernoulli.stan --o=/home/rick/.julia/v0.4/Stan/Examples/Bernoulli/tmp/bernoulli.cpp --no_main
Model name=bernoulli_model
Input file=/home/rick/.julia/v0.4/Stan/Examples/Bernoulli/tmp/bernoulli.stan
Output file=/home/rick/.julia/v0.4/Stan/Examples/Bernoulli/tmp/bernoulli.cpp
--- Linking C++ model ---
g++ -DBOOST_RESULT_OF_USE_TR1 -DBOOST_NO_DECLTYPE -DBOOST_DISABLE_ASSERTS -I src -I stan/src -isystem stan/lib/eigen_3.2.0 -isystem stan/lib/boost_1.54.0 -Wall -pipe -DEIGEN_NO_DEBUG -Wno-unused-local-typedefs -lpthread -O3 -o /home/rick/.julia/v0.4/Stan/Examples/Bernoulli/tmp/bernoulli src/cmdstan/main.cpp -include /home/rick/.julia/v0.4/Stan/Examples/Bernoulli/tmp/bernoulli.cpp -Lbin -lstan
ErrorException("failed processes:\n Process(`./bernoulli stan.sample num_samples=1200 num_warmup=1000 save_warmup=0 thin=1 stan.adapt engaged=1 gamma=0.05 delta=0.8 kappa=0.75 t0=10.0 init_buffer=75 term_buffer=50 window=25 algorithm=stan.hmc engine=stan.nuts max_depth=10 metric=Stan.diag_e stepsize=1.0 stepsize_jitter=1.0 stan.random seed=-1 init=2 id=1 data file=bernoulli_1.data.R output file=bernoulli_samples_1.csv refresh=100`, ProcessExited(64)) [64]\n Process(`./bernoulli stan.sample num_samples=1200 num_warmup=1000 save_warmup=0 thin=1 stan.adapt engaged=1 gamma=0.05 delta=0.8 kappa=0.75 t0=10.0 init_buffer=75 term_buffer=50 window=25 algorithm=stan.hmc engine=stan.nuts max_depth=10 metric=Stan.diag_e stepsize=1.0 stepsize_jitter=1.0 stan.random seed=-1 init=2 id=2 data file=bernoulli_2.data.R output file=bernoulli_samples_2.csv refresh=100`, ProcessExited(64)) [64]\n Process(`./bernoulli stan.sample num_samples=1200 num_warmup=1000 save_warmup=0 thin=1 stan.adapt engaged=1 gamma=0.05 delta=0.8 kappa=0.75 t0=10.0 init_buffer=75 term_buffer=50 window=25 algorithm=stan.hmc engine=stan.nuts max_depth=10 metric=Stan.diag_e stepsize=1.0 stepsize_jitter=1.0 stan.random seed=-1 init=2 id=3 data file=bernoulli_3.data.R output file=bernoulli_samples_3.csv refresh=100`, ProcessExited(64)) [64]\n Process(`./bernoulli stan.sample num_samples=1200 num_warmup=1000 save_warmup=0 thin=1 stan.adapt engaged=1 gamma=0.05 delta=0.8 kappa=0.75 t0=10.0 init_buffer=75 term_buffer=50 window=25 algorithm=stan.hmc engine=stan.nuts max_depth=10 metric=Stan.diag_e stepsize=1.0 stepsize_jitter=1.0 stan.random seed=-1 init=2 id=4 data file=bernoulli_4.data.R output file=bernoulli_samples_4.csv refresh=100`, ProcessExited(64)) [64]")
CMDSTAN_HOME found, but CmdStan not installed properly.
LoadError("/home/rick/.julia/v0.4/Stan/test/test_bernoulli.jl",10,LoadError("/home/rick/.julia/v0.4/Stan/Examples/Bernoulli/bernoulli.jl",44,MethodError(size,(nothing,1))))
No simulation runs have been performed.
INFO: Stan tests passed
INFO: No packages to install, update or remove
nothing
Interestingly, with a few commandline param changes ...
$ pwd
/home/rick/.julia/v0.4/Stan/Examples/Bernoulli/tmp
$ ./bernoulli sample num_samples=1200 num_warmup=1000 save_warmup=0 thin=1 adapt engaged=1 gamma=0.05 delta=0.8 kappa=0.75 t0=10.0 init_buffer=75 term_buffer=50 window=25 algorithm=hmc engine=nuts max_depth=10 metric=diag_e stepsize=1.0 stepsize_jitter=1.0 random seed=-1 init=2 id=1 data file=bernoulli_1.data.R output file=bernoulli_samples_1.csv refresh=100
method = sample (Default)
sample
num_samples = 1200
num_warmup = 1000 (Default)
save_warmup = 0 (Default)
thin = 1 (Default)
adapt
engaged = 1 (Default)
gamma = 0.050000000000000003 (Default)
delta = 0.80000000000000004 (Default)
kappa = 0.75 (Default)
t0 = 10 (Default)
init_buffer = 75 (Default)
term_buffer = 50 (Default)
window = 25 (Default)
algorithm = hmc (Default)
hmc
engine = nuts (Default)
nuts
max_depth = 10 (Default)
metric = diag_e (Default)
stepsize = 1 (Default)
stepsize_jitter = 1
id = 1
data
file = bernoulli_1.data.R
init = 2 (Default)
random
seed = 1566406245
output
file = bernoulli_samples_1.csv
diagnostic_file = (Default)
refresh = 100 (Default)
Gradient evaluation took 0 seconds
1000 transitions using 10 leapfrog steps per transition would take 0 seconds.
Adjust your expectations accordingly!
Iteration: 1 / 2200 [ 0%] (Warmup)
Iteration: 100 / 2200 [ 4%] (Warmup)
Iteration: 200 / 2200 [ 9%] (Warmup)
Iteration: 300 / 2200 [ 13%] (Warmup)
Iteration: 400 / 2200 [ 18%] (Warmup)
Iteration: 500 / 2200 [ 22%] (Warmup)
Iteration: 600 / 2200 [ 27%] (Warmup)
Iteration: 700 / 2200 [ 31%] (Warmup)
Iteration: 800 / 2200 [ 36%] (Warmup)
Iteration: 900 / 2200 [ 40%] (Warmup)
Iteration: 1000 / 2200 [ 45%] (Warmup)
Iteration: 1001 / 2200 [ 45%] (Sampling)
Iteration: 1100 / 2200 [ 50%] (Sampling)
Iteration: 1200 / 2200 [ 54%] (Sampling)
Iteration: 1300 / 2200 [ 59%] (Sampling)
Iteration: 1400 / 2200 [ 63%] (Sampling)
Iteration: 1500 / 2200 [ 68%] (Sampling)
Iteration: 1600 / 2200 [ 72%] (Sampling)
Iteration: 1700 / 2200 [ 77%] (Sampling)
Iteration: 1800 / 2200 [ 81%] (Sampling)
Iteration: 1900 / 2200 [ 86%] (Sampling)
Iteration: 2000 / 2200 [ 90%] (Sampling)
Iteration: 2100 / 2200 [ 95%] (Sampling)
Iteration: 2200 / 2200 [100%] (Sampling)
# Elapsed Time: 0.02 seconds (Warm-up)
# 0.05 seconds (Sampling)
# 0.07 seconds (Total)
Hello:
I'm porting, mostly done, the matlab code of PSIS, PSIS-LOO which implement Pareto smoothed importance sampling (PSIS) and PSIS leave-one-out cross-validation. I wonder if you are interested in merge it here or maybe go and try to merge in Mamba.jl.
Regards.
First of all, thanks for putting this up. It is greatly appreciated.
This might be more a question than an issue, but in any case. When using the stan.jl module on input data consisting of ~200K floating point values per thread, I observe a slow down due to the writing of the data into the <model>_<i>.data.R
files. When I manually monitor the tmp/
directory I estimate that julia spends 1-2 minutes on writing each of these files (the files are approx 1.5 MB each and I use a 1 year old macbook pro). Is this expected behaviour? This seems really slow to me, but there might be something else going on?
I can install the Julia Stan package without problems now but when I try the Bernoulli example I get:
julia> sim1 = stan(stanmodel, bernoullidata, ProjDir, diagnostics=false, CmdStanDir=CMDSTAN_HOME);
chdir /Users/feldt/usr/local/cmdstan-2.6.0: no such file or directory (ENOENT)
and there is indeed no such directory. To solve this I downloaded latest cmdstan from here:
https://github.com/stan-dev/cmdstan/releases
unpacked it, ran
make build
and then in julia set the CMDSTAN_HOME variable to where I had unpacked cmdstan. After this I could successfully run the bernoulli.jl example from Stan.jl.
Maybe updating the README to make these steps clear? Or did I miss something?
I've been trying to determine the cause of an error message:
ErrorException("failed processes:\n Process(./odePlatelets sample num_samples=1000 num_warmup=1000 save_warmup=0 thin=1 adapt engaged=1 gamma=0.05 delta=0.8 kappa=0.75 t0=10.0 init_buffer=75 term_buffer=50 window=25 algorithm=hmc engine=nuts max_depth=10 metric=diag_e stepsize=1.0 stepsize_jitter=1.0 random seed=-1 init=2 id=1 data file=odePlatelets_1.data.R output file=odePlatelets_samples_1.csv diagnostic_file=odePlatelets_diagnostics_1.csv refresh=100
, ProcessExited(70)) [70]\
What does the 70 represent?
Just tried the example in the README and things work but I get this warning:
julia> names(sim1) |> display
WARNING: names(v) is deprecated, use fieldnames(v) instead.
so maybe update the README?
PackageEvaluator.jl is a script that runs nightly. It attempts to load all Julia packages and run their tests (if available) on both the stable version of Julia (0.3) and the nightly build of the unstable version (0.4). The results of this script are used to generate a package listing enhanced with testing results.
Tests pass.
Package doesn't load.
Tests pass.
means that PackageEvaluator found the tests for your package, executed them, and they all passed.
Package doesn't load.
means that PackageEvaluator did not find tests for your package. Additionally, trying to load your package with using
failed.
This issue was filed because your testing status became worse. No additional issues will be filed if your package remains in this state, and no issue will be filed if it improves. If you'd like to opt-out of these status-change messages, reply to this message saying you'd like to and @IainNZ will add an exception. If you'd like to discuss PackageEvaluator.jl please file an issue at the repository. For example, your package may be untestable on the test machine due to a dependency - an exception can be added.
Test log:
>>> 'Pkg.add("Stan")' log
INFO: Installing Stan v0.0.2
INFO: Package database updated
INFO: METADATA is out-of-date a you may not have the latest version of Stan
INFO: Use `Pkg.update()` to get the latest versions of your packages
>>> 'using Stan' log
ERROR: DataFrames not found
in require at loading.jl:47
in include at ./boot.jl:245
in include_from_node1 at ./loading.jl:128
in reload_path at loading.jl:152
in _require at loading.jl:67
in require at loading.jl:51
in include at ./boot.jl:245
in include_from_node1 at loading.jl:128
in process_options at ./client.jl:285
in _start at ./client.jl:354
in _start_3B_3500 at /home/idunning/julia04/usr/bin/../lib/julia/sys.so
while loading /home/idunning/pkgtest/.julia/v0.4/Stan/src/Stan.jl, in expression starting on line 4
while loading /home/idunning/pkgtest/.julia/v0.4/Stan/testusing.jl, in expression starting on line 1
>>> test log
no tests to run
>>> end of log
I'm trying to use Stan in generative model to simulate stochastic processes and I get a Return code -5
When I check the run.log file I find the following error:
Must use algorithm=fixed_param for model that has no parameters
If I try to add the fixed param
stancode.jl 76 got unsupported keyword arguments "algorithm"
Can the algorithm keyword option be added to the Julia code? This option is available to PyStan and RStan users.
I would like to do it myself, but I'm just learning Julia and don't feel competent enough to give it a go.
Thanks.
julia> isfile(joinpath(ENV["CMDSTAN_HOME"], "bin/stanc"))
true
julia> VERSION
v"0.6.0-pre.beta.460"
julia> Pkg.build("Stan")
INFO: Building Rmath
INFO: Building Cairo
INFO: Building Stan
============================================================================[ ERROR: Stan ]============================================================================
LoadError: ArgumentError: Module Homebrew not found in current path.
Run `Pkg.add("Homebrew")` to install the Homebrew package.
while loading /home/tamas/.julia/v0.6/Stan/deps/build.jl, in expression starting on line 1
=======================================================================================================================================================================
===========================================================================[ BUILD ERRORS ]============================================================================
WARNING: Stan had build errors.
- packages with build errors remain installed in /home/tamas/.julia/v0.6
- build the package(s) and all dependencies with `Pkg.build("Stan")`
- build a single package by running its `deps/build.jl` script
=======================================================================================================================================================================
Same happens on v0.5
. Homebrew.jl
does not seem to support Linux. Can Stan.jl
be used on Linux?
Data vectors containing a single element get written to the R input data file like a scalar, according to https://github.com/goedman/Stan.jl/blob/master/src/stancode.jl#L182.
This causes a dimensionality error if I have just one data point:
std::exception::what: mismatch in number dimensions declared and found in context; processing stage=data initialization; variable name=y; dims declared=(1); dims found=()
Simpy removing the special treatment of single element vectors would solve this. Is there a reason for not doing this? Thanks!
When running the example, stan complains
sim1 = Stan.stan(m, data)
...
Exception: variable does not exist; processing stage=data initialization; variable name=N; base type=int
Diagnostic information:
Dynamic exception type: std::runtime_error
std::exception::what: variable does not exist; processing stage=data initialization; variable name=N; base type=int
ErrorException("failed processes:\n Process(`./lin.reg sample num_samples=1000 num_warmup=1000 save_warmup=0 thin=1 adapt engaged=1 gamma=0.05 delta=0.8 kappa=0.75 t0=10.0 init_buffer=75 term_buffer=50 window=25 algorithm=hmc engine=nuts max_depth=10 metric=diag_e stepsize=1.0 stepsize_jitter=1.0 random seed=-1 init=2 output file=lin.reg_samples_1.csv refresh=100`, ProcessExited(255)) [255]\n Process(`./lin.reg sample num_samples=1000 num_warmup=1000 save_warmup=0 thin=1 adapt engaged=1 gamma=0.05 delta=0.8 kappa=0.75 t0=10.0 init_buffer=75 term_buffer=50 window=25 algorithm=hmc engine=nuts max_depth=10 metric=diag_e stepsize=1.0 stepsize_jitter=1.0 random seed=-1 init=2 output file=lin.reg_samples_2.csv refresh=100`, ProcessExited(255)) [255]\n Process(`./lin.reg sample num_samples=1000 num_warmup=1000 save_warmup=0 thin=1 adapt engaged=1 gamma=0.05 delta=0.8 kappa=0.75 t0=10.0 init_buffer=75 term_buffer=50 window=25 algorithm=hmc engine=nuts max_depth=10 metric=diag_e stepsize=1.0 stepsize_jitter=1.0 random seed=-1 init=2 output file=lin.reg_samples_3.csv refresh=100`, ProcessExited(255)) [255]\n Process(`./lin.reg sample num_samples=1000 num_warmup=1000 save_warmup=0 thin=1 adapt engaged=1 gamma=0.05 delta=0.8 kappa=0.75 t0=10.0 init_buffer=75 term_buffer=50 window=25 algorithm=hmc engine=nuts max_depth=10 metric=diag_e stepsize=1.0 stepsize_jitter=1.0 random seed=-1 init=2 output file=lin.reg_samples_4.csv refresh=100`, ProcessExited(255)) [255]")
I hit this problem with my own code and the same problem. It is due to the fact that the data
are just one dictionary and not a vector of dictionaries. It works if I do Stan.stan(m, [data,data, data,data]) for the default 4 chains. That shouldn't be necessary and there is code in
src/stancode.jlto deal with it but obviously that fails. There is no
data file=XXX.R` in the call to cmdstan. I know I updated the readme and it worked then but it doesn't anymore. Would be good to check this behavior in the tests. The bernoulli example now has different data input for every chain.
Hi Rob, I believe I've found a bug in read_stanfit_samples
.
Minimal example:
const simplecode = "
data {real sigma;}
parameters {real y;}
model {y ~ normal(0,sigma);}
"
stanmodel = Stanmodel(Sample(thin=5), name="simple", model=simplecode);
sim = stan(stanmodel, [Dict("sigma" => 1.)], CmdStanDir=CMDSTAN_HOME);
gives
InexactError()
in peekchar(::IOStream) at ./iostream.jl:269
in #skipchars#287(::Char, ::Function, ::IOStream, ::Base.UTF8proc.#isspace) at ./iostream.jl:286
in (::Base.#kw##skipchars)(::Array{Any,1}, ::Base.#skipchars, ::IOStream, ::Function) at ./<missing>:0
in read_stanfit_samples(::Stan.Stanmodel, ::Bool, ::Bool) at /Users/jonalm/.julia/v0.5/Stan/src/stancode.jl:382
in #stan#16(::Bool, ::Bool, ::String, ::Function, ::Stan.Stanmodel, ::Array{Dict{String,Float64},1}, ::String) at /Users/jonalm/.julia/v0.5/Stan/src/stancode.jl:147
in (::Stan.#kw##stan)(::Array{Any,1}, ::Stan.#stan, ::Stan.Stanmodel, ::Array{Dict{String,Float64},1}, ::String) at ./<missing>:0 (repeats 2 times)
I've tried to understand the read_stanfit_samples
function, but I get lost after the sample file is opened on line 362 in stancode.jl.
I'm running julia 0.5 on Ubuntu 16.04. Installing CmdStan was no problem but I got this error with Stan.jl
julia> Pkg.add("Stan")
INFO: Cloning cache of Stan from https://github.com/goedman/Stan.jl.git
INFO: Installing Stan v1.0.2
INFO: Building Rmath
INFO: Building Cairo
INFO: Building Stan
==========================================================[ ERROR: Stan ]==========================================================
LoadError: ArgumentError: Module Homebrew not found in current path.
Run `Pkg.add("Homebrew")` to install the Homebrew package.
while loading /home/beaujean/.julia/v0.5/Stan/deps/build.jl, in expression starting on line 1
===================================================================================================================================
=========================================================[ BUILD ERRORS ]==========================================================
WARNING: Stan had build errors.
- packages with build errors remain installed in /home/beaujean/.julia/v0.5
- build the package(s) and all dependencies with `Pkg.build("Stan")`
- build a single package by running its `deps/build.jl` script
===================================================================================================================================
INFO: Package database updated
Now I read the Readme that says Stan.jl is mostly tested on Mac but why would you require Homebrew for any platform? I guess it only makes sense to have on Mac OS.
Since your build.jl
should do nothing if not on Mac OS, I guess you just have to avoid the first line
using BinDeps, Homebrew
that requires homebrew regardless of the operating system.
Hi,
In issue #39 I started a discussion on the future of Stan.jl by asking several questions. Hiding the discussion inside another issue is not right, so I'm moving it to its own issue.
Before tagging Stan.jl v"1.1.0" I have 3 items on my list of things to do:
With respect to 3 I'm leaning towards adding a new (multiple dispatch or keyword?) argument to stan() to make this work similar to data, e.g. sim1 = stan(stanmodel, bernoullidata, bernoulliinit)
in the BernoulliInitTheta example.
Other questions (and also based on Michael's response in #39 for item 5):
I asked folks for feedback on these and other topics such as "do folks like the coupling with Mamba diagnostics and Mamba graphics?" and "how important is access to Stan functions on C++ level?".
Over the last year I have certainly seen more R focus in the Stan team, how important is that? Hence item 8 and possibly item 4.
After 2 years is it time to do major surgery on Stan.jl? I'm also pretty busy right now, so no promises, even if I could tackle some of the larger topics.
Regards,
Rob
@JuliaRegistrator register()
The Julia function stan_summary
tries to call an executable $(CmdStanDir)/bin/stansummary
but that executable was not built in my copy of cmdstan-2.6.0
. Am I missing something?
@JuliaRegistrator register()
@JuliaRegistrator register()
Most of the links in the readme.md file do not work.
It would be nice if this could be fixed.
Please open an issue here or send email to [email protected].
Attempting to delete the tmp directory in Pkg.test("Stan") returns wit a EPERM rejection.
I am now using optimize method to do stan. But function read_stanfit
in read_stan_files.jl
does not parse the output csv properly.
Code File: unfair_dice_newton.jl
using Printf
using Stan, Mamba
model_script = "
data {
int<lower=0> N;
real y[N];
}
parameters{
real<lower=0, upper=1> p;
}
model{
target += log(1); // assume flat prior
for(i in 1:N) {
if(y[i] == 1 || y[i] == 6) {
target += log(p);
} else {
target += log((1.0 - 2.0 * p) / 4.0);
}
}
}
"
##### Making data begin #####
N = 1000
p = 0.2
q = (1.0 - 2p) / 4.0
x = rand(N)
p_hist = [p, q, q, q, q, p]
for i = 2:length(p_hist)
p_hist[i] += p_hist[i-1]
end
p_hist[end] = 1.0
function rnd2pt(x)
for i = 1:length(p_hist)
if x < p_hist[i]
return i
end
end
end
y = rnd2pt.(x)
##### Making data end #####
data = Dict(
"N" => N,
"y" => y
)
@printf("Now we are going to build stan model...\n")
stanmodel = Stanmodel(
Stan.Optimize();
name="Dice",
nchains=4,
model=model_script,
useMamba=true
)
stanmodel |> display
rc, sim = stan(
stanmodel,
[data],
"tmp"
;
CmdStanDir=ENV["CMDSTAN_HOME"],
init=Dict(
"p" => 0.1
)
)
if rc == 0
println(sim[1]["p"])
else
println("There are errors!!")
end
ERROR: LoadError: ArgumentError: cannot parse "#" as Float64
Stacktrace:
[1] _parse_failure(::Type, ::SubString{String}, ::Int64, ::Int64) at ./parse.jl:357 (repeats 2 times)
[2] tryparse_internal at ./parse.jl:353 [inlined]
[3] parse at ./parse.jl:363 [inlined]
[4] _broadcast_getindex_evalf at ./broadcast.jl:574 [inlined]
[5] _broadcast_getindex at ./broadcast.jl:557 [inlined]
[6] getindex at ./broadcast.jl:507 [inlined]
[7] macro expansion at ./broadcast.jl:838 [inlined]
[8] macro expansion at ./simdloop.jl:73 [inlined]
[9] copyto! at ./broadcast.jl:837 [inlined]
[10] copyto! at ./broadcast.jl:792 [inlined]
[11] copy at ./broadcast.jl:768 [inlined]
[12] materialize at ./broadcast.jl:748 [inlined]
[13] read_stanfit(::Stanmodel) at /home/tienyiah/.julia/packages/Stan/krRNa/src/utilities/read_stan_files.jl:109
[14] #stan#4(::Dict{String,Float64}, ::Bool, ::Bool, ::String, ::Function, ::Stanmodel, ::Array{Dict{String,Any},1}, ::String) at /home/tienyiah/.julia/packages/Stan/krRNa/src/main/stancode.jl:205
[15] (::getfield(Stan, Symbol("#kw##stan")))(::NamedTuple{(:CmdStanDir, :init),Tuple{String,Dict{String,Float64}}}, ::typeof(stan), ::Stanmodel, ::Array{Dict{String,Any},1}, ::String) at ./none:0
[16] top-level scope at none:0
[17] include at ./boot.jl:317 [inlined]
[18] include_relative(::Module, ::String) at ./loading.jl:1041
[19] include(::Module, ::String) at ./sysimg.jl:29
[20] exec_options(::Base.JLOptions) at ./client.jl:229
[21] _start() at ./client.jl:421
in expression starting at /home/tienyiah/projects/stan_practice/unfair_dice_newton.jl:66
After look into the code, I figured out that the computational part works fine. But when reading the output csv data file tmp/Dice_samples_1.csv
, it tries to read the last few lines which are line comments:
File: tmp/Dice_samples_1.csv
# stan_version_major = 2
# stan_version_minor = 18
# stan_version_patch = 0
# model = Dice_model
...
...
refresh = 100 (Default)
lp__,accept_stat__,stepsize__,treedepth__,n_leapfrog__,divergent__,energy__,p
# Adaptation terminated
# Step size = 0.981302
# Diagonal elements of inverse mass matrix:
# 0.00280974
-1788.04,0.579651,0.981302,1,3,0,1789.78,0.195009
-1788.38,0.748201,0.981302,2,3,0,1789.73,0.185326
...
...
-1787.98,0.635984,0.981302,2,3,0,1790.76,0.19273
-1787.98,0.877663,0.981302,1,3,0,1788.42,0.19273
-1788.3,0.922961,0.981302,2,3,0,1788.49,0.19844
-1787.98,1,0.981302,1,3,0,1788.24,0.192878
#
# Elapsed Time: 0.25 seconds (Warm-up)
# 0.27 seconds (Sampling)
# 0.52 seconds (Total)
#
I successfully avoid the problem by uncommenting 99 line of the file utilities/read_stan_files.jl
. But I am not sure if this is the correct way to fix it.
97 while true
98 j += 1
99 #skipchars(isspace, instream, linecomment='#') # This line should not be commented
100 line = Unicode.normalize(readline(instream), newline2lf=true)
101 flds = Float64[]
102 if eof(instream) && length(line) < 2
103 #println("EOF detected")
104 close(instream)
105 #return(tdict)
106 break
107 else
108 #println(split(strip(line)))
109 flds = parse.(Float64, (split(strip(line), ","))) # This line throws the error
I am currently using Stan v4.0.0. Below is my julia info
Julia Version 1.0.1
Commit 0d713926f8 (2018-09-29 19:05 UTC)
Platform Info:
OS: Linux (x86_64-pc-linux-gnu)
CPU: Intel(R) Core(TM) i7-4790 CPU @ 3.60GHz
WORD_SIZE: 64
LIBM: libopenlibm
LLVM: libLLVM-6.0.0 (ORCJIT, haswell)
Since updating to Julia .4, I've encountered numerous problems. I suspect that the problems might be related to the failed package test, which gives the following error message:
INFO: Testing Stan
Running tests for Stan-j0.3-v0.2.1:
test_utilities.jl *
ERROR: LoadError: could not open file /Users/chrisfisher/.julia/v0.4/Cairo/src/../deps/deps.jl
in include at /Applications/Julia-0.4.0.app/Contents/Resources/julia/lib/julia/sys.dylib
in include_from_node1 at /Applications/Julia-0.4.0.app/Contents/Resources/julia/lib/julia/sys.dylib
in include at /Applications/Julia-0.4.0.app/Contents/Resources/julia/lib/julia/sys.dylib
in include_from_node1 at /Applications/Julia-0.4.0.app/Contents/Resources/julia/lib/julia/sys.dylib
[inlined code] from none:2
in anonymous at no file:0
in process_options at /Applications/Julia-0.4.0.app/Contents/Resources/julia/lib/julia/sys.dylib
in _start at /Applications/Julia-0.4.0.app/Contents/Resources/julia/lib/julia/sys.dylib
while loading /Users/chrisfisher/.julia/v0.4/Cairo/src/Cairo.jl, in expression starting on line 7
ERROR: LoadError: LoadError: Failed to precompile Cairo to /Users/chrisfisher/.julia/lib/v0.4/Cairo.ji
in error at /Applications/Julia-0.4.0.app/Contents/Resources/julia/lib/julia/sys.dylib
in compilecache at loading.jl:383
in require at /Applications/Julia-0.4.0.app/Contents/Resources/julia/lib/julia/sys.dylib
in include at /Applications/Julia-0.4.0.app/Contents/Resources/julia/lib/julia/sys.dylib
in include_from_node1 at /Applications/Julia-0.4.0.app/Contents/Resources/julia/lib/julia/sys.dylib
in include at /Applications/Julia-0.4.0.app/Contents/Resources/julia/lib/julia/sys.dylib
in include_from_node1 at /Applications/Julia-0.4.0.app/Contents/Resources/julia/lib/julia/sys.dylib
[inlined code] from none:2
in anonymous at no file:0
in process_options at /Applications/Julia-0.4.0.app/Contents/Resources/julia/lib/julia/sys.dylib
in _start at /Applications/Julia-0.4.0.app/Contents/Resources/julia/lib/julia/sys.dylib
while loading /Users/chrisfisher/.julia/v0.4/Compose/src/cairo_backends.jl, in expression starting on line 4
while loading /Users/chrisfisher/.julia/v0.4/Compose/src/Compose.jl, in expression starting on line 141
ERROR: LoadError: Failed to precompile Compose to /Users/chrisfisher/.julia/lib/v0.4/Compose.ji
in error at /Applications/Julia-0.4.0.app/Contents/Resources/julia/lib/julia/sys.dylib
in compilecache at loading.jl:383
in require at /Applications/Julia-0.4.0.app/Contents/Resources/julia/lib/julia/sys.dylib
in include at /Applications/Julia-0.4.0.app/Contents/Resources/julia/lib/julia/sys.dylib
in include_from_node1 at /Applications/Julia-0.4.0.app/Contents/Resources/julia/lib/julia/sys.dylib
[inlined code] from none:2
in anonymous at no file:0
in process_options at /Applications/Julia-0.4.0.app/Contents/Resources/julia/lib/julia/sys.dylib
in _start at /Applications/Julia-0.4.0.app/Contents/Resources/julia/lib/julia/sys.dylib
while loading /Users/chrisfisher/.julia/v0.4/Gadfly/src/Gadfly.jl, in expression starting on line 9
ERROR: LoadError: LoadError: LoadError: LoadError: Failed to precompile Gadfly to /Users/chrisfisher/.julia/lib/v0.4/Gadfly.ji
in error at /Applications/Julia-0.4.0.app/Contents/Resources/julia/lib/julia/sys.dylib
in compilecache at loading.jl:383
in require at /Applications/Julia-0.4.0.app/Contents/Resources/julia/lib/julia/sys.dylib
in include at /Applications/Julia-0.4.0.app/Contents/Resources/julia/lib/julia/sys.dylib
in include_from_node1 at /Applications/Julia-0.4.0.app/Contents/Resources/julia/lib/julia/sys.dylib
in require at /Applications/Julia-0.4.0.app/Contents/Resources/julia/lib/julia/sys.dylib
in include at /Applications/Julia-0.4.0.app/Contents/Resources/julia/lib/julia/sys.dylib
in include_from_node1 at /Applications/Julia-0.4.0.app/Contents/Resources/julia/lib/julia/sys.dylib
in require at /Applications/Julia-0.4.0.app/Contents/Resources/julia/lib/julia/sys.dylib
in include at /Applications/Julia-0.4.0.app/Contents/Resources/julia/lib/julia/sys.dylib
in include_from_node1 at /Applications/Julia-0.4.0.app/Contents/Resources/julia/lib/julia/sys.dylib
[inlined code] from /Users/chrisfisher/.julia/v0.4/Stan/test/runtests.jl:22
in anonymous at no file:21
in include at /Applications/Julia-0.4.0.app/Contents/Resources/julia/lib/julia/sys.dylib
in include_from_node1 at /Applications/Julia-0.4.0.app/Contents/Resources/julia/lib/julia/sys.dylib
in process_options at /Applications/Julia-0.4.0.app/Contents/Resources/julia/lib/julia/sys.dylib
in _start at /Applications/Julia-0.4.0.app/Contents/Resources/julia/lib/julia/sys.dylib
while loading /Users/chrisfisher/.julia/v0.4/Mamba/src/Mamba.jl, in expression starting on line 6
while loading /Users/chrisfisher/.julia/v0.4/Stan/src/Stan.jl, in expression starting on line 5
while loading /Users/chrisfisher/.julia/v0.4/Stan/test/test_utilities.jl, in expression starting on line 1
while loading /Users/chrisfisher/.julia/v0.4/Stan/test/runtests.jl, in expression starting on line 20
LoadError: Stan had test errors
while loading In[9], in expression starting on line 1
in error at /Applications/Julia-0.4.0.app/Contents/Resources/julia/lib/julia/sys.dylib
in test at pkg/entry.jl:753
in anonymous at pkg/dir.jl:31
in cd at file.jl:22
in cd at pkg/dir.jl:31
in test at pkg.jl:71
================================[ ERROR: Stan ]=================================
failed process: Process(/Applications/Julia-0.4.0.app/Contents/Resources/julia/bin/julia --check-bounds=yes --code-coverage=none --color=no /Users/chrisfisher/.julia/v0.4/Stan/test/runtests.jl
, ProcessExited(1)) [1]
Needs a complete rewrite to make it easy to find the right example.
I'm a new user to Stan on julia, although I'm a long-time cmdstan user. I'm working on a problem where arrays of over a million elements need to be written to the .R file. I noticed that the stan() function never returned, seeming to get stuck just after opening the .R file but not writing to it. Upon investigation I found that update_R_file() puts each stan variable into a string before writing it, meaning that a large array generates a huge string. It's my guess that julia balks at this huge string. I modified update_R_file() to write arrays using writecsv() instead, and this solved the problem. My updated code is attached. I'm new to github, so pardon me if there is another way to upload proposed revisions. You'll note that the revised code is shorter as well.
Michael
function update_R_file{T<:Any}(file::String, dct::Dict{String, T}; replaceNaNs::Bool=true)
isfile(file) && rm(file)
strmout = open(file, "w")
str = ""
for entry in dct
str = """entry[1]"""" <- "
val = entry[2]
if length(val)==1 && length(size(val))==0
# Scalar
str = str"$(val)\n"
elseif length(val)==1 && length(size(val))==1
# Single element vector
str = str_"$(val[1])\n"
elseif length(val)>1 && length(size(val))==1
# Vector
str = str_"structure(c("
for i in 1:length(val)
str = str*"$(val[i])"
if i < length(val)
str = str*", "
end
end
=## with this code
write(strmout, str)
str = ""
writecsv(strmout, val');
str = str*"), .Dim=c($(length(val))))\n"
elseif length(val)>1 && length(size(val))>1
# Array
str = str*"structure(c("
for i in 1:length(val)
str = str*"$(val[i])"
if i < length(val)
str = str*", "
end
end
=## with this code
write(strmout, str)
str = ""
writecsv(strmout, val[:]');
dimstr = "c"*string(size(val))
str = str*"), .Dim=$(dimstr))\n"
end
write(strmout, str)
end
close(strmout)
end
In the README,
p = plot(sim, [:trace, :mean, :density, :autocor], legend=true);
draw(p, ncol=4, filename="summaryplot", fmt=:svg)
but the second line gives me an error:
julia> draw(p, ncol=4, filename="summaryplot", fmt=:svg)
ERROR: MethodError: `isless` has no method matching isless(::Float64, ::Void)
Closest candidates are:
isless(::Float64, !Matched::Float64)
isless(::AbstractFloat, !Matched::AbstractFloat)
isless(::Real, !Matched::AbstractFloat)
...
in max at operators.jl:57
in apply_coordinate at /home/tamas/.julia/v0.4/Gadfly/src/coord.jl:232
in render_prepared at /home/tamas/.julia/v0.4/Gadfly/src/Gadfly.jl:776
in render at /home/tamas/.julia/v0.4/Gadfly/src/Gadfly.jl:722
in draw at /home/tamas/.julia/v0.4/Mamba/src/output/plot.jl:45
Using 4.1, but get the same with latest nightly.
Workaround: dropping the [:trace, :mean, :density, :autocor]
argument.
When sampling a model with:
model = Stanmodel(Sample(save_warmup=1),...)
sim = Stan.stan(model,data,...)
Only the one sample is returned in the Mamba.Chains object sim
.
However, the model_samples.csv file has the expected number of samples (num_warmup+num_samples) in it.
I should also note that the Stan executable returns the message:
Warning: non-fatal error reading adapation data
.
From Eric Chameroy on Discourse:
Hi everyone,
When I load Stan.jl, I receive the following message:
WARNING: using Stan.CMDSTAN_HOME in module Main conflicts with an existing identifier.
I had created a .juliarc.jl file containing the statement:
CMDSTAN_HOME = “C:\Users\USERNAME\CmdStan” (I’m using Windows 10)
as stated in the Stan.jl documentation for setting up CmdStan.
What does this warning mean and is there a way to fix it? Considering it’s a warning and not an error, do I need to be concerned with it at all?
I have tried defining CMDSTAN_HOME
as a Julia variable in my ~/.juliarc.jl
file and as an environment variable before I start jupyter-notebook
. Neither of these seems to do the trick. I always end up in my IJulia session with errors like
sim1 = stan(ratstan);
LoadError: UndefVarError: CMDSTAN_HOME not defined
while loading In[15], in expression starting on line 1
in stan at /home/bates/.julia/v0.4/Stan/src/stancode.jl:13
I notice that the checks for CMDSTAN_HOME
and JULIA_SVG_BROWSER
are part of the src/Stan.jl
file but not wrapped in an __init__
function. It is probably better to assign these values as global const
in an __init__
function, which will be called when the package is loaded. Also a more elegant approach than
try
CMDSTAN_HOME = ENV["CMDSTAN_HOME"]
catch e
...
end
is
if haskey(ENV, "CMDSTAN_HOME")
global const CMDSTAN_HOME = ENV["CMDSTAN_HOME"]
else
error(...)
end
I could create a pull request with such changes if you wish. You could take a look at the RCall package sources to see some of this in action.
I installed cmdstan
using:
brew install cmdstan
I then added the path in .juliarc.jl
CMDSTAN_HOME = "/usr/local/Cellar/cmdstan/2.7.0/"
I believe this is the correct directory, it has bin/
, include/
, and share/
directories within it. I've confirmed that the variable CMDSTAN_HOME
is set appropriately when I launch julia.
When I either try to reproduce the example given in the README or run Pkg.test()
I get errors like:
ErrorException("failed process: Process(`make /Users/alex/.julia/v0.4/Stan/Examples/Bernoulli/tmp/bernoulli`, ProcessExited(2)) [2]")
Pkg.test()
also outputs this line:
CMDSTAN_HOME found, but CmdStan not installed properly.
Is there a way I can test that homebrew installed cmdstan correctly? Any other thoughts on what the problem could be? Thanks in advance. -- Alex
Sebastian reported this on the Stan mailing list as slow. I've seen stack overflows in R and index errors in Julia.
Hi Rob-
I think there is an issue initializing chains. When I print theta in your simple example, it prints the correct value. However, in my hierarchical example, print(omega); should yield 3, but produces a random value. I could not find an error with my code unfortunately.
Best regards,
Chris
Hi,
This is more of an annoyance than a bug. I am using stan.jl with stan 2.20.0 in Atom. After I run Stanmodel
, it prints that information of the summary everytime I use the repl for something.
Eg.:
bernoullistanmodel = "
data {
int<lower=0> N;
int<lower=0,upper=1> y[N];
}
parameters {
real<lower=0,upper=1> theta;
}
model {
theta ~ beta(1,1);
y ~ bernoulli(theta);
}
"
stanmodel = Stanmodel(name="bernoulli", model=bernoullistanmodel);
And it will print a bunch of information about the model. Now do something else, like 2+2
and it will print the info all over again.
This is a Mac OS related bug, it does not happens on Windows. I am using Mojave.
Thanks in advance
Hi,
thanks for Stan.jl, it is a great package!
As of version 2.7.0 Stan supports variational inference as well. At the moment, is there any way to use the ADVI variational method from Stan.jl?
Thanks,
Sebastian
@JuliaRegistrator register()
Not sure yet how to test and fix this.
Hi @RalphAS,
Thanks for the patch. Are you using Stan.jl v2.0.0 or master (after Pkg.checkout("Stan")
?
In the master version, the call to stanmodel() has an additional keyword argument mambaThinning
which is set to 1 by default. This means by default thinning occurs in CmdStan, but optionally you can not use CmdStan and postpone thinning until later. One place where that might be useful is the CmdStan problem/issue that thinning and attempting to save warmups is not working correctly. See also http://goedman.github.io/Stan.jl/latest/VERSIONS.html#Version-2.0.0-1 .
I apologies for not having updated METADATA yet, but this is because I am at the tail end of a rather complicated move ( #46 ). I will spend some time on cleaning this up over the next 2 weeks.
Hope it is ok I respond in an issue, not directly on the PR.
Best,
Rob
I can't find a way to set initial values for my sampler.
Stanmodel.jl Stanmodel type has init
which I believe should act as the container for initial values
type Stanmodel
name::String
nchains::Int
...
init::Init
output::Output
tmpdir::String
end
which is defined to be of type Init
type Init
init::Int64
end
Init(;init::Number=2) = Init(init)
So it only accepts one integer.
In pystan one can give a dictionary in following manner:
init = {"theta1": 0.5, "theta2": 0.9}
I tried giving a dictionary, self-made Init type with the attributes corresponding to the parameter values I'd like to initialize but could not work around it.
PackageEvaluator.jl is a script that runs nightly. It attempts to load all Julia packages and run their tests (if available) on both the stable version of Julia (0.3) and the nightly build of the unstable version (0.4). The results of this script are used to generate a package listing enhanced with testing results.
Tests pass.
Package doesn't load.
Tests pass.
means that PackageEvaluator found the tests for your package, executed them, and they all passed.
Package doesn't load.
means that PackageEvaluator did not find tests for your package. Additionally, trying to load your package with using
failed.
This issue was filed because your testing status became worse. No additional issues will be filed if your package remains in this state, and no issue will be filed if it improves. If you'd like to opt-out of these status-change messages, reply to this message saying you'd like to and @IainNZ will add an exception. If you'd like to discuss PackageEvaluator.jl please file an issue at the repository. For example, your package may be untestable on the test machine due to a dependency - an exception can be added.
Test log:
>>> 'Pkg.add("Stan")' log
INFO: Cloning cache of Stan from git://github.com/goedman/Stan.jl.git
INFO: Installing Stan v0.0.2
INFO: Package database updated
INFO: METADATA is out-of-date a you may not have the latest version of Stan
INFO: Use `Pkg.update()` to get the latest versions of your packages
>>> 'using Stan' log
ERROR: DataFrames not found
in require at loading.jl:47
in include at ./boot.jl:245
in include_from_node1 at ./loading.jl:128
in reload_path at loading.jl:152
in _require at loading.jl:67
in require at loading.jl:51
in include at ./boot.jl:245
in include_from_node1 at loading.jl:128
in process_options at ./client.jl:285
in _start at ./client.jl:354
in _start_3B_1699 at /home/idunning/julia03/usr/bin/../lib/julia/sys.so
while loading /home/idunning/pkgtest/.julia/v0.3/Stan/src/Stan.jl, in expression starting on line 4
while loading /home/idunning/pkgtest/.julia/v0.3/Stan/testusing.jl, in expression starting on line 1
>>> test log
no tests to run
>>> end of log
@JuliaRegistrator register()
I am trying to initialize the starting values for my model. As an example, I tried to run BernoulliInitTheta. However, I received the following error:
TypeError: typeassert: expected Number, got Array{Dict{String,Any},1}
in (::Core.#kw#Type)(::Array{Any,1}, ::Type{Stan.Init}) at :0
in include_string(::String, ::String) at loading.jl:441
in include_string(::String, ::String) at sys.dylib:?
in eval(::Module, ::Any) at boot.jl:234
in eval(::Module, ::Any) at sys.dylib:?
in (::Atom.##65#68)() at eval.jl:40
in withpath(::Atom.##65#68, ::Void) at utils.jl:30
in withpath(::Function, ::Void) at eval.jl:46
in macro expansion at eval.jl:109 [inlined]
in (::Atom.##64#67{Dict{String,Any}})() at task.jl:60
Any help would be greatly appreciated.
I was comparing the results of a Bayesian SLOPE regression model with one that uses Optim for minimizing the loss function in SLOPE but ran into problems since Newton is exported by both packages. Not sure what is the best way to avoid these things but reporting it anyway since this type of comparison may not be super-uncommon... ;)
WARNING: both Optim and Stan export "Newton"; uses of it in module Main must be qualified
if rc == 0
# the following line is not needed because it is already defined above. It also won't work, because stan
# returns a tuple and we need to bind the second value
# sim1 = stan(stanmodel, [bernoullidata], ProjDir, CmdStanDir=CMDSTAN_HOME)
println("Subset Sampler Output")
sim = sim1[1:1000, ["lp__", "theta", "accept_stat__"], :]
describe(sim)
end
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.