GithubHelp home page GithubHelp logo

fangq / mcx Goto Github PK

View Code? Open in Web Editor NEW
118.0 26.0 68.0 24.8 MB

Monte Carlo eXtreme (MCX) - GPU-accelerated photon transport simulator

Home Page: http://mcx.space

License: Other

Shell 0.39% Makefile 0.73% Cuda 2.33% MATLAB 2.95% Pascal 47.53% C 20.49% C++ 2.05% Perl 0.48% Batchfile 0.02% Python 0.56% HTML 1.32% CMake 0.07% Inno Setup 0.16% Roff 0.45% Assembly 0.56% Jupyter Notebook 19.91%
cuda c monte-carlo optical-imaging photon-transport 3d matlab pascal physics-simulation ray-tracing

mcx's Introduction

Monte Carlo eXtreme (MCX) - CUDA Edition

Mex and Binaries
Linux Python Module
MacOS Python Module
Windows Python Module

Table of Content:

What's New

MCX v2024.2 contains both major new features and critical bug fixes. It is a strongly recommended upgrade for all users.

Specifically, MCX v2024.2 received three important new features:

  • user-defined photon launch angle distribution (#13)
  • simulate multiple sources in a single session (#163)
  • per-voxel mua/mus/g/n format support (#203)

Similar to the user-defined phase-function feature included in MCX v2023, the first feature allows users to define the zenith angle distribution for launching a photon, relative to the source-direction vector, also via a discretized inverse CDF (cumulative distribution function). In MATLAB/Octave, they can be set as cfg.invcdf or cfg.angleinvcdf, respectively. We provided ready-to-use demo scripts in mcxlab/examples/demo_mcxlab_phasefun.m and demo_mcxlab_launchangle.m.

The second feature allow users to specify more then one sources of the same type when defining srcpos, srcdir, srcparam1 and srcparam2. Each source can have independent weight controled by the 4th element of srcpos.

Aside from new features, a severe bug was discovered that affects all pattern and pattern3d simulations that do not use photon-sharing, please see

#212

Because of this bug, MCX/MCX-CL in older releases has been simulating squared pattern/pattern3d data instead of the pattern itself. If your simulation uses these two source types and you do not use photon-sharing (i.e. simulating multiple patterns together), please upgrade your MCX/MCX-CL and rerun your simulations. This bug only affect volumetric fluence/flux outputs; it does not affect diffuse reflectance outputs or binary patterns.

It also includes a bug fix from #195 regarding precision loss when saving diffuse reflectance.

We want to thank Haohui Zhang for reporting a number of critical issues (#195 and #212), Liam Keegan for contributing a patch to extend the slit source (#214). ShijieYan and fanyuyen have also contributed to the bug fixes and improvements.

The detailed updates can be found in the below change log

  • 2024-03-13 [abdee14] [bug] fix multi-source replay bug, close #215
  • 2024-03-11 [9250a0d] [doc] make final doc update, bump pmcx to v0.3.2
  • 2024-03-10 [4e7f404] [ci] revert to windows-2019, add help info for #214, add note on nvidia-uvm
  • 2024-03-10 [2750d70] [ci] choco install is failing on Windows, see actions/runner-images#9477
  • 2024-03-10 [7b6e0e0] [optimize] cut hyperboloid gaussian register use from 15 to 3, #127,#214
  • 2024-03-10 [a2279d6] [optimize] reduce gaussian slit register use from 9 to 2, #214
  • 2024-03-08 [4708e10] [doc] Create pull_request_template.md
  • 2024-03-08 [14ca45d] [feat] use 'make register' to report register counts in the Makefile
  • 2024-03-07 [411a007] [feat] add gaussian broadening to slit source (contributed by Liam Keegan, #214)
  • 2024-03-04 [1e6c403] [doc] update instructions on running mcx on hybrid GPU Linux laptops
  • 2024-03-04 [0344d84] [bug] fix cuda core count for Ada and Blackwell
  • 2024-02-27 [558dbab] [pmcx] bump pmcx to 0.2.12 after fixing critical bug #212
  • 2024-02-27 [8e03878] *[bug] critical: fix double-multiplication of pattern launched weight, fix #212
  • 2024-02-23 [61ae0b8] [bug] fix detid value #209
  • 2024-02-23 [90e2419] [bug] fix pmcx dettpsf overwrites input bug
  • 2024-02-21 [744fba2] [bug] update srcparam2 after volume preprocessing close #206
  • 2024-02-18 [dbb23be] [ci] test removing lazhelphtml.pas to avoid error on linux/macos runners
  • 2024-02-18 [4c88de2] [ci] test fix for lazarus 3.0 build error on macos action
  • 2024-02-18 [54f732e] [mcxcloud] support backup servers, support x/y/z slice view and time-gates
  • 2024-02-16 [8a8ff90] [ci] install libomp before downgrade xcode
  • 2024-02-16 [99ca38f] [ci] test libomp error on macos runners
  • 2024-02-13 [cb9b6c2] [doc] update openjdata to neurojson
  • 2024-02-04 [007ecce] [feat] accept 3-element srcparam1/srcparam2 in mcxlab/pmcx
  • 2024-01-23 [20b0aa2] [minor] debug addlog callback error
  • 2024-01-23 [f63f73b] [bug] fix chrome webpage error messages
  • 2024-01-11 [73fe834] [mcxcloud] add the missing flag to show optional json fields
  • 2024-01-01 [0eb1a48] [doc] update changelog
  • 2024-01-01 [4232764] [mcxlab] add speed comparison between different media formats
  • 2024-01-01 [3bca606] [minor] fix typo
  • 2024-01-01 [51003bc] *[format] automatic format all matlab script with miss_hit
  • 2024-01-01 [9951dd8] [minor] update copyright year
  • 2024-01-01 [0f48b8f] [format] update source unit header info
  • 2024-01-01 [6707eaa] [bug] fix detp output bug after adding srcid, #163
  • 2024-01-01 [49b5cef] [ci] update lazarus on windows to avoid build crash
  • 2024-01-01 [9dc832d] *[feat] support --srcid to simulate all, one or separate sources, #163
  • 2023-12-31 [f5b4aaa] [minor] fix file name spelling
  • 2023-12-30 [7e1aec0] *[feat] simulate multiple sources, close #163
  • 2023-12-29 [59f18a7] *[feat] complete per-voxel mua/mus/g/n format support, close #203
  • 2023-12-29 [1e3b031] [feat] initial implementation of mua/mus/g/n all float format
  • 2023-12-29 [1100e36] [bug] fix mcx2json bug when exporting 4D vol, fix #200
  • 2023-12-17 [9831c7e] [debug] print jdata compression message inside matlab
  • 2023-11-27 [28e5101] [pmcx] bump pmcx version to 0.2.7
  • 2023-11-27 [fae5b72] [pmcx] typecast traj.id from float to uint32, fix #199
  • 2023-11-16 [4eec905] [mcxlab] add demo script comparing conv vs direct area src
  • 2023-11-10 [6771752] [ci] fix matlab mex error after mingw64 was removed, matlab-actions/setup-matlab#75
  • 2023-11-08 [f4aec48] [ci] test conda command to install octave on mac
  • 2023-11-08 [c976cbf] [ci] brew refuses to install octave, switch to conda
  • 2023-11-07 [325a522] [release] post v2023 release action
  • 2023-11-07 [a710ab3] allow converting integer cfg.vol to json
  • 2023-10-31 [d6c64e4] [test] fix rng test after make double
  • 2023-10-30 [08361eb] [pmcx] bump pmcx to v0.2.6 with dref fix #195
  • 2023-10-30 [9220578] *[bug] apply #41 like 2xfloat-buffer for dref accumulation, fix #195
  • 2023-10-24 [961d059] pattern json data rstrict to single,set show_opt_in option
  • 2023-10-24 [5f130fc] use 2d pattern by default
  • 2023-10-24 [db608f7] use jq to format json schema; add Source.Pattern in schema
  • 2023-10-14 [4c365f9] update zh-cn translation
  • 2023-10-12 [8d23726] fix windows ci error
  • 2023-10-12 [2904cc3] avoid error when compressing binaries
  • 2023-10-12 [2ebe3de] include language locale files in github CI build
  • 2023-10-12 [5782cff] enable DefaultTranslator for i18n support in mcxstudio
  • 2023-10-11 [833d117] add compiled mo locale file
  • 2023-10-11 [ad298c1] add initial translation for simplified Chinese
  • 2023-10-11 [aa15780] add strings from additional forms
  • 2023-10-10 [14285a0] *prepare for adding i18n support
  • 2023-10-10 [c5496ac] *force invcdf/angleinvcdf even count of float, reapply 53d7ac0, fix #193
  • 2023-10-09 [ca1bf2b] use focal-length to select interpolation or discrete mode, #129
  • 2023-10-08 [be8b8c3] *support user-defined launch angle profile, fix #129
  • 2023-10-03 [2ad9307] update winget package files to the v2023 release
  • 2023-10-03 [26ede84] rename winget package file
  • 2023-10-03 [2e71a51] treat cfg->bc as a null-ending string, #191 #192
  • 2023-10-03 [68db492] make cfg->bc a null terminated string
  • 2023-10-02 [35170f9] Merge pull request #192 from lkeegan/pmcx_bc_overflow_error
  • 2023-10-02 [3c77170] Fix buffer overflow error when bc has 12 characters

Introduction

Monte Carlo eXtreme (MCX) is a fast physically-accurate photon simulation software for 3D heterogeneous complex media. By taking advantage of the massively parallel threads and extremely low memory latency in a modern graphics processing unit (GPU), this program is able to perform Monte Carlo (MC) simulations at a blazing speed, typically hundreds to a thousand times faster than a single-threaded CPU-based MC implementation.

MCX is written in C and NVIDIA CUDA. It only be executed on NVIDIA GPUs. If you want to run hardware-accelerated MCX simulations on AMD/Intel GPUs or CPUs, please download MCX-CL (MCX for OpenCL), which is written in OpenCL. MCX and MCX-CL are highly compatible.

Due to the nature of the underlying MC algorithms, MCX and MCX-CL are ray-tracing/ray-casting software under-the-hood. Compared to commonly seen ray-tracing libraries used in computer graphics or gaming engines, MCX-CL and MCX have many unique characteristics. The most important difference is that MCX/MCX-CL are rigorously based on physical laws. They are numerical solvers to the underlying radiative transfer equation (RTE) and their solutions have been validated across many publications using optical instruments and experimental measurements. In comparison, most graphics-oriented ray-tracers have to make many approximations in order to achieve fast rendering, enable to provide quantitatively accurate light simulation results. Because of this, MCX/MCX-CL have been extensively used by biophotonics research communities to obtain reference solutions and guide the development of novel medical imaging systems or clinical applications. Additionally, MCX/MCX-CL are volumetric ray-tracers; they traverse photon-rays throughout complex 3-D domains and computes physically meaningful quantities such as spatially resolved fluence, flux, diffuse reflectance/transmittance, energy deposition, partial pathlengths, among many others. In contrast, most graphics ray-tracing engines only trace the RGB color of a ray and render it on a flat 2-D screen. In other words, MCX/MCX-CL gives physically accurate 3-D light distributions while graphics ray-tracers focus on 2-D rendering of a scene at the camera. Nonetheless, they share many similarities, such as ray-marching computation, GPU acceleration, scattering/absorption handling etc.

The algorithm of this software is detailed in the References [Fang2009,Yu2018,Yan2020]. A short summary of the main features includes:

  • 3D heterogeneous media represented by voxelated array
  • support complex sources including wide-field and pattern illuminations
  • boundary reflection support
  • time-resolved photon transport simulations
  • saving photon partial path lengths and trajectories
  • optimized random number generators
  • build-in flux/fluence normalization to output Green's functions
  • user adjustable voxel resolution
  • improved accuracy with atomic operations
  • cross-platform graphical user interface
  • native Matlab/Octave support for high usability
  • flexible JSON interface for future extensions
  • multi-GPU support
  • advanced features: photon-replay, photon-sharing, and more

This software can be used on Windows, Linux and Mac OS. MCX is written in C/CUDA and requires NVIDIA GPUs (support for AMD/Intel CPUs/GPUs via ROCm is still under development). A more portable OpenCL implementation of MCX, i.e. MCXCL, was announced on July, 2012 and supports almost all NVIDIA/AMD/Intel CPU and GPU models. If your hardware does not support CUDA, please download MCXCL from the below URL:

https://mcx.space/wiki/index.cgi?Learn#mcxcl

Requirement and Installation

Please read this section carefully. The majority of failures using MCX were found related to incorrect installation of NVIDIA GPU driver.

Please browse https://mcx.space/#documentation for step-by-step instructions.

For MCX-CUDA, the requirements for using this software include

  • a CUDA capable NVIDIA graphics card
  • pre-installed NVIDIA graphics driver

You must make sure that your NVIDIA graphics driver was installed properly. A list of CUDA capable cards can be found at [2]. The oldest GPU architecture that MCX source code can be compiled is Fermi (sm_20). Using the latest NVIDIA card is expected to produce the best speed. The officially released binaries (including mex files and pmcx modules) can run on NVIDIA GPUs as old as Kepler (GTX-730, sm_35). All MCX binaries can run directly on future generations of NVIDIA GPUs without needing to be recompiled, therefore forward-compatible.

In the below webpage, we summarized the speed differences between different generations of NVIDIA GPUs

https://mcx.space/gpubench/

For simulations with large volumes, sufficient graphics memory is also required to perform the simulation. The minimum amount of graphics memory required for a MC simulation is Nx*Ny*Nz bytes for the input tissue data plus Nx*Ny*Nz*Ng*4*2 bytes for the output flux/fluence data - where Nx,Ny,Nz are the dimensions of the tissue volume, Ng is the number of concurrent time gates, 4 is the size of a single-precision floating-point number, 2 is for the extra memory needed to ensure output accuracy (#41). MCX does not require double-precision support in your hardware.

MCX stores optical properties and detector positions in the constant memory. Usually, NVIDIA GPUs provides about 64 kB constant memory. As a result, we can only the total number of optical properties plus the number of detectors can not exceed 4000 (4000 * 4 * 4 = 64 k).

In addition, MCX stores detected photon data inside the shared memory, which also ranges between 42 kB to 100 kB per stream processor across different GPU generations. If your domain contains many medium types, it is possible that the allocation of the shared memory can exceed the limit. You will also receive an "out of memory" error.

To install MCX, you need to download the binary executable compiled for your computer architecture (32 or 64bit) and platform, extract the package and run the executable under the {mcx root}/bin directory.

For Windows users, you must make sure you have installed the appropriate NVIDIA driver for your GPU. You should also configure your OS to run CUDA simulations. This requires you to open the mcx/setup/win64 folder using your file explorer, right-click on the apply_timeout_registry_fix.bat file and select “Run as administrator”. After confirmation, you should see a windows command window with message

  Patching your registry
  Done
  Press any key to continue ...

You MUST REBOOT your Windows computer to make this setting effective. The above patch modifies your driver settings so that you can run MCX simulations for longer than a few seconds. Otherwise, when running MCX for over a few seconds, you will get a CUDA error: “unspecified error”.

Please see the below link for details

https://mcx.space/wiki/index.cgi?Doc/FAQ#I_am_getting_a_kernel_launch_timed_out_error_what_is_that

If you use Linux, you may enable Intel integrated GPU (iGPU) for display while leaving your NVIDIA GPU dedicated for computing using nvidia-prime, see

https://forums.developer.nvidia.com/t/solved-run-cuda-on-dedicated-nvidia-gpu-while-connecting-monitors-to-intel-hd-graphics-is-this-possible/47690/6

or choose one of the 4 other approaches in this blog post

https://nvidia.custhelp.com/app/answers/detail/a_id/3029/~/using-cuda-and-x

We noticed that running Ubuntu Linux 22.04 with a 6.5 kernel on a laptop with a hybrid GPU with an Intel iGPU and an NVIDIA GPU, you must configure the laptop to use the NVIDIA GPU as the primary GPU by choosing "NVIDIA (Performance Mode)" in the PRIME Profiles section of NVIDIA X Server Settings. You can also run

sudo prime-select nvidia

to achieve the same goal. Otherwise, the simulation may hang your system after running for a few seconds. A hybrid GPU laptop combing an NVIDIA GPU with an AMD iGPU does not seem to have this issue if using Linux.

In addition, NVIDIA drirver (520 or newer) has a known glitch running on Linux kernel 6.x (such as those in Ubuntu 22.04). See

https://forums.developer.nvidia.com/t/dev-nvidia-uvm-io-error-on-ubuntu-22-04-520-to-535-driver-versions/262153

When the laptop is in the "performance" mode and wakes up from suspension, MCX or any CUDA program fails to run with an error

MCX ERROR(-999):unknown error in unit mcx_core.cu:2523

This is because the kernel module nvida-uvm fails to be reloaded after suspension. If you had an open MATLAB session, you must close MATLAB first, and run the below commands (if MATLAB is open, you will see rmmod: ERROR: Module nvidia_uvm is in use)

sudo rmmod /dev/nvidia-uvm
sudo modprobe nvidia-uvm

after the above command, MCX should be able to run again.

New generations of Mac computers no longer support NVIDIA or AMD GPUs. you will have to use the OpenCL version of MCX, MCX-CL by downloading it from

https://mcx.space/wiki/?Learn#mcxcl

Running Simulations

To run a simulation, the minimum input is a configuration (text) file, and, if the input file does not contain built-in domain shape descriptions, an external volume file (a binary file with a specified voxel format via -K/--mediabyte). Typing mcx without any parameters prints the help information and a list of supported parameters, as shown below:

###############################################################################
#                      Monte Carlo eXtreme (MCX) -- CUDA                      #
#          Copyright (c) 2009-2024 Qianqian Fang <q.fang at neu.edu>          #
#                https://mcx.space/  &  https://neurojson.io/                 #
#                                                                             #
# Computational Optics & Translational Imaging (COTI) Lab- http://fanglab.org #
#   Department of Bioengineering, Northeastern University, Boston, MA, USA    #
###############################################################################
#    The MCX Project is funded by the NIH/NIGMS under grant R01-GM114365      #
###############################################################################
#  Open-source codes and reusable scientific data are essential for research, #
# MCX proudly developed human-readable JSON-based data formats for easy reuse.#
#                                                                             #
#Please visit our free scientific data sharing portal at https://neurojson.io/#
# and consider sharing your public datasets in standardized JSON/JData format #
###############################################################################
$Rev::d593a0$v2024.2 $Date::2024-03-04 00:04:10 -05$ by $Author::Qianqian Fang$
###############################################################################

usage: mcx <param1> <param2> ...
where possible parameters include (the first value in [*|*] is the default)

== Required option ==
 -f config     (--input)       read an input file in .json or .inp format
                               if the string starts with '{', it is parsed as
                               an inline JSON input file
      or
 --bench ['cube60','skinvessel',..] run a buint-in benchmark specified by name
                               run --bench without parameter to get a list

== MC options ==
 -n [0|int]    (--photon)      total photon number (exponential form accepted)
                               max accepted value:9.2234e+18 on 64bit systems
 -r [1|+/-int] (--repeat)      if positive, repeat by r times,total= #photon*r
                               if negative, divide #photon into r subsets
 -b [1|0]      (--reflect)     1 to reflect photons at ext. boundary;0 to exit
 -B '______'   (--bc)          per-face boundary condition (BC), 6 letters for
    /case insensitive/         bounding box faces at -x,-y,-z,+x,+y,+z axes;
                               overwrite -b if given. 
                               each letter can be one of the following:
                               '_': undefined, fallback to -b
                               'r': like -b 1, Fresnel reflection BC
                               'a': like -b 0, total absorption BC
                               'm': mirror or total reflection BC
                               'c': cyclic BC, enter from opposite face

                               if input contains additional 6 letters,
                               the 7th-12th letters can be:
                               '0': do not use this face to detect photon, or
                               '1': use this face for photon detection (-d 1)
                               the order of the faces for letters 7-12 is 
                               the same as the first 6 letters
                               eg: --bc ______010 saves photons exiting at y=0
 -u [1.|float] (--unitinmm)    defines the length unit for the grid edge
 -U [1|0]      (--normalize)   1 to normalize flux to unitary; 0 save raw
 -E [0|int|mch](--seed)        set random-number-generator seed, -1 to generate
                               if an mch file is followed, MCX "replays" 
                               the detected photon; the replay mode can be used
                               to calculate the mua/mus Jacobian matrices
 -z [0|1]      (--srcfrom0)    1 volume origin is [0 0 0]; 0: origin at [1 1 1]
 -k [1|0]      (--voidtime)    when src is outside, 1 enables timer inside void
 -Y [0|int]    (--replaydet)   replay only the detected photons from a given 
                               detector (det ID starts from 1), used with -E 
                               if 0, replay all detectors and sum all Jacobians
                               if -1, replay all detectors and save separately
 -V [0|1]      (--specular)    1 source located in the background,0 inside mesh
 -e [0.|float] (--minenergy)   minimum energy level to trigger Russian roulette
 -g [1|int]    (--gategroup)   number of maximum time gates per run

== GPU options ==
 -L            (--listgpu)     print GPU information only
 -t [16384|int](--thread)      total thread number
 -T [64|int]   (--blocksize)   thread number per block
 -A [1|int]    (--autopilot)   1 let mcx decide thread/block size, 0 use -T/-t
 -G [0|int]    (--gpu)         specify which GPU to use, list GPU by -L; 0 auto
      or
 -G '1101'     (--gpu)         using multiple devices (1 enable, 0 disable)
 -W '50,30,20' (--workload)    workload for active devices; normalized by sum
 -I            (--printgpu)    print GPU information and run program
 --atomic [1|0]                1: use atomic operations to avoid thread racing
                               0: do not use atomic operation (not recommended)

== Input options ==
 -P '{...}'    (--shapes)      a JSON string for additional shapes in the grid.
                               only the root object named 'Shapes' is parsed 
                               and added to the existing domain defined via -f 
                               or --bench
 -j '{...}'    (--json)        a JSON string for modifying all input settings.
                               this input can be used to modify all existing 
                               settings defined by -f or --bench
 -K [1|int|str](--mediabyte)   volume data format, use either a number or a str
       voxel binary data layouts are shown in {...}, where [] for byte,[i:]
       for 4-byte integer, [s:] for 2-byte short, [h:] for 2-byte half float,
       [f:] for 4-byte float; on Little-Endian systems, least-sig. bit on left
                               1 or byte: 0-128 tissue labels
                               2 or short: 0-65535 (max to 4000) tissue labels
                               4 or integer: integer tissue labels 
                              96 or asgn_float: mua/mus/g/n 4xfloat format
                                {[f:mua][f:mus][f:g][f:n]}
                              97 or svmc: split-voxel MC 8-byte format
                                {[n.z][n.y][n.x][p.z][p.y][p.x][upper][lower]}
                              98 or mixlabel: label1+label2+label1_percentage
                                {[label1][label2][s:0-32767 label1 percentage]}
                              99 or labelplus: 32bit composite voxel format
                                {[h:mua/mus/g/n][s:(B15-16:0/1/2/3)(label)]}
                             100 or muamus_float: 2x 32bit floats for mua/mus
                                {[f:mua][f:mus]}; g/n from medium type 1
                             101 or mua_float: 1 float per voxel for mua
                                {[f:mua]}; mus/g/n from medium type 1
                             102 or muamus_half: 2x 16bit float for mua/mus
                                {[h:mua][h:mus]}; g/n from medium type 1
                             103 or asgn_byte: 4x byte gray-levels for mua/s/g/n
                                {[mua][mus][g][n]}; 0-255 mixing prop types 1&2
                             104 or muamus_short: 2x short gray-levels for mua/s
                                {[s:mua][s:mus]}; 0-65535 mixing prop types 1&2
       when formats 99 or 102 is used, the mua/mus values in the input volume
       binary data must be pre-scaled by voxel size (unitinmm) if it is not 1.
       pre-scaling is not needed when using these 2 formats in mcxlab/pmcx
 -a [0|1]      (--array)       1 for C array (row-major); 0 for Matlab array

== Output options ==
 -s sessionid  (--session)     a string to label all output file names
 -O [X|XFEJPMRL](--outputtype) X - output flux, F - fluence, E - energy deposit
    /case insensitive/         J - Jacobian (replay mode),   P - scattering, 
                               event counts at each voxel (replay mode only)
                               M - momentum transfer; R - RF/FD Jacobian
                               L - total pathlength
 -d [1|0-3]    (--savedet)     1 to save photon info at detectors; 0 not save
                               2 reserved, 3 terminate simulation when detected
                               photon buffer is filled
 -w [DP|DSPMXVW](--savedetflag)a string controlling detected photon data fields
    /case insensitive/         1 D  output detector ID (1)
                               2 S  output partial scat. even counts (#media)
                               4 P  output partial path-lengths (#media)
                               8 M  output momentum transfer (#media)
                              16 X  output exit position (3)
                              32 V  output exit direction (3)
                              64 W  output initial weight (1)
      combine multiple items by using a string, or add selected numbers together
      by default, mcx only saves detector ID and partial-path data
 -x [0|1]      (--saveexit)    1 to save photon exit positions and directions
                               setting -x to 1 also implies setting '-d' to 1.
                               same as adding 'XV' to -w.
 -X [0|1]      (--saveref)     1 to save diffuse reflectance at the air-voxels
                               right outside of the domain; if non-zero voxels
                               appear at the boundary, pad 0s before using -X
 -m [0|1]      (--momentum)    1 to save photon momentum transfer,0 not to save.
                               same as adding 'M' to the -w flag
 -q [0|1]      (--saveseed)    1 to save photon RNG seed for replay; 0 not save
 -M [0|1]      (--dumpmask)    1 to dump detector volume masks; 0 do not save
 -H [1000000] (--maxdetphoton) max number of detected photons
 -S [1|0]      (--save2pt)     1 to save the flux field; 0 do not save
 -F [jnii|...](--outputformat) fluence data output format:
                               mc2 - MCX mc2 format (binary 32bit float)
                               jnii - JNIfTI format (https://neurojson.org)
                               bnii - Binary JNIfTI (https://neurojson.org)
                               nii - NIfTI format
                               hdr - Analyze 7.5 hdr/img format
                               tx3 - GL texture data for rendering (GL_RGBA32F)
    the bnii/jnii formats support compression (-Z) and generate small files
    load jnii (JSON) and bnii (UBJSON) files using below lightweight libs:
      MATLAB/Octave: JNIfTI toolbox   https://neurojson.org/download/jnifti
      MATLAB/Octave: JSONLab toolbox  https://neurojson.org/download/jsonlab
      Python:        PyJData:         https://neurojson.org/download/pyjdata
      JavaScript:    JSData:          https://neurojson.org/download/jsdata
 -Z [zlib|...] (--zip)         set compression method if -F jnii or --dumpjson
                               is used (when saving data to JSON/JNIfTI format)
                               0 zlib: zip format (moderate compression,fast) 
                               1 gzip: gzip format (compatible with *.gz)
                               2 base64: base64 encoding with no compression
                               3 lzip: lzip format (high compression,very slow)
                               4 lzma: lzma format (high compression,very slow)
                               5 lz4: LZ4 format (low compression,extrem. fast)
                               6 lz4hc: LZ4HC format (moderate compression,fast)
 --dumpjson [-,0,1,'file.json']  export all settings, including volume data using
                               JSON/JData (https://neurojson.org) format for
                               easy sharing; can be reused using -f
                               if followed by nothing or '-', mcx will print
                               the JSON to the console; write to a file if file
                               name is specified; by default, prints settings
                               after pre-processing; '--dumpjson 2' prints 
                               raw inputs before pre-processing

== User IO options ==
 -h            (--help)        print this message
 -v            (--version)     print MCX revision number
 -l            (--log)         print messages to a log file instead
 -i            (--interactive) interactive mode

== Debug options ==
 -D [0|int]    (--debug)       print debug information (you can use an integer
  or                           or a string by combining the following flags)
 -D [''|RMPT]                  1 R  debug RNG
    /case insensitive/         2 M  store photon trajectory info
                               4 P  print progress bar
                               8 T  save trajectory data only, disable flux/detp
      combine multiple items by using a string, or add selected numbers together

== Additional options ==
 --root         [''|string]    full path to the folder storing the input files
 --gscatter     [1e9|int]      after a photon completes the specified number of
                               scattering events, mcx then ignores anisotropy g
                               and only performs isotropic scattering for speed
 --srcid  [0|-1,0,1,2,..]      -1 simulate multi-source separately;0 all sources
                               together; a positive integer runs a single source
 --internalsrc  [0|1]          set to 1 to skip entry search to speedup launch
 --maxvoidstep  [1000|int]     maximum distance (in voxel unit) of a photon that
                               can travel before entering the domain, if 
                               launched outside (i.e. a widefield source)
 --maxjumpdebug [10000000|int] when trajectory is requested (i.e. -D M),
                               use this parameter to set the maximum positions
                               stored (default: 1e7)

== Example ==
example: (list built-in benchmarks)
       mcx --bench
or (list supported GPUs on the system)
       mcx -L
or (use multiple devices - 1st,2nd and 4th GPUs - together with equal load)
       mcx --bench cube60b -n 1e7 -G 1101 -W 10,10,10
or (use inline domain definition)
       mcx -f input.json -P '{"Shapes":[{"ZLayers":[[1,10,1],[11,30,2],[31,60,3]]}]}'
or (use inline json setting modifier)
       mcx -f input.json -j '{"Optode":{"Source":{"Type":"isotropic"}}}'
or (dump simulation in a single json file)
       mcx --bench cube60planar --dumpjson

To further illustrate the command line options, below one can find a sample command

mcx -A 0 -t 16384 -T 64 -n 1e7 -G 1 -f input.json -r 2 -s test -g 10 -d 1 -w dpx -b 1

the command above asks mcx to manually (-A 0) set GPU threads, and launch 16384 GPU threads (-t) with every 64 threads a block (-T); a total of 1e7 photons (-n) are simulated by the first GPU (-G 1) and repeat twice (-r) - i.e. total 2e7 photons; the media/source configuration will be read from a JSON file named input.json (-f) and the output will be labeled with the session id “test” (-s); the simulation will run 10 concurrent time gates (-g) if the GPU memory can not simulate all desired time gates at once. Photons passing through the defined detector positions are saved for later rescaling (-d); refractive index mismatch is considered at media boundaries (-b).

Historically, MCX supports an extended version of the input file format (.inp) used by tMCimg. However, we are phasing out the .inp support and strongly encourage users to adopt JSON formatted (.json) input files. Many of the advanced MCX options are only supported in the JSON input format.

A legacy .inp MCX input file looks like this:

1000000              # total photon, use -n to overwrite in the command line
29012392             # RNG seed, negative to generate, use -E to overwrite
30.0 30.0 0.0 1      # source position (in grid unit), the last num (optional) sets --srcfrom0 (-z)
0 0 1 0              # initial directional vector, 4th number is the focal-length, 0 for collimated beam, nan for isotropic
0.e+00 1.e-09 1.e-10 # time-gates(s): start, end, step
semi60x60x60.bin     # volume ('unsigned char' binary format, or specified by -K/--mediabyte)
1 60 1 60            # x voxel size in mm (isotropic only), dim, start/end indices
1 60 1 60            # y voxel size, must be same as x, dim, start/end indices 
1 60 1 60            # y voxel size, must be same as x, dim, start/end indices
1                    # num of media
1.010101 0.01 0.005 1.37  # scat. mus (1/mm), g, mua (1/mm), n
4       1.0          # detector number and default radius (in grid unit)
30.0  20.0  0.0  2.0 # detector 1 position (real numbers in grid unit) and individual radius (optional)
30.0  40.0  0.0      # ..., if individual radius is ignored, MCX will use the default radius
20.0  30.0  0.0      #
40.0  30.0  0.0      # 
pencil               # source type (optional)
0 0 0 0              # parameters (4 floats) for the selected source
0 0 0 0              # additional source parameters

Note that the scattering coefficient mus=musp/(1-g).

The volume file (semi60x60x60.bin in the above example), can be read in two ways by MCX: row-major[3] or column-major depending on the value of the user parameter -a. If the volume file was saved using matlab or fortran, the byte order is column-major, and you should use -a 0 or leave it out of the command line. If it was saved using the fwrite() in C, the order is row-major, and you can either use -a 1.

You may replace the binary volume file by a JSON-formatted shape file. Please refer to Section V for details.

The time gate parameter is specified by three numbers: start time, end time and time step size (in seconds). In the above example, the configuration specifies a total time window of [0 1] ns, with a 0.1 ns resolution. That means the total number of time gates is 10.

MCX provides an advanced option, -g, to run simulations when the GPU memory is limited. It specifies how many time gates to simulate concurrently. Users may want to limit that number to less than the total number specified in the input file - and by default it runs one gate at a time in a single simulation. But if there's enough memory based on the memory requirement in Section II, you can simulate all 10 time gates (from the above example) concurrently by using -g 10 in which case you have to make sure the video card has at least 60*60*60*10*5=10MB of free memory. If you do not include the -g, MCX will assume you want to simulate just 1 time gate at a time.. If you specify a time-gate number greater than the total number in the input file, (e.g, -g 20) MCX will stop when the 10 time-gates are completed. If you use the autopilot mode (-A), then the time-gates are automatically estimated for you.

Using JSON-formatted input files

Starting from version 0.7.9, MCX accepts a JSON-formatted input file in addition to the conventional tMCimg-like input format. JSON (JavaScript Object Notation) is a portable, human-readable and “fat-free” text format to represent complex and hierarchical data. Using the JSON format makes a input file self-explanatory, extensible and easy-to-interface with other applications (like MATLAB).

A sample JSON input file can be found under the examples/quicktest folder. The same file, qtest.json, is also shown below:

{
    "Help": {
      "[en]": {
        "Domain::VolumeFile": "file full path to the volume description file, can be a binary or JSON file",
        "Domain::Dim": "dimension of the data array stored in the volume file",
        "Domain::OriginType": "similar to --srcfrom0, 1 if the origin is [0 0 0], 0 if it is [1.0,1.0,1.0]",
	"Domain::LengthUnit": "define the voxel length in mm, similar to --unitinmm",
        "Domain::Media": "the first medium is always assigned to voxels with a value of 0 or outside of
                         the volume, the second row is for medium type 1, and so on. mua and mus must 
                         be in 1/mm unit",
        "Session::Photons": "if -n is not specified in the command line, this defines the total photon number",
        "Session::ID": "if -s is not specified in the command line, this defines the output file name stub",
        "Forward::T0": "the start time of the simulation, in seconds",
        "Forward::T1": "the end time of the simulation, in seconds",
        "Forward::Dt": "the width of each time window, in seconds",
        "Optode::Source::Pos": "the grid position of the source, can be non-integers, in grid unit",
        "Optode::Detector::Pos": "the grid position of a detector, can be non-integers, in grid unit",
        "Optode::Source::Dir": "the unitary directional vector of the photon at launch",
        "Optode::Source::Type": "source types, must be one of the following: 
                   pencil,isotropic,cone,gaussian,planar,pattern,fourier,arcsine,disk,fourierx,fourierx2d,
		   zgaussian,line,slit,pencilarray,pattern3d",
        "Optode::Source::Param1": "source parameters, 4 floating-point numbers",
        "Optode::Source::Param2": "additional source parameters, 4 floating-point numbers"
      }
    },
    "Domain": {
	"VolumeFile": "semi60x60x60.bin",
        "Dim":    [60,60,60],
        "OriginType": 1,
	"LengthUnit": 1,
        "Media": [
             {"mua": 0.00, "mus": 0.0, "g": 1.00, "n": 1.0},
             {"mua": 0.005,"mus": 1.0, "g": 0.01, "n": 1.0}
        ]
    },
    "Session": {
	"Photons":  1000000,
	"RNGSeed":  29012392,
	"ID":       "qtest"
    },
    "Forward": {
	"T0": 0.0e+00,
	"T1": 5.0e-09,
	"Dt": 5.0e-09
    },
    "Optode": {
	"Source": {
	    "Pos": [29.0, 29.0, 0.0],
	    "Dir": [0.0, 0.0, 1.0],
	    "Type": "pencil",
	    "Param1": [0.0, 0.0, 0.0, 0.0],
	    "Param2": [0.0, 0.0, 0.0, 0.0]
	},
	"Detector": [
	    {
		"Pos": [29.0,  19.0,  0.0],
		"R": 1.0
	    },
            {
                "Pos": [29.0,  39.0,  0.0],
                "R": 1.0
            },
            {
                "Pos": [19.0,  29.0,  0.0],
                "R": 1.0
            },
            {
                "Pos": [39.0,  29.0,  0.0],
                "R": 1.0
            }
	]
    }
}

A JSON input file requiers several root objects, namely Domain, Session, Forward and Optode. Other root sections, like Help, will be ignored. Each object is a data structure providing information indicated by its name. Each object can contain various sub-fields. The orders of the fields in the same level are flexible. For each field, you can always find the equivalent fields in the *.inp input files. For example, The VolumeFile field under the Domain object is the same as Line#6 in qtest.inp; the RNGSeed under Session is the same as Line#2; the Optode.Source.Pos is the same as the triplet in Line#3; the Forward.T0 is the same as the first number in Line#5, etc.

An MCX JSON input file must be a valid JSON text file. You can validate your input file by running a JSON validator, for example http://jsonlint.com/ You should always use "" to quote a “name” and separate parallel items by “,”.

MCX accepts an alternative form of JSON input, but using it is not recommended. In the alternative format, you can use “rootobj_name.field_name: value to represent any parameter directly in the root level. For example

{
    "Domain.VolumeFile": "semi60x60x60.json",
    "Session.Photons": 10000000,
    ...
}

You can even mix the alternative format with the standard format. If any input parameter has values in both formats in a single input file, the standard-formatted value has higher priority.

To invoke the JSON-formatted input file in your simulations, you can use the -f command line option with MCX, just like using an .inp file. For example:

  mcx -A 1 -n 20 -f onecube.json -s onecubejson

The input file must have a .json suffix in order for MCX to recognize. If the input information is set in both command line, and input file, the command line value has higher priority (this is the same for .inp input files). For example, when using -n 20, the value set in Session/Photons is overwritten to 20; when using -s onecubejson, the Session/ID value is modified. If your JSON input file is invalid, MCX will quit and point out where the format is incorrect.

Using JSON-formatted shape description files

Starting from v0.7.9, MCX can also use a shape description file in the place of the volume file. Using a shape-description file can save you from making a binary .bin volume. A shape file uses more descriptive syntax and can be easily understood and shared with others.

Samples on how to use the shape files are included under the example/shapetest folder.

The sample shape file, shapes.json, is shown below:

{
  "MCX_Shape_Command_Help":{
     "Shapes::Common Rules": "Shapes is an array object. The Tag field sets the voxel value for each
         region; if Tag is missing, use 0. Tag must be smaller than the maximum media number in the
         input file.Most parameters are in floating-point (FP). If a parameter is a coordinate, it
         assumes the origin is defined at the lowest corner of the first voxel, unless user overwrite
         with an Origin object. The default origin of all shapes is initialized by user's --srcfrom0
         setting: if srcfrom0=1, the lowest corner of the 1st voxel is [0,0,0]; otherwise, it is [1,1,1]",
     "Shapes::Name": "Just for documentation purposes, not parsed in MCX",
     "Shapes::Origin": "A floating-point (FP) triplet, set coordinate origin for the subsequent objects",
     "Shapes::Grid": "Recreate the background grid with the given dimension (Size) and fill-value (Tag)",
     "Shapes::Sphere": "A 3D sphere, centered at C0 with radius R, both have FP values",
     "Shapes::Box": "A 3D box, with lower corner O and edge length Size, both have FP values",
     "Shapes::SubGrid": "A sub-section of the grid, integer O- and Size-triplet, inclusive of both ends",
     "Shapes::XLayers/YLayers/ZLayers": "Layered structures, defined by an array of integer triples:
          [start,end,tag]. Ends are inclusive in MATLAB array indices. XLayers are perpendicular to x-axis, and so on",
     "Shapes::XSlabs/YSlabs/ZSlabs": "Slab structures, consisted of a list of FP pairs [start,end]
          both ends are inclusive in MATLAB array indices, all XSlabs are perpendicular to x-axis, and so on",
     "Shapes::Cylinder": "A finite cylinder, defined by the two ends, C0 and C1, along the axis and a radius R",
     "Shapes::UpperSpace": "A semi-space defined by inequality A*x+B*y+C*z>D, Coef is required, but not Equ"
  },
  "Shapes": [
     {"Name":     "Test"},
     {"Origin":   [0,0,0]},
     {"Grid":     {"Tag":1, "Size":[40,60,50]}},
     {"Sphere":   {"Tag":2, "O":[30,30,30],"R":20}},
     {"Box":      {"Tag":0, "O":[10,10,10],"Size":[10,10,10]}},
     {"Subgrid":  {"Tag":1, "O":[13,13,13],"Size":[5,5,5]}},
     {"UpperSpace":{"Tag":3,"Coef":[1,-1,0,0],"Equ":"A*x+B*y+C*z>D"}},
     {"XSlabs":   {"Tag":4, "Bound":[[5,15],[35,40]]}},
     {"Cylinder": {"Tag":2, "C0": [0.0,0.0,0.0], "C1": [15.0,8.0,10.0], "R": 4.0}},
     {"ZLayers":  [[1,10,1],[11,30,2],[31,50,3]]}
  ]
 }

A shape file must contain a Shapes object in the root level. Other root-level fields are ignored. The Shapes object is a JSON array, with each element representing a 3D object or setting. The object-class commands include Grid, Sphere, Box etc. Each of these object include a number of sub-fields to specify the parameters of the object. For example, the Sphere object has 3 subfields, O, R and Tag. Field O has a value of 1x3 array, representing the center of the sphere; R is a scalar for the radius; Tag is the voxel values. The most useful command is [XYZ]Layers. It contains a series of integer triplets, specifying the starting index, ending index and voxel value of a layered structure. If multiple objects are included, the subsequent objects always overwrite the overlapping regions covered by the previous objects.

There are a few ways for you to use shape description records in your MCX simulations. You can save it to a JSON shape file, and put the file name in Line#6 of your .inp file, or set as the value for Domain.VolumeFile field in a .json input file. In these cases, a shape file must have a suffix of .json.

You can also merge the Shapes section with a .json input file by simply appending the Shapes section to the root-level object. You can find an example, jsonshape_allinone.json, under examples/shapetest. In this case, you no longer need to define the VolumeFile field in the input.

Another way to use Shapes is to specify it using the -P (or --shapes) command line flag. For example:

     mcx -f input.json -P '{"Shapes":[{"ZLayers":[[1,10,1],[11,30,2],[31,60,3]]}]}'

This will first initialize a volume based on the settings in the input .json file, and then rasterize new objects to the domain and overwrite regions that are overlapping.

For both JSON-formatted input and shape files, you can use the JSONlab toolbox [4] to load and process in MATLAB.

Output data formats

MCX may produces several output files depending user's simulation settings. Overall, MCX produces two types of outputs, 1) data accummulated within the 3D volume of the domain (volumetric output), and 2) data stored for each detected photon (detected photon data).

Volumetric output

By default, MCX stores a 4D array denoting the fluence-rate at each voxel in the volume, with a dimension of NxNyNz*Ng, where Nx/Ny/Nz are the voxel dimension of the domain, and Ng is the total number of time gates. The output data are stored in the format of single-precision floating point numbers. One may choose to output different physical quantities by setting the -O option. When the flag -X/--saveref is used, the output volume may contain the total diffuse reflectance only along the background-voxels adjacent to non-zero voxels. A negative sign is added for the diffuse reflectance raw output to distinguish it from the fuence data in the interior voxels.

When photon-sharing (simultaneous simulations of multiple patterns) or photon-replay (the Jacobian of all source/detector pairs) is used, the output array may be extended to a 5D array, with the left-most/fastest index being the number of patterns Ns (in the case of photon-sharing) or src/det pairs (in replay), denoted as Ns.

Several data formats can be used to store the 3D/4D/5D volumetric output.

mc2 files

Starting in MCX v2023, .mc2 files are no longer the default output format for MCX binary. Instead, JSON based JNIfTI (.jnii) files are used.

The .mc2 format is simply a binary dump of the entire volumetric data output, consisted of the voxel values (single-precision floating-point) of all voxels and time gates. The file contains a continuous buffer of a single-precision (4-byte) 5D array of dimension Ns*Nx*Ny*Nz*Ng, with the fastest index being the left-most dimension (i.e. column-major, similar to MATLAB/FORTRAN).

To load the mc2 file, one should call loadmc2.m and must provide explicitly the dimensions of the data. This is because mc2 file does not contain the data dimension information.

Saving to .mc2 volumetric file is depreciated as we are transitioning towards JNIfTI/JData formatted outputs (.jnii).

nii files

The NIfTI-1 (.nii) format is widely used in neuroimaging and MRI community to store and exchange ND numerical arrays. It contains a 352 byte header, followed by the raw binary stream of the output data. In the header, the data dimension information as well as other metadata is stored.

A .nii output file can be generated by using -F nii in the command line.

The .nii file is widely supported among data processing platforms, including MATLAB and Python. For example

jnii files

Starting in MCX v2023, JSON based JNIfTI (.jnii) files are used as the default volumetric data output format.

The JNIfTI format represents the next-generation scientific data storage and exchange standard and is part of the US NIH-funded NeuroJSON initiative (https://neurojson.org) led by the MCX author Dr. Qianqian Fang. The NeuroJSON project aims at developing easy-to-parse, human-readable and easy-to-reuse data storage formats based on the ubiquitously supported JSON/binary JSON formats and portable JData data annotation keywords. In short, .jnii file is simply a JSON file with capability of storing binary strongly-typed data with internal compression and built in metadata.

The format standard (Draft 1) of the JNIfTI file can be found at

https://github.com/NeuroJSON/jnifti

A .jnii output file can be generated by using -F jnii in the command line.

The .jnii file can be potentially read in nearly all programming languages because it is 100% comaptible to the JSON format. However, to properly decode the ND array with built-in compression, one should call JData compatible libraries, which can be found at https://neurojson.org/#software

Specifically, to parse/save .jnii files in MATLAB, you should use

To parse/save .jnii files in Python, you should use

In Python, the volumetric data is loaded as a dict object where data['NIFTIData'] is a NumPy ndarray object storing the volumetric data.

bnii files

The binary JNIfTI file is also part of the JNIfTI specification and the NeuroJSON project. In comparison to text-based JSON format, .bnii files can be much smaller and faster to parse. The .bnii format is also defined in the BJData specification

https://github.com/NeuroJSON/bjdata

and is the binary interface to .jnii. A .bnii output file can be generated by using -F bnii in the command line.

The .bnii file can be potentially read in nearly all programming languages because it was based on UBJSON (Universal Binary JSON). However, to properly decode the ND array with built-in compression, one should call JData compatible libraries, which can be found at https://neurojson.org/#software

Specifically, to parse/save .jnii files in MATLAB, you should use one of

To parse/save .jnii files in Python, you should use

In Python, the volumetric data is loaded as a dict object where data['NIFTIData'] is a NumPy ndarray object storing the volumetric data.

Detected photon data

If one defines detectors, MCX is able to store a variety of photon data when a photon is captured by these detectors. One can selectively store various supported data fields, including partial pathlengths, exit position and direction, by using the -w/--savedetflag flag. The storage of detected photon information is enabled by default, and can be disabled using the -d flag.

The detected photon data are stored in a separate file from the volumetric output. The supported data file formats are explained below.

mch files

The .mch file, or MC history file, is stored by default, but we strongly encourage users to adpot the newly implemented JSON/.jdat format for easy data sharing.

The .mch file contains a 256 byte binary header, followed by a 2-D numerical array of dimensions #savedphoton * #colcount as recorded in the header.

typedef struct MCXHistoryHeader{
	char magic[4];                 // magic bits= 'M','C','X','H'
	unsigned int  version;         // version of the mch file format 
	unsigned int  maxmedia;        // number of media in the simulation 
	unsigned int  detnum;          // number of detectors in the simulation 
	unsigned int  colcount;        // how many output files per detected photon 
	unsigned int  totalphoton;     // how many total photon simulated 
	unsigned int  detected;        // how many photons are detected (not necessarily all saved) 
	unsigned int  savedphoton;     // how many detected photons are saved in this file 
	float unitinmm;                // what is the voxel size of the simulation
	unsigned int  seedbyte;        // how many bytes per RNG seed
	float normalizer;              // what is the normalization factor
	int respin;                    // if positive, repeat count so total photon=totalphoton*respin; if negative, total number is processed in respin subset 
	unsigned int  srcnum;          // number of sources for simultaneous pattern sources 
	unsigned int  savedetflag;     // number of sources for simultaneous pattern sources 
    unsigned int  totalsource;     // total source number when multiple sources are defined
	int reserved[1];               // reserved fields for future extension 
} History;

When the -q flag is set to 1, the detected photon initial seeds are also stored following the detected photon data, consisting of a 2-D byte array of #savedphoton * #seedbyte.

To load the mch file, one should call loadmch.m in MATLAB/Octave.

Saving to .mch history file is depreciated as we are transitioning towards JSON/JData formatted outputs (.jdat).

jdat files

When -F jnii is specified, instead of saving the detected photon into the legacy .mch format, a .jdat file is written, which is a pure JSON file. This file contains a hierachical data record of the following JSON structure

{
   "MCXData":{
       "Info":{
           "Version":
           "MediaNum":
           "DetNum":
           ...
           "Media":{
               ...
           }
       },
       "PhotonData":{
           "detid":
           "nscat":
           "ppath":
           "mom":
           "p":
           "v":
           "w0":
       },
       "Trajectory":{
           "photonid":
           "p":
           "w0":
       },
       "Seed":[
           ...
       ]
   }
}

where "Info" is required, and other subfields are optional depends on users' input. Each subfield in this file may contain JData 1-D or 2-D array constructs to allow storing binary and compressed data.

Although .jdat and .jnii have different suffix, they are both JSON/JData files and can be opened/written by the same JData compatible libraries mentioned above, i.e.

For MATLAB

For Python

In Python, the volumetric data is loaded as a dict object where data['MCXData']['PhotonData'] stores the photon data, data['MCXData']['Trajectory'] stores the trajectory data etc.

Photon trajectory data

For debugging and plotting purposes, MCX can output photon trajectories, as polylines, when -D M flag is attached, or mcxlab is asked for the 5th output. Such information can be stored in one of the following formats.

mct files

By default, MCX stores the photon trajectory data in to a .mct file MC trajectory, which uses the same binary format as .mch but renamed as .mct. This file can be loaded to MATLAB using the same loadmch.m function.

Using .mct file is depreciated and users are encouraged to migrate to .jdat file as described below.

jdat files

When -F jnii is used, MCX merges the trajectory data with the detected photon and seed data and saved as a JSON-compatible .jdat file. The overall structure of the .jdat file as well as the relevant parsers can be found in the above section.

Using MCXLAB in MATLAB and Octave

MCXLAB is the native MEX version of MCX for MATLAB and GNU Octave. It includes the entire MCX code in a MEX function which can be called directly inside MATLAB or Octave. The input and output files in MCX are replaced by convenient in-memory struct variables in MCXLAB, thus, making it much easier to use and interact. MATLAB/Octave also provides convenient plotting and data analysis functions. With MCXLAB, your analysis can be streamlined and simplified without involving disk files.

Please read the mcxlab/README.txt file for more details on how to install and use MCXLAB.

Please also browse this interactive Jupyter Notebook based MCXLAB tutorial to see a suite of examples showing the key functionalities of MCXLAB (using GNU Octave).

Using PMCX in Python

PMCX is the native binary binding of MCX for Python 3.6 or newer. Similar to MCXLAB, PMCX can run GPU-based simulations inside Python environment with efficient in-memory inputs and outputs.

Please read the pmcx/README.txt file for more details on how to install and use PMCX.

Please also browse this interactive Jupyter Notebook based PMCX tutorial to see a suite of examples showing the key functionalities of PMCX.

Using MCX Studio GUI

MCX Studio is a graphics user interface (GUI) for MCX. It gives users a straightforward way to set the command line options and simulation parameters. It also allows users to create different simulation tasks and organize them into a project and save for later use. MCX Studio can be run on many platforms such as Windows, GNU Linux and Mac OS.

To use MCX Studio, it is suggested to put the mcxstudio binary in the same directory as the mcx command; alternatively, you can also add the path to mcx command to your PATH environment variable.

Once launched, MCX Studio will automatically check if mcx binary is in the search path, if so, the “GPU” button in the toolbar will be enabled. It is suggested to click on this button once, and see if you can see a list of GPUs and their parameters printed in the output field at the bottom part of the window. If you are able to see this information, your system is ready to run MCX simulations. If you get error messages or not able to see any usable GPU, please check the following:

  • are you running MCX Studio/MCX on a computer with a supported card?
  • have you installed the CUDA/NVIDIA drivers correctly?
  • did you put mcx in the same folder as mcxstudio or add its path to PATH?

If your system has been properly configured, you can now add new simulations by clicking the “New” button. MCX Studio will ask you to give a session ID string for this new simulation. Then you are allowed to adjust the parameters based on your needs. Once you finish the adjustment, you should click the “Verify” button to see if there are missing settings. If everything looks fine, the “Run” button will be activated. Click on it once will start your simulation. If you want to abort the current simulation, you can click the “Stop” button.

You can create multiple tasks with MCX Studio by hitting the “New” button again. The information for all session configurations can be saved as a project file (with .mcxp extension) by clicking the “Save” button. You can load a previously saved project file back to MCX Studio by clicking the “Load” button.

Interpreting the Output

MCX output consists of two parts, the flux volume file and messages printed on the screen.

Output files

An mc2 file contains the fluence-rate distribution from the simulation in the given medium. By default, this fluence-rate is a normalized solution (as opposed to the raw probability) therefore, one can compare this directly to the analytical solutions (i.e. Green's function). The order of storage in the mc2 files is the same as the input file: i.e., if the input is row-major, the output is row-major, and so on. The dimensions of the file are Nx, Ny, Nz, and Ng where Ng is the total number of time gates.

By default, MCX produces the Green's function of the fluence rate for the given domain and source. Sometime it is also known as the time-domain “two-point” function. If you run MCX with the following command

  mcx -f input.inp -s output ....

the fluence-rate data will be saved in a file named “output.dat” under the current folder. If you run MCX without -s output, the output file will be named as input.inp.dat.

To understand this further, you need to know that a fluence-rate (Phi(r,t)) is measured by number of particles passing through an infinitesimal spherical surface per unit time at a given location regardless of directions. The unit of the MCX output is “W/mm2 = J/(mm2s)”, if it is interpreted as the “energy fluence-rate” [6], or “1/(mm2s)”, if the output is interpreted as the “particle fluence-rate” [6].

The Green's function of the fluence-rate means that it is produced by a unitary source. In simple terms, this represents the fraction of particles/energy that arrives a location per second under the radiation of 1 unit (packet or J) of particle or energy at time t=0. The Green's function is calculated by a process referred to as the “normalization” in the MCX code and is detailed in the MCX paper [6] (MCX and MMC outputs share the same meanings).

Please be aware that the output flux is calculated at each time-window defined in the input file. For example, if you type

 0.e+00 5.e-09 1e-10  # time-gates(s): start, end, step

in the 5th row in the input file, MCX will produce 50 fluence-rate snapshots, corresponding to the time-windows at [0 0.1] ns, [0.1 0.2]ns ... and [4.9,5.0] ns. To convert the fluence rate to the fluence for each time-window, you just need to multiply each solution by the width of the window, 0.1 ns in this case. To convert the time-dependent fluence-rate to continuous-wave (CW) fluence (fluence in short), you need to integrate the fluence-rate along the time dimension. Assuming the fluence-rate after 5 ns is negligible, then the CW fluence is simply sum(flux_i*0.1 ns, i=1,50). You can read mcx/examples/validation/plotsimudata.m and mcx/examples/sphbox/plotresults.m for examples to compare an MCX output with the analytical fluence-rate/fluence solutions.

One can load an .mc2 output file into Matlab or Octave using the loadmc2 function in the {mcx root}/utils folder.

To get a continuous-wave solution, run a simulation with a sufficiently long time window, and sum the flux along the time dimension, for example

   mcx=loadmc2('output.mc2',[60 60 60 10],'float');
   cw_mcx=sum(mcx,4);

Note that for time-resolved simulations, the corresponding solution in the results approximates the flux at the center point of each time window. For example, if the simulation time window setting is [t0,t0+dt,t0+2dt,t0+3dt...,t1], the time points for the snapshots stored in the solution file is located at [t0+dt/2, t0+3*dt/2, t0+5*dt/2, ... ,t1-dt/2]

A more detailed interpretation of the output data can be found at http://mcx.sf.net/cgi-bin/index.cgi?MMC/Doc/FAQ#How_do_I_interpret_MMC_s_output_data

MCX can also output “current density” (J(r,t), unit W/m^2, same as Phi(r,t)) - referring to the expected number of photons or Joule of energy flowing through a unit area pointing towards a particular direction per unit time. The current density can be calculated at the boundary of the domain by two means:

  1. using the detected photon partial path output (i.e. the second output of mcxlab.m), one can compute the total energy E received by a detector, then one can divide E by the area/aperture of the detector to obtain the J(r) at a detector (E should be calculated as a function of t by using the time-of-fly of detected photons, the E(t)/A gives J(r,t); if you integrate all time gates, the total E/A gives the current I(r), instead of the current density).
  2. use -X 1 or --saveref/cfg.issaveref option in mcx to enable the diffuse reflectance recordings on the boundary. the diffuse reflectance is represented by the current density J(r) flowing outward from the domain.

The current density has, as mentioned, the same unit as fluence rate, but the difference is that J(r,t) is a vector, and Phi(r,t) is a scalar. Both measuring the energy flow across a small area (the are has direction in the case of J) per unit time.

You can find more rigorous definitions of these quantities in Lihong Wang's Biomedical Optics book, Chapter 5.

Console print messages

Timing information is printed on the screen (stdout). The clock starts (at time T0) right before the initialization data is copied from CPU to GPU. For each simulation, the elapsed time from T0 is printed (in ms). Also the accumulated elapsed time is printed for all memory transaction from GPU to CPU.

When a user specifies -D P in the command line, or set cfg.debuglevel='P', MCX or MCXLAB prints a progress bar showing the percentage of completition.

Best practices guide

To maximize MCX's performance on your hardware, you should follow the best practices guide listed below:

Use a middle-range or enthusiastic-grade GPU, use multiple of them if possible

MCX is highly scalable, providing linear-speedup as long as you provide the GPU cores it can use. As a result, the better the GPU you use, the higher the speed you can get. An enthusiastic-grade GPU, such as RTX 4070Ti (~$700), can be 12x faster than an low-end laptop RTX 4050 GPU even within the same generation.

MCX can readily take advantage of multiple GPUs if you have it installed. The MCX simulation speed scales nearly linearly as the number of GPUs increases. So, to maximize MCX performance, get at least a middle-level or high-end consumer grade GPU; if you need more speed, throw in more GPUs will cut down the runtime.

Launch as many threads as possible

It has been shown that MCX's speed is related to the thread number (-t). Generally, the more threads, the better speed, until all GPU resources are fully occupied. For higher-end GPUs, a thread number over 10,000 is recommended. Please use the autopilot mode, -A, to let MCX determine the “optimal” thread number when you are not sure what to use.

Acknowledgement

MCX contains modified versions of the below source codes from other open-source projects (with a compatible license).

cJSON library by Dave Gamble

GLScene library for Lazarus by GLScene developers

Texture3D sample project by Jürgen Abel

Synapse communication library for Lazarus

  • Files: mcxstudio/synapse/*
  • Copyright (c) 1999-2017, Lukas Gebauer
  • URL: http://www.ararat.cz/synapse/
  • License: MIT License or LGPL version 2 or later or GPL version 2 or later
  • Comment: A subset of the Synapse units is included as part of the MCX source code tree to allow compilation of the MCX Studio binary on various platforms without needing to install the full package.

ZMat data compression unit

LZ4 data compression library

LZMA/Easylzma data compression library

  • Files: src/zmat/easylzma/*
  • Copyright: 2009, Lloyd Hilaiel, 2008, Igor Pavlov
  • License: public-domain
  • Comment: All the cruft you find here is public domain. You don't have to credit anyone to use this code, but my personal request is that you mention Igor Pavlov for his hard, high quality work.

myslicer toolbox by Anders Brun

MCX Filter submodule

pymcx Python module

Pybind11

Reference

  • [Fang2009] Qianqian Fang and David A. Boas, "Monte Carlo Simulation of Photon Migration in 3D Turbid Media Accelerated by Graphics Processing Units," Optics Express, vol. 17, issue 22, pp. 20178-20190 (2009).

  • [Yu2018] Leiming Yu, Fanny Nina-Paravecino, David Kaeli, Qianqian Fang, “Scalable and massively parallel Monte Carlo photon transport simulations for heterogeneous computing platforms,” J. Biomed. Opt. 23(1), 010504 (2018).

  • [Yan2020] Shijie Yan and Qianqian Fang* (2020), "Hybrid mesh and voxel based Monte Carlo algorithm for accurate and efficient photon transport modeling in complex bio-tissues," Biomed. Opt. Express, 11(11) pp. 6262-6270. https://www.osapublishing.org/boe/abstract.cfm?uri=boe-11-11-6262

If you use MCX in your research, the author of this software would like you to cite the above papers in your related publications.

Links:

mcx's People

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

mcx's Issues

About comments of 2nd command

In your readme file, you mentioned that "the 2nd command above will launch 2048 GPU threads (-t) with every 64 threads a block (-T); "

The 2nd command is:
mcx -t 16384 -T 64 -n 1e7 -f input.inp -s test -r 2 -g 10 -d 1 -b 1 -G 1

The thread number is 16384, not 2048.
I am confuse about it.

Thanks

mcxcl.mexw64 crash on debuglevel = 'M'

Matlab crashes when attempting to execute the mcxcl mex function when the debug level is set to 'M'. I am attempting to retrieve the photon trajectories and it is my understanding from the documentation that in order to do so the debug level must first be set to 'M'.

This is the line of code that calls the mcxlabcl function
[fluxs,detps,vol,seed,trajectory]=mcxlabcl(cfgs(2),'mcxcl');

I've attached the matlab dump file here.
I'm using the 09/26/2018 nightly build of MCXStudio for Windows.
matlab_crash_dump.17332-1.txt

Support axis-direction-specific boundary condition specifiers

MCX supports Fresnel reflection (--reflect 1) or total absorption (--reflect 0) and total reflection (setting n of media 0 to less than 1) boundary condition for all 6 facets of the domain (+x/-x/+y/-y/+z/-z). It will gain much more flexible to allow users to specify BC on a specific axis direction.

Also, a cyclic boundary could be potentially useful, although it is not very obvious when it will be used.

error in __init__.py

Hello, first thanks for the source code and for all the work.

I was playing with the program and I just encounter an error using the python init.py file.

If I don't save the photons seed I encounter an error when I try to use the load_mch python function. It look like it come from the line 107 with the variable photon_seed

If I understand it correctly the line should be just after the line 85 photon_seed.append(np.asarray(seeds).reshape((seed_byte,saved_photon), order='F')) or after, just before return mch_data, header, photon_seed

There is also some problem (or misunderstanding) with the header variable. header.medium dose not exist therefore is M = header[""maxmedia"]+1

Question about light source

Dr. Fang,

Thank you for this software. I'd like to know if there is a way to configure the photon fluence rate of the source?

Thanks,
Chaitanya

Errors using mcxlab or mcx with vol.bin files

In the mcx-users google group answer, I attached some scripts that should reproduce the errors as well as one script that should fix the mcxlab error.

First use 'test_script.m' to convert a cfg to a json and reproduce the mcxlab error.
Then, comment the line 36 in 'test_script.m' and run it again to use a modified version of 'json2mcx.m' that fixes this error.

In order to reproduce the mcx error, the 'test.sh' shell script has to be executed (with the mcx executable in the same directory) with the json file as input, that was created in the previous step by running 'test_script.m'.

Multiple GPU support

To launch MCX simulations on multiple GPUs to maximize computational throughput. This can be done similar to MCXCL, by creating multiple threads and launch kernels to multiple GPUs.

GPU required specifications

Hey, I try to run this code:

clear cfg;
cfg.unitinmm=0.1; %set the grid to 100um
cfg.nphoton=1e7;

prop=[ 0.019 7.8   0.89 1.37     % medium 1: skin & skull
   0.004 0.009 0.89 1.37     % medium 2: CSF
   0.02  9.0   0.89 1.37     % medium 3: gray matter
   0.08 40.9   0.84 1.37    % medium 4: white matter
   ];

cfg.prop=[0 0 1 1];
% define layer structure
cfg.vol=ones(500,500,1500);
j=15;
for i=1:100 % define 100 layers with random brain optical properties
cfg.vol(:,:,j:j+1)=i;
j=j+15;
randomm=randi([1,4]);
cfg.prop=[cfg.prop;prop(randomm,:)];
end
cfg.vol=uint8(cfg.vol);

% define the source position
cfg.srcpos=[200,250,0]+1;
cfg.srcdir=[0 0 1];

% time-domain simulation parameters
cfg.tstart=0;
cfg.tend=5e-9;
cfg.tstep=5e-10;

% GPU thread configuration
cfg.autopilot=1;
cfg.gpuid=1;

%% running simulation with boundary reflection enabled
cfg.isreflect=1; % enable reflection at exterior boundary
cfg.isrefint=1;  % enable reflection at interior boundary too
cfg.issavedet=1; % enable recording partial pathlength of detected photons
cfg.detpos=[101 251 1 2];
for i=1:1
tic;
[f,det,vol,seeds]=mcxlab(cfg);
toc;
end
fprintf(sprintf('simulation number: %d is done..............................',i));

And I got this warning and error.

WARNING: GPU memory can not hold all time gates, disabling normalization to allow multiple runs
MCXLAB ERROR -2 in unit mcx_core.cu:1756: out of memory
Error from thread (0): out of memory
C++ Error: MCXLAB Terminated due to an exception!
Elapsed time is 2.533999 seconds.

I know my GPU is not that strong (GEFORCE 930MX) and I am willing to upgrade my GPU but what are the required specifications of the GPU to run simulations like this properly

Scattering occurs with a zero scattering coefficient because of non-zero g

For a non-scattering medium, I expected the photons to travel in straight lines from their respective light source.
However, there seem to be some scattering events happening because of a non-zero g.
I appended one minimal example as a plot.

The file "test1.json" describes a pencil beam on the top of a non-scattering but absorbing medium heading straight into the medium (src_dir.z=-1).
The output showed photons mostly travelling in a straight line but some scattered deviations.

test1

No GPU device found

I'm using the vs project from mcx with vs2013.
I find that function "mcx_list_gpu" in mcextreme.c can't transport "mcxconfig" to mcx_core.cu because I print out value "mcxconfig.deviceid[0]" both sides and find they are different.
So, the program always report "No GPU device found".

Factor sqrt(2) missing in source type "gaussian"

Hi,
I think that there is a sqrt(2) missing in line 384 in mcx_core.cu:

  r=sqrtf(-logf(rand_uniform01(t)))*gcfg->srcparam1.x;
 // should be:
// r=sqrtf(-2 * logf(rand_uniform01(t)))*gcfg->srcparam1.x;

as the shape of the sampled Gauss curve has a smaller sigma as expected from the value given to srcparam1.x.
In other words: with the current code, the Gauss curve has a sigma of approx. 1 / sqrt(2) * srcparam1.x, which is contradicting the documentation which says that srcparam1.x is the sigma of the Gauss curve.
If you run example/srcbenchmark/run_3benchmark-gaussian.sh, you get:
output_fit
However, I do not have a math reference supporting the missing sqrt(2) factor.

Maxwell GPU may get locked in P2 state when running mcx

I noticed that my Maxwell (980Ti) GPU occasionally drops to a low-clock state after running MCX for some time. I posted my question at the CUDA forum:

https://devtalk.nvidia.com/default/topic/917213/cuda-programming-and-performance/maxwell-suddernly-becomes-10x-slower/

someone pointed out the P2 state. More googling reveals a couple of similar incidences

http://www.overclock.net/t/1553214/lower-memory-clocks-and-locked-p2-power-state-on-the-gtx-970
https://www.reddit.com/r/nvidia/comments/3au46o/gtx_970_p2_memory_clock_drops/
http://www.prepar3d.com/forum/viewtopic.php?t=109003

I am not entirely sure if this is caused by MCX on a Maxwell, or it is a bug of Maxwell itself.

Unkown error running mcxlab in mcx_core.cu

MCX studio is running perfect on my machine in the lab with a GeForce GTX 745 Cuda enabled graphics card. However, when I attempt to run any example MATLAB script with MCX I receive the following error message:

MCXLAB ERROR -30 in unit mcx_core.cu:1756: unknown error
Error from thread (0): unknown error
C++ Error: MCXLAB Terminated due to an exception!

I'm using the the 10/01/18 nightly build version of MCX.

Thanks

Non-Uniform Photon Launch Distributions from Sources

Hello Dr. Fang,

I have noticed that I am having an artifact occur when running a low scattering, Beer's law validation on mcxlab using Windows 10, NVIDIA GTX 1060, cuda 9.1, MATLAB R2017a. This artifact shows a non-uniform distribution of photons from the source when using a disk or Gaussian source (and a planar-like source based on your own diagnostics). Below is your diagnosis of the problem:

I confirm that I was able to reproduce this issue on windows 10 with
CUDA 8 using the latest mcx code. I can also confirm that this does not
show on Linux based binaries.

with a little bit more investigation, I found the issue is more extensive
than just the disk source, this also seem to happen on Gaussian
source, or planar-like sources (4 equal-distant points along the edge,
see attached figure). yes, this only happens on windows, and is
only observable with low scattering.

just by looking at this figure, I think it suggests some random number
generator issue. It looks like one of the two random numbers determining
the x/y position of the photon tends to drop to 0 for some reason.
and there is a period associated with this drop, likely some sort of
self-correlation.

I will investigate this further. The current RNG used is a new one
(xorshift128+) compared to the one when this was initially reported.
so, I am not entirely sure why this issue persists. Because xorshift128+
also involves 64bit data structures to store RNG states, so, there might
also be a possibility of the Windows nvidia driver has some sort of bug
related to 64bit data processing.

In the attached .zip file is the MATLAB code I have been using, as well as the raw results generated and some images of the fluence distribution.

Thank you for looking into this problem.

Best regards,
Jeremy
testMCX.zip

Planar source defects

When using the 'planar' source with mcxlab I receive a non-uniform planar illumination. Here's a plot of the normalized fluence on the surface of a phantom close to the source:

fluencedefects

For various parameter values, I get the same 4 defects. Changing photon number, size and shape of the planar source don't appear to make them go away, nor does adding a focal length for the source. Here's the code I use,

First the setup of the phantom:

unitinmm=.2; dim = (1/unitinmm)*[60,40, 30];
volume_center = [dim(1)/2+1;dim(2)/2+1;dim(3)/2+1];
% Defining simple rectangular phantom
object_data=uint8(ones(dim(1),dim(2), dim(3)));
object_props=[0.00001 0.0 0.9 1.33;0.003 5 0.9 1.33];
fluence=illuminate_PreXion(object_data,object_props,volume_center,unitinmm);

Matlab function that defines the illuminating geometry and calls mcx:

illuminate_PreXion.txt
(save as .m file to run)

GPU performance database and speed contest

Create and maintain an online GPU performance database by allowing users to submit timing info to a list of standardized benchmarks.

This requires:

[x] make MCXLAB aware of hardware configurations (similar to mcx -L)
[x] develop a CGI program on the MCX's web server to accept user contributed benchmark submissions
[x] write a matlab script to benchmark the GPU using MCX kernels and a standardized problem
[x] use urlread() to post the timing info and the hardware info to MCX's website
[x ] compile the results, rank by speed, and dynamically update

Segmentation violation detected

clear cfg;

cfg.unitinmm=0.1; %set the grid to 100um
cfg.nphoton=3e7;%3e7 Photons

prop=[ 0.019 7.8   0.89 1.37     % medium 1: skin & skull 7-8mm
   0.02  9.0   0.89 1.37     % medium 3: gray matter the rest
   ];

cfg.prop=[0 0 1 1];
%define layer structure
cfg.vol=ones(300,200,100);
j=1;
for h=1:70 % define 150 layers with random brain optical properties
cfg.vol(:,:,j:j+1)=h;
j=j+1;
cfg.prop=[cfg.prop;prop(1,:)];
end
for h=71:99 % define 150 layers with random brain optical properties
cfg.vol(:,:,j:j+1)=h;
j=j+1;
cfg.prop=[cfg.prop;prop(2,:)];
end
cfg.vol=uint8(cfg.vol);

% define the source position
cfg.srcpos=[150,100,0]+1;
cfg.srcdir=[0 0 1];

% time-domain simulation parameters
cfg.tstart=0;%time start
cfg.tend=10e-6;%time end
cfg.tstep=100e-9;%time step

% GPU thread configuration
cfg.autopilot=1;
cfg.gpuid='1111';

cfg.issavedet=1; % enable recording partial pathlength of detected photons
cfg.detpos=[51 101 1 2];%define the detector position(S-D= 100*0.1mm= 1cm)

fprintf('running simulation ... this takes about 20 seconds on a 4* TITAN X (Pascal) \n');

for i=1:15 %run the simulation 15 times
tic;
[f(i),det(i),vol(i),seeds(i)]=mcxlab(cfg);
toc;
clear f;
fprintf(sprintf('simulation number: %d is done..............................',i));
end

I ran this code on Ubuntu 16.04.2 LTS server with 4 TITAN X (Pascal)
and it worked.
a few days later I tried to run it again and the Matlab crashed with this error


   Segmentation violation detected at Wed Nov 14 10:33:05 2018

Configuration:
Crash Decoding : Disabled - No sandbox or build area path
Crash Mode : continue (default)
Current Graphics Driver: Unknown software
Current Visual : None
Default Encoding : UTF-8
Deployed : false
GNU C Library : 2.23 stable
Host Name : ZigZagDL
MATLAB Architecture : glnxa64
MATLAB Entitlement ID: 5144975
MATLAB Root : /usr/local/MATLAB/R2017a
MATLAB Version : 9.2.0.556344 (R2017a)
OpenGL : software
Operating System : Linux 4.4.0-77-generic #98-Ubuntu SMP Wed Apr 26 08:34:02 UTC 2017 x86_64
Processor ID : x86 Family 6 Model 79 Stepping 1, GenuineIntel
Virtual Machine : Java 1.7.0_60-b19 with Oracle Corporation Java HotSpot(TM) 64-Bit Server VM mixed mode
Window System : No active display

Fault Count: 1

Abnormal termination:
Segmentation violation

Register State (from fault):
RAX = 00007f07ef4d06c0 RBX = 00007f07ad2f4010
RCX = 00000000000009c0 RDX = 00000000000059b0
RSP = 00007f08651272c8 RBP = 00007f0865128570
RSI = 00007f07ad2f9000 RDI = 00007f07ef4d56b0

R8 = 00007f07fbb59720 R9 = 0000000000000002
R10 = 0000000000000822 R11 = 00007f087e414b50
R12 = 00007f0865128c78 R13 = 0000000000000000
R14 = 00000000000059b0 R15 = 000000000000000c

RIP = 00007f08827fe566 EFL = 0000000000010283

CS = 0033 FS = 0000 GS = 0000

Stack Trace (from fault):
[ 0] 0x00007f08827fe566 /lib/x86_64-linux-gnu/libc.so.6+01365350
[ 1] 0x00007f07b752f132 /home/student1/Eidan/mcxlab/mcx.mexa64+00168242 mexFunction+00002858
[ 2] 0x00007f0873a716a7 bin/glnxa64/libmex.so+00186023 mexRunMexFile+00000439
[ 3] 0x00007f0873a694a3 bin/glnxa64/libmex.so+00152739
[ 4] 0x00007f0873a6a295 bin/glnxa64/libmex.so+00156309
[ 5] 0x00007f0872d5c1f3 bin/glnxa64/libmwm_dispatcher.so+00782835 ZN8Mfh_file16dispatch_fh_implEMS_FviPP11mxArray_tagiS2_EiS2_iS2+00000947
[ 6] 0x00007f0872d5cabe bin/glnxa64/libmwm_dispatcher.so+00785086 ZN8Mfh_file11dispatch_fhEiPP11mxArray_tagiS2+00000030
[ 7] 0x00007f086f0814a0 bin/glnxa64/libmwm_lxe.so+11719840
[ 8] 0x00007f086f081a2b bin/glnxa64/libmwm_lxe.so+11721259
[ 9] 0x00007f086f15ba7c bin/glnxa64/libmwm_lxe.so+12614268
[ 10] 0x00007f086f106fc1 bin/glnxa64/libmwm_lxe.so+12267457
[ 11] 0x00007f086ea01a00 bin/glnxa64/libmwm_lxe.so+04905472
[ 12] 0x00007f086ea03c84 bin/glnxa64/libmwm_lxe.so+04914308
[ 13] 0x00007f086ea00224 bin/glnxa64/libmwm_lxe.so+04899364
[ 14] 0x00007f086e9fbb21 bin/glnxa64/libmwm_lxe.so+04881185
[ 15] 0x00007f086e9fbfb9 bin/glnxa64/libmwm_lxe.so+04882361
[ 16] 0x00007f086e9ffa10 bin/glnxa64/libmwm_lxe.so+04897296
[ 17] 0x00007f086e9ffadf bin/glnxa64/libmwm_lxe.so+04897503
[ 18] 0x00007f086eaf9a1c bin/glnxa64/libmwm_lxe.so+05921308
[ 19] 0x00007f086eafe083 bin/glnxa64/libmwm_lxe.so+05939331
[ 20] 0x00007f086ef80b0e bin/glnxa64/libmwm_lxe.so+10668814
[ 21] 0x00007f086f0730ce bin/glnxa64/libmwm_lxe.so+11661518
[ 22] 0x00007f0872d5c1f3 bin/glnxa64/libmwm_dispatcher.so+00782835 ZN8Mfh_file16dispatch_fh_implEMS_FviPP11mxArray_tagiS2_EiS2_iS2+00000947
[ 23] 0x00007f0872d5cabe bin/glnxa64/libmwm_dispatcher.so+00785086 ZN8Mfh_file11dispatch_fhEiPP11mxArray_tagiS2+00000030
[ 24] 0x00007f086f0814a0 bin/glnxa64/libmwm_lxe.so+11719840
[ 25] 0x00007f086f081a2b bin/glnxa64/libmwm_lxe.so+11721259
[ 26] 0x00007f086f0eb9b1 bin/glnxa64/libmwm_lxe.so+12155313
[ 27] 0x00007f086ea01a00 bin/glnxa64/libmwm_lxe.so+04905472
[ 28] 0x00007f086ea03c84 bin/glnxa64/libmwm_lxe.so+04914308
[ 29] 0x00007f086ea00224 bin/glnxa64/libmwm_lxe.so+04899364
[ 30] 0x00007f086e9fbb21 bin/glnxa64/libmwm_lxe.so+04881185
[ 31] 0x00007f086e9fbfb9 bin/glnxa64/libmwm_lxe.so+04882361
[ 32] 0x00007f086e9ffa10 bin/glnxa64/libmwm_lxe.so+04897296
[ 33] 0x00007f086e9ffadf bin/glnxa64/libmwm_lxe.so+04897503
[ 34] 0x00007f086eaf9a1c bin/glnxa64/libmwm_lxe.so+05921308
[ 35] 0x00007f086eafe083 bin/glnxa64/libmwm_lxe.so+05939331
[ 36] 0x00007f086ef80b0e bin/glnxa64/libmwm_lxe.so+10668814
[ 37] 0x00007f086ef4cdd4 bin/glnxa64/libmwm_lxe.so+10456532
[ 38] 0x00007f086ef4d14a bin/glnxa64/libmwm_lxe.so+10457418
[ 39] 0x00007f086ef4e919 bin/glnxa64/libmwm_lxe.so+10463513
[ 40] 0x00007f086efd0155 bin/glnxa64/libmwm_lxe.so+10994005
[ 41] 0x00007f086efd079d bin/glnxa64/libmwm_lxe.so+10995613
[ 42] 0x00007f0872d09ede bin/glnxa64/libmwm_dispatcher.so+00446174
[ 43] 0x00007f0872cf0628 bin/glnxa64/libmwm_dispatcher.so+00341544 ZN13Mfh_MATLAB_fn11dispatch_fhEiPP11mxArray_tagiS2+00000616
[ 44] 0x00007f086f0814a0 bin/glnxa64/libmwm_lxe.so+11719840
[ 45] 0x00007f086f081a2b bin/glnxa64/libmwm_lxe.so+11721259
[ 46] 0x00007f086f0eb9b1 bin/glnxa64/libmwm_lxe.so+12155313
[ 47] 0x00007f086ea01a00 bin/glnxa64/libmwm_lxe.so+04905472
[ 48] 0x00007f086ea03c84 bin/glnxa64/libmwm_lxe.so+04914308
[ 49] 0x00007f086ea00224 bin/glnxa64/libmwm_lxe.so+04899364
[ 50] 0x00007f086e9fbb21 bin/glnxa64/libmwm_lxe.so+04881185
[ 51] 0x00007f086e9fbfb9 bin/glnxa64/libmwm_lxe.so+04882361
[ 52] 0x00007f086e9ffa10 bin/glnxa64/libmwm_lxe.so+04897296
[ 53] 0x00007f086e9ffadf bin/glnxa64/libmwm_lxe.so+04897503
[ 54] 0x00007f086eaf9a1c bin/glnxa64/libmwm_lxe.so+05921308
[ 55] 0x00007f086eafe083 bin/glnxa64/libmwm_lxe.so+05939331
[ 56] 0x00007f086ef80b0e bin/glnxa64/libmwm_lxe.so+10668814
[ 57] 0x00007f086ef4cdd4 bin/glnxa64/libmwm_lxe.so+10456532
[ 58] 0x00007f086ef4d14a bin/glnxa64/libmwm_lxe.so+10457418
[ 59] 0x00007f086ef4e919 bin/glnxa64/libmwm_lxe.so+10463513
[ 60] 0x00007f086efc32be bin/glnxa64/libmwm_lxe.so+10941118
[ 61] 0x00007f086efc364a bin/glnxa64/libmwm_lxe.so+10942026
[ 62] 0x00007f087112441b bin/glnxa64/libmwbridge.so+00189467
[ 63] 0x00007f08711250cd bin/glnxa64/libmwbridge.so+00192717 _Z8mnParserv+00000829
[ 64] 0x00007f0873022df3 bin/glnxa64/libmwmcr.so+00552435
[ 65] 0x00007f0873024f9e bin/glnxa64/libmwmcr.so+00561054
[ 66] 0x00007f0873025619 bin/glnxa64/libmwmcr.so+00562713 _ZN5boost6detail17task_shared_stateINS_3_bi6bind_tIvPFvRKNS_8functionIFvvEEEENS2_5list1INS2_5valueIS6_EEEEEEvE6do_runEv+00000025
[ 67] 0x00007f0873023f06 bin/glnxa64/libmwmcr.so+00556806
[ 68] 0x00007f08737f1739 bin/glnxa64/libmwiqm.so+00857913
[ 69] 0x00007f08737de06c bin/glnxa64/libmwiqm.so+00778348 _ZN5boost6detail8function21function_obj_invoker0ISt8functionIFNS_3anyEvEES4_E6invokeERNS1_15function_bufferE+00000028
[ 70] 0x00007f08737ddd4c bin/glnxa64/libmwiqm.so+00777548 _ZN3iqm18PackagedTaskPlugin7executeEP15inWorkSpace_tagRN5boost10shared_ptrIN14cmddistributor17IIPCompletedEventEEE+00000428
[ 71] 0x00007f08737bbfba bin/glnxa64/libmwiqm.so+00638906
[ 72] 0x00007f08737aa15f bin/glnxa64/libmwiqm.so+00565599
[ 73] 0x00007f08737ac609 bin/glnxa64/libmwiqm.so+00574985
[ 74] 0x00007f088415201a bin/glnxa64/libmwservices.so+02469914
[ 75] 0x00007f08841535cf bin/glnxa64/libmwservices.so+02475471
[ 76] 0x00007f0884153d7a bin/glnxa64/libmwservices.so+02477434 _Z25svWS_ProcessPendingEventsiib+00000186
[ 77] 0x00007f087302356e bin/glnxa64/libmwmcr.so+00554350
[ 78] 0x00007f0873023901 bin/glnxa64/libmwmcr.so+00555265
[ 79] 0x00007f0873011206 bin/glnxa64/libmwmcr.so+00479750
[ 80] 0x00007f0882a816ba /lib/x86_64-linux-gnu/libpthread.so.0+00030394
[ 81] 0x00007f08827b782d /lib/x86_64-linux-gnu/libc.so.6+01075245 clone+00000109
[ 82] 0x0000000000000000 +00000000

This error was detected while a MEX-file was running. If the MEX-file
is not an official MathWorks function, please examine its source code
for errors. Please consult the External Interfaces Guide for information
on debugging MEX-files.

I tried to reduce the number of photons but even 1e6 won't work. any suggestions?

Use precise grid-space ray-tracing

MCX has been using a 1-grid advance step size since the beginning, derived from the tMCimg era. My previous argument regarding this inaccurate in-grid ray-tracing scheme is that after sufficient sampling of the paths, the solution will converge to the correct result. See this mailing list comment:

https://groups.google.com/forum/?hl=en#!searchin/mcx-users/precise$20/mcx-users/I7YCRf4aNnY/8qunTY6UPo4J

However, for this to converge, it may take a lot more photons than I originally expected. I recently compared MCX and MMC (which does precise ray-tracing) and found that the fluence distributions from the two solutions match very well for homogeneous solutions, but not for heterogeneous cases with scattering contrasts (inclusions with absorption contrast also matches very well). This suggested the problem of photon-path related inaccuracy exist in MCX.

Implementing precise and fast grid-space ray-tracing in MCX is expected to solve this problem.

Improve Volume Rendering Preview of Light Sources in MCXStudio

Currently, the "preview" function in the volume designer of MCXStudio could only visualize some of the different types of light sources, including Pencil, Disk, Gaussian, and etc. Some other types of source are either not shown or incorrectly represented.

photon angle (z direction) issue in mcxlab

submitted by @yaoruoyang:

In mcxlab, the output angle of a small number of photons (along z direction) have the wrong sign, we need to fix that (not sure about the cause).

Also, there's a small possibility that the angle of z is NaN (when the sum square of x angle and y angle is close to 1), this is very likely caused by the rounding number issue.

Please use the attached scrip to reproduce the issue and for debugging.

% In mcxlab, the output angle of a small number of photons (along z direction)
% have the wrong sign, we need to fix that (not sure about the cause)

% Also, there's a small possibility that the angle of z is NaN (when the
% sum square of x angle and y angle is close to 1), this is very likely
% caused by the rounding number issue.

%% test 1, reflection mode

clear cfg cfgs
cfg.nphoton=1e9;
cfg.vol=uint8(ones(60,60,60));
cfg.srcpos=[29.5, 29.5, -0.01];
cfg.srcdir=[0 0 1];

cfg.gpuid=1;
cfg.autopilot=1;

cfg.prop=[0 0 1 1;0.005 1 0 1.37];

cfg.tstart=0;
cfg.tend=5e-9;
cfg.tstep=5e-9;

cfg.issrcfrom0=1;
cfg.isreflect=1;
cfg.issaveexit=1;

cfg.detpos = [29.5, 39.5, -0.01, 3];

% calculate the flux distribution with the given config
[flux1,detp1,~,~] = mcxlab(cfg);

count1 = length(find(detp1.v(:,3)>0));
count2 = sum(isnan(detp1.v(:,3)));

%% test 2, transmission mode

clear cfg cfgs
cfg.nphoton=1e9;
cfg.vol=uint8(ones(60,60,20));
cfg.srcpos=[29.5, 29.5, -0.01];
cfg.srcdir=[0 0 1];

cfg.gpuid=1;
cfg.autopilot=1;

cfg.prop=[0 0 1 1;0.005 1 0 1.37];

cfg.tstart=0;
cfg.tend=5e-9;
cfg.tstep=5e-9;

cfg.issrcfrom0=1;
cfg.isreflect=1;
cfg.issaveexit=1;

cfg.detpos = [29.5, 29.5, 20.01, 3];

% calculate the flux distribution with the given config
[flux2,detp2,~,~] = mcxlab(cfg);

count3 = length(find(detp2.v(:,3)<0));
count4 = sum(isnan(detp2.v(:,3)));

Cannot compile drand48_r_libgwc32

I installed cygwin with gcc packages and copied the header files from libgwc32/include/glibc to cygwin/usr/include according to instructions. I still get an error. Could you please help me ?
obrazek

Typo in handling --root makes MCXStudio unable to save data files in the correct folder

This is an issue reported by Sean Huang (sean.huang at aut.ac.nz). MCXStudio completes the simulation, but the output file are not located in the desired folder, see the below email exchange, and the screen capture.

the output from MCX Studio output windows shows

unknown verbose option: --root

On 03/11/2018 05:22 PM, Sean Huang wrote:

Hi Qianqian,

Yes, after the simulation is completed in the mcxstudio, I right clicked the icon and selected ‘open output file’, I only saw ‘windows batch file, JSON file and SH file’, not the results files.

Regards,

Sean

From: Qianqian Fang [mailto:[email protected]]
Sent: Monday, 12 March 2018 10:17 AM
To: Sean Huang [email protected]
Subject: Re: MMC or MCX

On 03/11/2018 04:49 PM, Sean Huang wrote:

Hi Qianqian,

I tried the nightly build software, but the results are not saved in the output file for ploting after the run has completed?

are you saying mcx simulation is completed in mcxstudio, but it could not find the output file?

if you right click on the icon to the left panel (session icons), and select "open output folder", do you see the output files?

Qianqian

Regards,

Sean

image001

Trajectory saving only works with mcxlab

At the moment, it seems that within mcx_core.cu, cfg->exportdebugdata does nothing but saves the photon trajectories. After that, the user needs mcxlab to visualize and process the debug information. It would be good to have it directly within the mcx executable which creates a debug file with the necessary trajectory information.

Fluence artifact / bug when using gaussian source type

when using a homogeneous block absorber (without scattering) with a gaussian-type light source, the fluence distribution shows two artifacts:

  • concentric circles are visible
  • the voxels in the center (along the beam axis) have lower fluence (see attached figure)
    gaussian_fluence_artifact

Re-enable printf based debugging

The DEVICE_EMULATION mode is no longer available, but printf is directly supported for all CC>=2.0 devices. Need to define a list of useful commands to help debugging.

Hi Dr

I want to copy the methods from Minsu Kim's paper(Development of a laboratory spectral
backscattering instrument: design and simulation)and to simulate a photon from the incident beam hits the cuvette. Then I want to determine the optimal design of the backscattering measurement system. Please you give me some sugguestions if I use your software.

Support 1D and 2D simulations

Need to support 1D and 2D simulations. Only need to restrict the scattering operations and simplifications in hitgrid.

make mex

Hi, I have some issue with the make mex .

Because mex is also a LateX package I was not able to compile mcxlab.

I had to change this line:

MEX=mex

by MEX=matlab-mex and it seems to work properly.

I am on linuxMint and I have the package matlab-support distro integration for local MATLAB installations so maybe it's the problem.

I just wanted to point out this problem so other people can avoid having the same problem. Because at first I didn't understand why you would need LateX to compile the code for matlab. It took me some time to find the problem and the solution....

Compilation error on the file mcx_utils.c

Hi,

When using make I have an error for the file mcx_utils.c at the line 688. It seems to be a problem with the curly bracket befor the for . It close the function mcx_prepdomain too early.

If I delete it, it compile with no problem.

By the way thanks for sharing your code!

Reduce large floating point accumulation error in high photon simulations

MCX has been using atomic operations for fluence accumulation by default since a few years ago. However, a drop in fluence intensity in large photon simulations has been observed. For example, running the below script using the current MCX github code, you can get the below plot

clear cfg
cfg.vol=uint8(ones(60,60,60));
cfg.srcpos=[30 30 1];
cfg.srcdir=[0 0 1];
cfg.gpuid=1;
cfg.autopilot=1;
cfg.prop=[0 0 1 1;0.005 1 0 1.37];
cfg.tstart=0;
cfg.tend=5e-9;
cfg.tstep=cfg.tend;

figure
hold on;
for i=6:9
    cfg.nphoton=10^i;
    flux=mcxlab(cfg);
    plot(log10(abs(squeeze(flux.data(30,30,:)))),'-','color', rand(1,3));
end

accum_error_old

The reason for the drop in intensity was not due to data racing, like the case when non-atomic operations were used, but the accumulations of the round-off errors. In the region near the source, the energy deposit quickly increases to a large value. When adding a new energy deposit (which is a very small value) on top of a large value, the accuracy becomes a problem.

This is a serious problem because, with the increase of GPU computing capacity, most people would choose to run large photon simulations. We must be able to run large photon numbers without loosing accuracy.

There are a few solutions to such problem.

The easiest solution is to change the energy storage to double. However, consumer GPUs have extremely poor double performance, so moving to double precision addition can likely lead to drop in speed.

The standard way to sum a small values with a large floating point value is the Kahan summation. This is what we used in MMC. However, this requires multiple step operations with additional storage space. When combining with the atomic operation, atomic Kahan summation is very difficult to be implemented in the GPU.

Another idea is to use repetitions (-r) to split a large simulation into smaller chunks, and sum the solutions together. For example, for 1e9 photons with 10 respin, we run 10x 10^8 photon simulations. This can reduce the round-off error, but the repeated launch of the kernel causes a large overhead, sometimes, significantly higher than the kernel execution itself. In addition, even simulate at 1e8 photons, from the above plot, the drop in intensity remains noticeable.

A robust method is needed to obtain stable and converging solution especially at large photon numbers.

GLIBC_PRIVATE error

I downloaded the mcx binary for Linux 64-bit and I get this error on running mcx:
mcx: relocation error: mcx: symbol __get_cpu_features, version GLIBC_PRIVATE not defined in file libc.so.6 with link time reference

I am running on RedHat7.

Do I need to compile mcx from source?

Support more forms of phase functions

Currently, MCX only supports Henyey-Greenstein phase function. Need to add more phase function options to make MCX a general MC simulator. Expect only simple changes.

Use integers to represent media indices

Currently, we use an unsigned char to represent a label of a medium label. Because the highest bit of the char is used to represent the detector label, so, the max number of tissues mcx can represent is only 2^7=128.

Since there is no storage benefit to use a char variable inside the cuda kernel because of byte alignment, it probably would not add any overhead, if not faster, to use an integer array to represent the domain.

If we expand the label to 32bit, we can use a subset of bits for special masking to speed up some of the calculations.

Bug in source type "disk"

When using the source type "disk" the launched photon density has a dip in the center of the circle and along the radial axis.
output_300x300
The output has been generated using this json parameter file

The volume file is:
{ "Shapes": [ {"Name": "cubic300"}, {"Origin": [0,0,0]}, {"Grid": {"Tag":1, "Size":[300,300,60]}} ] }
and the run command:
mcx -A -f 8benchmark-disk_modified.json -r 100 -b 1 8benchmark-disk_modified "$@"

Python plot script is here

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.