filipdominec / python-meep-install

Robust compilation procedure of latest MEEP and python-meep on Linux

Important note:

As of 2017, the compilation script was reported to fail on Ubuntu 17.04 due to numpy API deprecation.

I recommend to try the new official guide at https://meep.readthedocs.io/en/latest/Build_From_Source/#building-from-source

---

Overview of python-meep-install

Setting up the electromagnetic simulation environment based on MEEP (http://ab-initio.mit.edu/wiki/index.php/Meep) is not straightforward. Author has spent several days making the simulation work on different 32/64-bit systems, use HDF5 libraries with multiprocessing support etc., and this experience has motivated the publication of this script.

It should automatically install MEEP, Python-meep and related programs/libraries at different linux distributions.

Use

Invocation

On supported systems, you can simply run the 'python-meep-install.sh' script as root, and after 5-10 minutes you should get a working environment.

If everything works and you can 'import meep_mpi' in your Python2 console, do not forget there are practical examples of its use at https://github.com/FilipDominec/python-meep-utils

Testing and debugging

If you wish to file a bug report, it is important to record the whole output of the installation script. To this end, call it as such:

sudo ./python-meep-install.sh 2>&1 | tee logs/`date +%y%m%d`-`lsb_release -sd | tr ' ' '_' `-`uname -m`.log

You may either attach the file to an e-mail to the author, or start a new issue on github. Your contribution will be welcome!

User options

By editing the --- Settings ----- section at the beginning of the 'python-meep-install.sh' script, you can choose which implementation of the MPI protocol will be used (openmpi, mpich, mpich2), or choose none to disable multiprocessing.

One can also select whether Python2 or Python3 is used, but the latter does not work. The current version of the python-meep sources fails to install with Python3 (tested on Ubuntu 18.04 and Fedora28). For more details, see issue #12.

System compatibility:

Tested to work fully on:

• Ubuntu: 12.10, 14.10, 15.04, 15.10 (64-bit)
• Ubuntu: 16.04, 16.10, 17.04 (64-bit) (these versions had previously issues with guile and friends, they appear to work, however)
• Debian Jessie

Experimental/partially working:

• Python-meep works also perfectly on Ubuntu 16.04 (64-bit) and 17.04 (64-bit), but the meep scheme interpreter fails since it is compiled against "too new" version of guile 2.0.13 (see https://github.com/stevengj/meep/issues/57). Previous Ubuntu versions include guile 2.0.11 or older, with which meep (or meep-mpi) work fine.

Known not to work on:

• Ubuntu: 18.04 (64-bit) compiles without error, but Python 2 reports: ImportError: No module named weave
• Fedora 28 (and other RPM-based systems)
  • On import meep_mpi, I get an error AttributeError: module object has no attribute 'weave'.
    • 2018-07-30 Using python2, this is critical error: I cannot make python-meep to work on Fedora 28 (64-bit)
    • 2018-07-30 Using python3, meep could not be compiled (due to 'PyFile_Check' incompatibility in the installer, see above)
  • Additionally multiprocessing on Redhat-based distros requires manual enabling after installation (tested with openmpi only)
    • make sure that openmpi is there: dnf install openmpi (actually the installer does this for you)
    • as suggested at https://ask.fedoraproject.org/en/question/59399/module-command-not-found-in-fedora-21/ one needs to run as root: dnf install environment-modules-4.1.3*
    • log in to a new terminal session as a normal user
    • then module load mpi/openmpi-x86_64
    • module load mpi and you can run your meep-based scripts. Note this is apparently not needed on Ubuntu/Debian.
  • optional library 'harminv' is missing, can be perhaps manually compiled if needed
  • dependency 'pkg-config' is reported missing (not known if this is serious)
• Windows+Cygwin: h5utils did not compile correctly due to libpng error? (reported June 2017, triaged)

Possible issues

• When some installation step fails, you will perhaps notice it at the end of the whole procedure since Python-meep compilation will be missing some dependencies. You will have to compare your compilation output against the files in the log/ directory.
• The eigenmode source requires the 'mpb-dev' package, which is not available in Ubuntu 15.04 or earlier. You can still download the source and compile MPB on older systems, but in such a case, make sure you use the '-fPIC' compiler option.
• Downloading of libctl-3.2.1.tar.gz seems to hang for several minutes sometimes (May 2017). Not a serious trouble, wait for it.
• Sometimes the apt-get installer refuses to work, reporting either that "the dpkg database is locked" or that it "can not find the source code for ...". I guess this is due to the updater launching during the run of the installation script. It may help to re-run the script again.

python-meep-install's Issues

meep-python did not install

171009-Ubuntu16.04-x86_64.log
Hello I need to use meep in python. this is the log file. It would be really appreciaterd if you could help me. I have anaconda 3 installed on ubuntu 16.04. i can import meep in python, but mpb and other functions like vector3 do not work. Thanks
vishal.log

h5utils not compiling with Cygwin

From @emma-branigan on June 29, 2017 11:43

I've been trying to install Meep and Python-Meep on Win7 with Cygwin by following the tutorial linked below:
http://novelresearch.weebly.com/installing-meep-in-windows-8-via-cygwin.html

I've also had a look at two other tutorials:
https://www.mail-archive.com/[email protected]/msg03793.html
http://thread.gmane.org/gmane.comp.science.electromagnetism.meep.general/3428

All three seem to be similar but with small differences. I'm unsure as to which method will best install Meep on a Windows 7 platform. When following the 1st tutorial I didn't get past the first hurdle. When compiling h5utils, the following error was the result:

make[1]: Entering directory '/cygdrive/c/cygwin/usr/local/h5utils-1.12.1'
gcc -DHAVE_CONFIG_H -I. -I/usr/local/include -g -O2 -MT writepng.o -MD -MP -MF .deps/writepng.Tpo -c -o writepng.o writepng.c
writepng.c: In function ‘writepng’:
writepng.c:312:24: error: dereferencing pointer to incomplete type ‘png_struct {aka struct png_struct_def}’
if (setjmp(png_ptr->jmpbuf)) {
^
writepng.c:437:19: error: dereferencing pointer to incomplete type ‘png_info {aka struct png_info_def}’
free(info_ptr->palette);
^
make[1]: *** [Makefile:426: writepng.o] Error 1
make[1]: Leaving directory '/cygdrive/c/cygwin/usr/local/h5utils-1.12.1'
make: *** [Makefile:278: all] Error 2

The author dismisses any errors with h5utils in this tutorial saying that MatLab is more capable for processing. However, it seems that due to this failure h5topng is not found and without having access to MatLab, my outputs are useless.

When trying to run 'ring.ctl' from the Meep/examples folder I also get the following:

creating output file "./ring-eps-000000.00.h5"...
Backtrace:
In /pub/devel/guile/yaakov/guile-2.0.14-1.i686/src/guile-2.0.14/module/ice-9/boot-9.scm:
160: 12 [catch #t #<catch-closure 801d0950> ...]
In unknown file:
?: 11 [apply-smob/1 #<catch-closure 801d0950>]
In /pub/devel/guile/yaakov/guile-2.0.14-1.i686/src/guile-2.0.14/module/ice-9/eval.scm:
432: 10 [eval # #]
432: 9 [eval # #]
In unknown file:
?: 8 [primitive-load "ring.ctl"]
In /pub/devel/guile/yaakov/guile-2.0.14-1.i686/src/guile-2.0.14/module/ice-9/eval.scm:
432: 7 [eval # #]
In /pub/devel/guile/yaakov/guile-2.0.14-1.i686/src/guile-2.0.14/module/ice-9/boot-9.scm:
710: 6 Exception thrown while printing backtrace:
ERROR: In procedure delete-meep-volume: Wrong type argument in position 1: #<finalized smob 80447d00>
ERROR: In procedure %run-finalizers:
ERROR: In procedure delete-meep-volume: Wrong type argument in position 1: #<finalized smob 80447ef0>
Some deprecated features have been used. Set the environment
variable GUILE_WARN_DEPRECATED to "detailed" and rerun the
program to get more information. Set it to "no" to suppress
this message.

Any thoughts on what might be happening with Meep would be very helpful. It's been a number of days trying to get it to install correctly on both Windows and Ubuntu.

Emma

Copied from original issue: FilipDominec/python-meep-utils#7

`mpb_verbosity' can not be used when making a shared object; recompile with -fPIC

/usr/bin/ld: /usr/local/lib/libmpb.a(mpi_utils.o): warning: relocation against stdout@@GLIBC_2.2.5' in read-only section .text'
/usr/bin/ld: /usr/local/lib/libmpb.a(libmatrices_la-eigensolver.o): relocation R_X86_64_PC32 against symbol `mpb_verbosity' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: final link failed: bad value
collect2: error: ld returned 1 exit status
make[4]: *** [Makefile:590: libmeep.la] Error 1
make[4]: Leaving directory '/home/zulnorain/meep-1.28.0/src'
make[3]: *** [Makefile:728: all-recursive] Error 1
make[3]: Leaving directory '/home/zulnorain/meep-1.28.0/src'
make[2]: *** [Makefile:511: all] Error 2
make[2]: Leaving directory '/home/zulnorain/meep-1.28.0/src'
make[1]: *** [Makefile:518: all-recursive] Error 1
make[1]: Leaving directory '/home/zulnorain/meep-1.28.0'
make: *** [Makefile:425: all] Error 2

actually i'm trying to install meep for c and C++ on linux ubuntu i have all the required dependencies but still facing this issue while installing meep.
i'm having this issues i tried every possible method but can't, if anybody know kindly guide.

ImportError: No module named weave

On Ubuntu 18.04 and Fedora 28, I observed that the installation finishes correctly, but I cannot use meep. This seems independent of multiprocessing options.

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/dist-packages/meep_mpi.py", line 5988, in <module>
    import scipy.weave
ImportError: No module named weave

Unable to locate package serial-bin

Non-MPI installation reports

E: Unable to locate package serial-bin

at the place where the MPI-aware installation reports

openmpi-bin is already the newest version (1.10.2-8ubuntu1).

some hpp files missing

On Saturday 18 June I downloaded and ran sudo python-meep-install.sh on ubuntu17.04. The install failed with the following message:

swigging meep_mpi.i to meep_mpi_wrap.cpp
swig -python -c++ -w503,389,302,362,314,509,451 -I/usr/local/include -o meep_mpi_wrap.cpp meep_mpi.i
meep_common.i:87: Error: Unable to find 'meep/mympi.hpp'
meep_common.i:88: Error: Unable to find 'meep/vec.hpp'
meep_common.i:89: Error: Unable to find 'meep.hpp'
error: command 'swig' failed with exit status 1

I ran the sh script before the Guile fix (2.013-->2.011) and it worked fine. The fix must have changed something.

error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

ubuntu_16.04.log.txt
there is an error when I execute the python-meep-install.sh.
does this error matters?
need some help.

Does it work with python 3?

Hi, I am wondering that whether this meep will work with python 3 script?

Thanks a lot!

Guile (2.0.13) is incompatible with meep: Temporary workaround

It appears that the latest version of Guile (2.0.13) is incompatible with meep. The first meep tutorial (straight waveguide) does not run with Guile 2.0.13 installed. Here is a summary of what I did to get it to work (in single processor mode)

1. Remove Guile 2.0.13 using apt remove
2. Download deb packages for Guile 2.0.11, Guile-dev 2.0.11, Guile-libs 2.0.11
3. Use dpkg -i for installation in the following sequence: (a) Guile-lib, (b) Guile, (c) Guile-dev
4. Use the python-meep-install.sh script, but edit it so as to choose the “serial” option and comment out the line that invokes installation of guile.

I found that the install script runs to completion. Even though the meep mode is “serial” (single processor, not multiprocessor) the command to run meep is still meep-mpi.
A check of the guile version shows 2.0.11 installed.

The first tutorial on the Meep tutorial web page (straight wave guide) ran without error and generated correct h5 files. The uses h5utils to convert the h5 files to .png files ran without error.

ubuntu 16.04: missing -fPIC in MPB and "Unknown SWIG preprocessor directive"

Hi @FilipDominec,
I'm having issues getting things to work on the latest ubuntu. Attached is the log file.

Thanks
ubuntu-16.04.txt

Thanks

Sorry, I created this issue just for saying thanks.
I used to have MEEP on 15.10, but failed to install manually on 16.04. You saved my life. I had a issue about import scipy.weave (my scipy is 0.19.1), but I replaced it by import weave. All is good now.
Thanks,

no module named meep

Hi

after running the install command and trying to run the waveguide sample python throws an error saying "no module named meep". Python finds the meep_mpi module but can't find the meep module. Any reason for this? Also when I try and run the straight waveguide sample using meep_mpi instead I get an error "MPI_ABORT was invoked on rank 0 in communicator MPI_COMM_WORLD with errorcode 1", so I can't run it using meep_mpi. Meep_mpi works though when I run the python script from "http://photonics.intec.ugent.be/download/pub_2579.pdf" Running on ubuntu 14.04. Thanks

Best
Yanni

Updates for Fedora 23

Thanks for providing this script; I've been fighting with building meep-mpi for Fedora for quite some time until I found your project.

I saw that you had partially written a conversion to allow the use of yum-based systems. I made a few more tweaks, and the script now runs happily to build meep-mpi with openmpi on Fedora 23.

I've uploaded a diff with the changes I made. Primarily, I updated a few more installation names, the installation commands, and specified the MPI compilers on the autogen command (which gets around the problem seen in the previous issue.

CC=/lib64/openmpi/bin/mpicc CXX=/lib64/openmpi/bin/mpicxx F77=/lib64/openmpi/bin/mpif77 ./autogen.sh $meep_opt --enable-maintainer-mode --enable-shared --prefix=/usr/local # exits with 1 ?

I was also able to get copies of h5utils and harminv from the fedora copr repository hmaarrfk/meep, so that those tools are available.

python-meep-install-fedora23.txt

does it work on Fedora 22 ?

Hi,

I'm having lots of issues installing python-meep on Fedora 22. I succesfully installed MEEP --with-mpi option, but have had no luck installing python meep.

In particular, it fails when I do a ./make-mpi like this:

swig -python -c++ -w503,389,302,362,314,509,451 -I/usr/local/include -o meep_mpi_wrap.cpp meep_mpi.i
meep-site-init.py:1: Error: Unknown SWIG preprocessor directive: initialisations (if this is a block of target language code, delimit it with %{ and %})

Any ideas?

I was able to get it a bit further by changing Line1 of meep-site-init.py
from: #initialisations specific for .... ....
to: /#initialisations specific for .... ..../

Output looks ilke this:

sudo ./make-mpi
Directories for SWIG -I option : ['/usr/local/include']
Directories for -L option : ['/usr/local/lib']
running clean
removing 'build/lib.linux-x86_64-2.7' (and everything under it)
'build/bdist.linux-x86_64' does not exist -- can't clean it
'build/scripts-2.7' does not exist -- can't clean it
removing 'build'
Directories for SWIG -I option : ['/usr/local/include']
Directories for -L option : ['/usr/local/lib']
running build_ext
building '_meep_mpi' extension
swigging meep_mpi.i to meep_mpi_wrap.cpp
swig -python -c++ -w503,389,302,362,314,509,451 -I/usr/local/include -o meep_mpi_wrap.cpp meep_mpi.i
creating build
creating build/temp.linux-x86_64-2.7
gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/local/include -I/usr/include/python2.7 -c meep_mpi_wrap.cpp -o build/temp.linux-x86_64-2.7/meep_mpi_wrap.o
In file included from /usr/include/numpy/ndarraytypes.h:1804:0,
from /usr/include/numpy/ndarrayobject.h:17,
from /usr/include/numpy/arrayobject.h:4,
from custom.hpp:19,
from meep_mpi_wrap.cpp:3937:
/usr/include/numpy/npy_1_7_deprecated_api.h:15:2: warning: #warning "Using deprecated NumPy API, disable it by " "#defining NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]
#warning "Using deprecated NumPy API, disable it by "
....

continues for some time, but finally says:
g++ -pthread -shared -Wl,-z,relro build/temp.linux-x86_64-2.7/meep_mpi_wrap.o -L/usr/local/lib -L/usr/lib64 -lmeep_mpi -lpython2.7 -o build/lib.linux-x86_64-2.7/_meep_mpi.so
/bin/ld: cannot find -lmeep_mpi
collect2: error: ld returned 1 exit status
error: command 'g++' failed with exit status 1

Any ideas? I have very little clue about how python-meep is put together, but I badly need this to work!!

Thanks in advance,