Comments (8)
I'm using zlib/1.2.7 and hdf5/1.8.10, and compiling for the compute nodes (not front nodes).
You are one up on me; IOR with HDF5 won't even compile for me on BG/Q. :) Did you have to make any modifications to get it to build?
I'm using zlib/1.2.7 and hdf5/1.8.10, and compiling for the compute nodes (not front nodes).
Your screenshot says 1.8.8. Are you sure about that?
PS: As per contract with IBM, I was told by my line manager that we have to use to the version 2.10.3 of IOR... is this version fully compatible with Blue Gene Q? Any known issues?
Sorry, I haven't used that version of IOR in at least two years. Off the top of my head, I know I changed the way IOR gets the host name to identify tasks per node. I think that made IOR's output a little more sane. I cannot remember if anything was more clearly broken than that.
from ior.
You are right, in my test I used 1.8.8 but I've just tested 1.8.10 and it returns the same error.
About the compilation process, see if this helps:
I didn't have to modify the source code, just past the parameters into the ./configure and make steps.
To be clear, I use modules to load the zlib and hdf5 dependencies.
For the compute nodes compilation, I loaded our compiler (wrapper/xl).
Here are the variables set by the modules:
HDF5
setenv HDF5_ROOT {bgsys}/local/hdf5/1.8.10
setenv HDF5_INCDIR {bgsys}/local/hdf5/1.8.10/include
setenv HDF5_LIBDIR {bgsys}/local/hdf5/1.8.10/lib
setenv HDF5_LINK_OPTS -L{bgsys}/local/hdf5/1.8.10/lib -lhdf5
ZLIB
module-whatis loads the zlib environment for BG/Q CNs
setenv ZLIB_ROOT {bgsys}/local/zlib/1.2.7
setenv ZLIB_INCDIR {bgsys}/local/zlib/1.2.7/include
setenv ZLIB_LIBDIR {bgsys}/local/zlib/1.2.7/lib
setenv ZLIB_LINK_OPTS -L{bgsys}/local/zlib/1.2.7/lib -lz
Compiler
module-whatis loads the BG/Q mpi xl wrappers environment
prepend-path PATH {bgsys}/drivers/ppcfloor/comm/xl/bin
prepend-path MANPATH {bgsys}/drivers/ppcfloor/comm/xl/man
setenv CC {bgsys}/drivers/ppcfloor/comm/xl/bin/mpixlc
setenv CXX {bgsys}/drivers/ppcfloor/comm/xl/bin/mpixlcxx
setenv FC {bgsys}/drivers/ppcfloor/comm/xl/bin/mpixlf90
In order to configure it for hdf5 building, I run:
./configure --with-hdf5=yes CFLAGS=-I$HDF5_INCDIR LDFLAGS="-L$HDF5_LIBDIR -L$ZLIB_LIBDIR" LIBS="-lm"
Then I just run make normally, and the make command would come up like this:
mpixlc -I/bgsys/local/hdf5/1.8.10/include -L/bgsys/local/hdf5/1.8.10/lib -L/bgsys/local/zlib/1.2.7/lib -o ior ior.o utilities.o parse_options.o aiori-POSIX.o aiori-MPIIO.o aiori-HDF5.o -lhdf5 -lz -lm
I noticed now that I get two warning in the compilation process:
"aiori-HDF5.c", line 551.30: 1506-098 (E) Missing argument(s).
"aiori-HDF5.c", line 555.34: 1506-098 (E) Missing argument(s).
This is the function where the execution dies, maybe its related to this issue?
Then when running, I get the following output:
HDF5-DIAG: Error detected in HDF5 (1.8.10) MPI-process 0:
#000: H5D.c line 152 in H5Dcreate2(): not link creation property list
major: Invalid arguments to routine
minor: Inappropriate type
.
IOR-3.0.1: MPI Coordinated Test of Parallel I/O
Began: Tue Jul 16 09:47:10 2013
Command line used: ./IOR/src/ior -a HDF5
Machine: CNK bgqio10-ib
Test 0 started: Tue Jul 16 09:47:10 2013
Summary:
api = HDF5-1.8.10 (Parallel)
test filename = testFile
access = single-shared-file
ordering in a file = sequential offsets
ordering inter file= no tasks offsets
clients = 1 (1 per node)
repetitions = 1
xfersize = 262144 bytes
blocksize = 1 MiB
aggregate filesize = 1 MiB
access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
** error **
ERROR in aiori-HDF5.c (line 553): cannot create data set.
** exiting **
Hope this helps
from ior.
I noticed now that I get two warning in the compilation process:
"aiori-HDF5.c", line 551.30: 1506-098 (E) Missing argument(s).
"aiori-HDF5.c", line 555.34: 1506-098 (E) Missing argument(s).This is the function where the execution dies, maybe its related to this issue?
Yes, I would very much suspect that to be the case. The "(E)" denotes: "Error conditions exist that the compiler can correct, but the program might not produce the expected results."
I am a bit surprised that the compiler doesn't halt, actually. I would dig into that error.
from ior.
Ok thank you. I will start looking into that. Let me know if you need help with the BGQ compilation.
from ior.
I too am surprised you could build this far.
"aiori-HDF5.c", line 555.34: 1506-098 (E) Missing argument(s).
means IOR is trying to use old-style HDF5 arguments. There is a backwards compatiblity option, though:
add -DH5_USE_16_API to your cflags.
Maybe IOR should just do that automatically. I shudder to think anyone is actually running HDF5-1.6.x, but even if they were that define would not affect them.
from ior.
I guess some day I'll set up my github life so I can "send a pull request. Until then, I blat patches into the issue tracker:
From bf979035e87250970e022527d44bf745ff20cf87 Mon Sep 17 00:00:00 2001
From: Rob Latham <[email protected]>
Date: Thu, 18 Jul 2013 21:31:54 +0000
Subject: [PATCH] backwards compatiblity macro
Since aiori-HDF5 uses old 1.6-style function calls (create, open, etc),
turn on the backwards compatiblity here (the one place HDF5 calls are
made)
---
src/aiori-HDF5.c | 3 +++
1 files changed, 3 insertions(+), 0 deletions(-)
diff --git a/src/aiori-HDF5.c b/src/aiori-HDF5.c
index 2f222f0..7649be0 100644
--- a/src/aiori-HDF5.c
+++ b/src/aiori-HDF5.c
@@ -19,6 +19,9 @@
#include <stdio.h> /* only for fprintf() */
#include <stdlib.h>
#include <sys/stat.h>
+/* HDF5 routines here still use the old 1.6 style. Nothing wrong with that but
+ * save users the trouble of passing this flag through configure */
+#define H5_USE_16_API
#include <hdf5.h>
#include <mpi.h>
--
1.7.1
from ior.
Thanks, Rob! I added that change to master.
from ior.
It worked!! Thank you for you help Rob.
from ior.
Related Issues (19)
- Missing #include "utilities.h" in parse_options.c HOT 1
- File locking when using hdf5, collective mode HOT 2
- feature request: take hints from environment HOT 1
- Not linking due to undefined reference to gpfs_fcntl HOT 6
- ior hintfile is too sensitive to whitespace
- When IOR deletes an MPI-IO file, it might not delete it HOT 3
- next ior release? HOT 6
- HDF5 multi-dimensional datasets configuration
- ior unlinks files from / HOT 1
- POSIX on single-shared-file write tests HOT 5
- Error building IOR HOT 1
- test->filePerProc is ignored in MPIIO_GetFileSize
- "Not a valid data type or _FillValue type mismatch" error when run NCMPI version
- PLEASE KILL THIS REPO
- cross-compiling IOR HOT 1
- fix IOR-hdf5 driver compile error HOT 1
- no need for "less than 2 GiB check" in pnetcdf version of ior HOT 1
- Error in HDF5 benchmark on ubuntu 12.04: PMPI_Bcast(1438): Invalid datatype and PMPI_Bcast(1448): Null Datatype pointer HOT 5
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from ior.