hpcraink / fsprj2 Goto Github PK
View Code? Open in Web Editor NEWFiles for Forschungsproject
License: BSD 3-Clause "New" or "Revised" License
fsprj2's People
Contributors
Stargazers
Watchers
Forkers
therealthingyfsprj2's Issues
Error handling for MPI wrapper
Enable error handling in MPI wrapper like handling of errno value for POSIX wrapper.
To check for an MPI error the return value of the MPI function and in some cases the error field in the MPI_Status struct has to be checked. The return value is MPI_SUCCESS or an implementation defined value on failures. To process the implementation defined values without implementing logic for each supported implementation the routine MPI_ERROR_CLASS translates an error code to an implementation independent error class. The error class can be checked and converted to an string with MPI_ERROR_STRING. See "MPI: A Message-Passing Interface Standard Version 3.1" page 19 and 347 ff.
The error class MPI_ERR_IN_STATUS signals that the error is provided in the MPI_Status struct. E.g. thats the case if multiple requests are completed by a single call of a MPI function and each request has to return an separate error value (see page 30 of "MPI: A Message-Passing Interface Standard Version 3.1").
Create unique id for an MPI file handle
It is possible to use the memory address of MPI_File for the MPI_File_open and the MPI_File_close functions as an id. This functions get the file handle as a reference and therefore the wrappers can log the address pointer as an unique id for the file handle. But other functions like MPI_File_read or MPI_File_write get the same file handle by value and not by reference. The memory address of MPI_File in these functions is the address of a copy of the original file handle. So the memory address is not a usable id to identify all function calls to the same file handle.
MPI_File itself can not be used as an unique id because this structure (or pointer to a structure, like in OpenMPI) is implementation defined and values in it can be changed during runtime. So it is not possible to use a value out of MPI_File or a hash or a hex dump of the whole structure (or pointer).
Style guide
defining and documenting style guide with guidelines like
- source formatting (line break, spacing, ...)
- naming conventions
Refactor existing source to fulfill style guide.
Include style guide into contributing guidelines.
MPI File-I/O Wrapper
Implementing wrapper for all MPI File-I/O functions.
Offset in MPI_File_read_at* and MPI_File_write_at* is not in bytes but in etype units relative to the current view
see "MPI: A Message-Passing Interface Standard Version 3.1" page 508. Analysis of offset must interpret set of view and etype.
fprintf is used before pointer to fprintf is initialized
In init_basic in event.c fprintf is used before pointer to fprintf is initialized. init_wrapper() has to be called before error handling is done (e.g. before first call of getenv()).
TODO: Cleanup repository
Performing basic operations (like git status) usually takes 3-5 sec (e.g., after commit) due to indexing (b/c of the large repository size).
Current possible culprits:
> git clone --depth 1 [email protected]:hpcraink/fsprj2.git
Cloning into 'fsprj2'...
remote: Enumerating objects: 368, done.
remote: Counting objects: 100% (368/368), done.
remote: Compressing objects: 100% (330/330), done.
remote: Total 368 (delta 23), reused 264 (delta 16), pack-reused 0
Receiving objects: 100% (368/368), 156.80 MiB | 11.88 MiB/s, done.
Resolving deltas: 100% (23/23), done.
Updating files: 100% (305/305), done.
> du -d 1 -h -I .git . | sort -hr
212M .
108M ./IOTrace_Analyze
53M ./dokumentation
18M ./papers
17M ./poster
10M ./Grafana
3.2M ./meetings
1.5M ./libiotrace
400K ./architecture
356K ./notes
168K ./darshan-logs
4.0K ./.github
> du -d 1 -h -I .git IOTrace_Analyze | sort -hr
108M IOTrace_Analyze/test
108M IOTrace_Analyze
460K IOTrace_Analyze/src
Possible Solutions:
- Remove superfluous stuff and use locally a shallow clone of the repo
- Break up into multiple repos (one for libiotrace, IOTrace_Analyze, etc.)
Broken Release build
Note: # set(CMAKE_BUILD_TYPE Debug) in CMakeLists.txt was commented out
20.04.3 LTS (Focal Fossa)
dev@fpj-vm:/mnt/hgfs/fpj/fsprj2/libiotrace$ rm -rf build
dev@fpj-vm:/mnt/hgfs/fpj/fsprj2/libiotrace$ mkdir build
dev@fpj-vm:/mnt/hgfs/fpj/fsprj2/libiotrace$ (cd build && cmake -DCMAKE_BUILD_TYPE=Release .. && cmake --build .)
-- The C compiler identification is GNU 9.3.0
-- The CXX compiler identification is GNU 9.3.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Performing Test HAVE_ATTRIBUTE_NONNULL
-- Performing Test HAVE_ATTRIBUTE_NONNULL - Success
-- Performing Test HAVE_ATTRIBUTE_UNUSED
-- Performing Test HAVE_ATTRIBUTE_UNUSED - Success
-- Performing Test HAVE_ATTRIBUTE_CONSTRUCTOR
-- Performing Test HAVE_ATTRIBUTE_CONSTRUCTOR - Success
-- Performing Test HAVE_ATTRIBUTE_DESTRUCTOR
-- Performing Test HAVE_ATTRIBUTE_DESTRUCTOR - Success
-- Performing Test HAVE_ATTRIBUTE_THREAD
-- Performing Test HAVE_ATTRIBUTE_THREAD - Success
-- Performing Test HAVE_DECLSPEC_THREAD
-- Performing Test HAVE_DECLSPEC_THREAD - Failed
-- Performing Test HAVE_RECVMMSG_CONST_TIMESPEC
-- Performing Test HAVE_RECVMMSG_CONST_TIMESPEC - Failed
-- Looking for include file stdbool.h
-- Looking for include file stdbool.h - found
-- Looking for include file stdlib.h
-- Looking for include file stdlib.h - found
-- Looking for include file stdint.h
-- Looking for include file stdint.h - found
-- Looking for include file sys/fcntl.h
-- Looking for include file sys/fcntl.h - found
-- Looking for include file sys/types.h
-- Looking for include file sys/types.h - found
-- Looking for include file sys/stat.h
-- Looking for include file sys/stat.h - found
-- Looking for include file sys/syscall.h
-- Looking for include file sys/syscall.h - found
-- Looking for include file unistd.h
-- Looking for include file unistd.h - found
-- Looking for include file pthread.h
-- Looking for include file pthread.h - found
-- Looking for include file aio.h
-- Looking for include file aio.h - found
-- Looking for include file getmnt.h
-- Looking for include file getmnt.h - not found
-- Looking for include file sched.h
-- Looking for include file sched.h - found
-- Looking for include file mntent.h
-- Looking for include file mntent.h - found
-- Looking for include file utmp.h
-- Looking for include file utmp.h - found
-- Looking for include file utmpx.h
-- Looking for include file utmpx.h - found
-- Looking for open64
-- Looking for open64 - found
-- Looking for openat
-- Looking for openat - found
-- Looking for creat64
-- Looking for creat64 - found
-- Looking for pread
-- Looking for pread - found
-- Looking for pread64
-- Looking for pread64 - found
-- Looking for pwrite
-- Looking for pwrite - found
-- Looking for pwrite64
-- Looking for pwrite64 - found
-- Looking for lseek64
-- Looking for lseek64 - found
-- Looking for readv
-- Looking for readv - found
-- Looking for writev
-- Looking for writev - found
-- Looking for preadv
-- Looking for preadv - found
-- Looking for preadv64
-- Looking for preadv64 - found
-- Looking for pwritev
-- Looking for pwritev - found
-- Looking for pwritev64
-- Looking for pwritev64 - found
-- Looking for preadv2
-- Looking for preadv2 - found
-- Looking for preadv64v2
-- Looking for preadv64v2 - found
-- Looking for pwritev2
-- Looking for pwritev2 - found
-- Looking for pwritev64v2
-- Looking for pwritev64v2 - found
-- Looking for copy_file_range
-- Looking for copy_file_range - found
-- Looking for mmap
-- Looking for mmap - found
-- Looking for mmap64
-- Looking for mmap64 - found
-- Looking for munmap
-- Looking for munmap - found
-- Looking for msync
-- Looking for msync - found
-- Looking for mremap
-- Looking for mremap - found
-- Looking for fopen64
-- Looking for fopen64 - found
-- Looking for freopen64
-- Looking for freopen64 - found
-- Looking for fdopen
-- Looking for fdopen - found
-- Looking for fcloseall
-- Looking for fcloseall - found
-- Looking for flockfile
-- Looking for flockfile - found
-- Looking for ftrylockfile
-- Looking for ftrylockfile - found
-- Looking for funlockfile
-- Looking for funlockfile - found
-- Looking for fwide
-- Looking for fwide - found
-- Looking for fputc_unlocked
-- Looking for fputc_unlocked - found
-- Looking for fputwc_unlocked
-- Looking for fputwc_unlocked - found
-- Looking for putc_unlocked
-- Looking for putc_unlocked - found
-- Looking for putwc_unlocked
-- Looking for putwc_unlocked - found
-- Looking for fputs_unlocked
-- Looking for fputs_unlocked - found
-- Looking for fputws_unlocked
-- Looking for fputws_unlocked - found
-- Looking for putw
-- Looking for putw - found
-- Looking for fgetc_unlocked
-- Looking for fgetc_unlocked - found
-- Looking for fgetwc_unlocked
-- Looking for fgetwc_unlocked - found
-- Looking for getc_unlocked
-- Looking for getc_unlocked - found
-- Looking for getwc_unlocked
-- Looking for getwc_unlocked - found
-- Looking for getw
-- Looking for getw - found
-- Looking for getline
-- Looking for getline - found
-- Looking for getdelim
-- Looking for getdelim - found
-- Looking for fgets_unlocked
-- Looking for fgets_unlocked - found
-- Looking for fgetws_unlocked
-- Looking for fgetws_unlocked - found
-- Looking for fread_unlocked
-- Looking for fread_unlocked - found
-- Looking for fwrite_unlocked
-- Looking for fwrite_unlocked - found
-- Looking for fwprintf
-- Looking for fwprintf - found
-- Looking for vfwprintf
-- Looking for vfwprintf - found
-- Looking for fwscanf
-- Looking for fwscanf - found
-- Looking for vfscanf
-- Looking for vfscanf - found
-- Looking for vfwscanf
-- Looking for vfwscanf - found
-- Looking for feof_unlocked
-- Looking for feof_unlocked - found
-- Looking for ferror_unlocked
-- Looking for ferror_unlocked - found
-- Looking for clearerr_unlocked
-- Looking for clearerr_unlocked - found
-- Looking for ftello
-- Looking for ftello - found
-- Looking for ftello64
-- Looking for ftello64 - found
-- Looking for fseeko
-- Looking for fseeko - found
-- Looking for fseeko64
-- Looking for fseeko64 - found
-- Looking for fgetpos64
-- Looking for fgetpos64 - found
-- Looking for fsetpos64
-- Looking for fsetpos64 - found
-- Looking for fflush_unlocked
-- Looking for fflush_unlocked - found
-- Looking for setbuffer
-- Looking for setbuffer - found
-- Looking for setlinebuf
-- Looking for setlinebuf - found
-- Looking for madvise
-- Looking for madvise - found
-- Looking for posix_madvise
-- Looking for posix_madvise - found
-- Looking for sync
-- Looking for sync - found
-- Looking for syncfs
-- Looking for syncfs - found
-- Looking for fsync
-- Looking for fsync - found
-- Looking for fdatasync
-- Looking for fdatasync - found
-- Looking for fileno
-- Looking for fileno - found
-- Looking for dup3
-- Looking for dup3 - found
-- Looking for accept4
-- Looking for accept4 - found
-- Looking for pipe2
-- Looking for pipe2 - found
-- Looking for memfd_create
-- Looking for memfd_create - not found
-- Looking for tmpfile64
-- Looking for tmpfile64 - found
-- Looking for epoll_create
-- Looking for epoll_create - found
-- Looking for epoll_create1
-- Looking for epoll_create1 - found
-- Looking for mkstemp
-- Looking for mkstemp - found
-- Looking for mkostemp
-- Looking for mkostemp - found
-- Looking for mkstemps
-- Looking for mkstemps - found
-- Looking for mkostemps
-- Looking for mkostemps - found
-- Looking for eventfd
-- Looking for eventfd - found
-- Looking for inotify_init
-- Looking for inotify_init - found
-- Looking for inotify_init1
-- Looking for inotify_init1 - found
-- Looking for vfork
-- Looking for vfork - found
-- Looking for clone
-- Looking for clone - found
-- Looking for dirfd
-- Looking for dirfd - found
-- Looking for sendmmsg
-- Looking for sendmmsg - found
-- Looking for recvmmsg
-- Looking for recvmmsg - found
-- Looking for memrchr
-- Looking for memrchr - found
-- Looking for execvpe
-- Looking for execvpe - found
-- Looking for aio_read64
-- Looking for aio_read64 - found
-- Looking for aio_write64
-- Looking for aio_write64 - found
-- Looking for aio_init
-- Looking for aio_init - found
-- Looking for lio_listio64
-- Looking for lio_listio64 - found
-- Looking for aio_error64
-- Looking for aio_error64 - found
-- Looking for aio_return64
-- Looking for aio_return64 - found
-- Looking for aio_fsync64
-- Looking for aio_fsync64 - found
-- Looking for aio_suspend64
-- Looking for aio_suspend64 - found
-- Looking for aio_cancel64
-- Looking for aio_cancel64 - found
-- Looking for dlmopen
-- Looking for dlmopen - found
-- Looking for getmntent_r
-- Looking for getmntent_r - found
-- Looking for reallocarray
-- Looking for reallocarray - found
-- Looking for off64_t
-- Looking for off64_t - not found
-- Looking for O_CLOEXEC
-- Looking for O_CLOEXEC - found
-- Looking for O_DIRECTORY
-- Looking for O_DIRECTORY - found
-- Looking for O_NOFOLLOW
-- Looking for O_NOFOLLOW - found
-- Looking for O_TMPFILE
-- Looking for O_TMPFILE - found
-- Looking for O_DIRECT
-- Looking for O_DIRECT - found
-- Looking for O_NOATIME
-- Looking for O_NOATIME - found
-- Looking for O_PATH
-- Looking for O_PATH - found
-- Looking for O_LARGEFILE
-- Looking for O_LARGEFILE - found
-- Looking for RWF_APPEND
-- Looking for RWF_APPEND - found
-- Looking for MAP_32BIT
-- Looking for MAP_32BIT - found
-- Looking for MAP_ANONYMOUS
-- Looking for MAP_ANONYMOUS - found
-- Looking for MAP_DENYWRITE
-- Looking for MAP_DENYWRITE - found
-- Looking for MAP_EXECUTABLE
-- Looking for MAP_EXECUTABLE - found
-- Looking for MAP_FILE
-- Looking for MAP_FILE - found
-- Looking for MAP_GROWSDOWN
-- Looking for MAP_GROWSDOWN - found
-- Looking for MAP_HUGETLB
-- Looking for MAP_HUGETLB - found
-- Looking for MAP_HUGE_2MB
-- Looking for MAP_HUGE_2MB - not found
-- Looking for MAP_HUGE_1GB
-- Looking for MAP_HUGE_1GB - not found
-- Looking for MAP_LOCKED
-- Looking for MAP_LOCKED - found
-- Looking for MAP_NONBLOCK
-- Looking for MAP_NONBLOCK - found
-- Looking for MAP_NORESERVE
-- Looking for MAP_NORESERVE - found
-- Looking for MAP_POPULATE
-- Looking for MAP_POPULATE - found
-- Looking for MAP_STACK
-- Looking for MAP_STACK - found
-- Looking for MAP_UNINITIALIZED
-- Looking for MAP_UNINITIALIZED - not found
-- Looking for MADV_REMOVE
-- Looking for MADV_REMOVE - found
-- Looking for MADV_DONTFORK
-- Looking for MADV_DONTFORK - found
-- Looking for MADV_DOFORK
-- Looking for MADV_DOFORK - found
-- Looking for MADV_HWPOISON
-- Looking for MADV_HWPOISON - found
-- Looking for MADV_MERGEABLE
-- Looking for MADV_MERGEABLE - found
-- Looking for MADV_UNMERGEABLE
-- Looking for MADV_UNMERGEABLE - found
-- Looking for MADV_SOFT_OFFLINE
-- Looking for MADV_SOFT_OFFLINE - not found
-- Looking for MADV_HUGEPAGE
-- Looking for MADV_HUGEPAGE - found
-- Looking for MADV_NOHUGEPAGE
-- Looking for MADV_NOHUGEPAGE - found
-- Looking for MADV_DONTDUMP
-- Looking for MADV_DONTDUMP - found
-- Looking for MADV_DODUMP
-- Looking for MADV_DODUMP - found
-- Looking for MADV_FREE
-- Looking for MADV_FREE - found
-- Looking for MADV_WIPEONFORK
-- Looking for MADV_WIPEONFORK - found
-- Looking for MADV_KEEPONFORK
-- Looking for MADV_KEEPONFORK - found
-- Looking for SEEK_DATA
-- Looking for SEEK_DATA - found
-- Looking for SEEK_HOLE
-- Looking for SEEK_HOLE - found
-- Looking for HOST_NAME_MAX
-- Looking for HOST_NAME_MAX - found
-- Looking for _POSIX_HOST_NAME_MAX
-- Looking for _POSIX_HOST_NAME_MAX - found
-- Found Git: /usr/bin/git (found version "2.25.1")
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE
-- Looking for include file linux/kcmp.h
-- Looking for include file linux/kcmp.h - found
-- Found OpenMP_C: -fopenmp (found version "4.5")
-- Found OpenMP_CXX: -fopenmp (found version "4.5")
-- Found OpenMP: TRUE (found version "4.5")
-- Configuring done
-- Generating done
-- Build files have been written to: /mnt/hgfs/fpj/fsprj2/libiotrace/build
Scanning dependencies of target iotrace_control
[ 1%] Building C object src/CMakeFiles/iotrace_control.dir/libiotrace.c.o
[ 2%] Linking C shared library libiotrace_control.so
[ 2%] Built target iotrace_control
Scanning dependencies of target iotrace_object
[ 3%] Building C object src/CMakeFiles/iotrace_object.dir/event.c.o
[ 4%] Building C object src/CMakeFiles/iotrace_object.dir/init.c.o
[ 5%] Building C object src/CMakeFiles/iotrace_object.dir/llhttp/api.c.o
[ 6%] Building C object src/CMakeFiles/iotrace_object.dir/llhttp/http.c.o
[ 7%] Building C object src/CMakeFiles/iotrace_object.dir/llhttp/llhttp.c.o
[ 8%] Building C object src/CMakeFiles/iotrace_object.dir/os.c.o
[ 9%] Building C object src/CMakeFiles/iotrace_object.dir/utils.c.o
[ 10%] Building C object src/CMakeFiles/iotrace_object.dir/posix_io.c.o
[ 10%] Built target iotrace_object
Scanning dependencies of target iotrace_static
[ 11%] Building C object src/CMakeFiles/iotrace_static.dir/event.c.o
[ 12%] Building C object src/CMakeFiles/iotrace_static.dir/init.c.o
[ 13%] Building C object src/CMakeFiles/iotrace_static.dir/llhttp/api.c.o
[ 14%] Building C object src/CMakeFiles/iotrace_static.dir/llhttp/http.c.o
[ 15%] Building C object src/CMakeFiles/iotrace_static.dir/llhttp/llhttp.c.o
[ 16%] Building C object src/CMakeFiles/iotrace_static.dir/os.c.o
[ 17%] Building C object src/CMakeFiles/iotrace_static.dir/utils.c.o
[ 18%] Building C object src/CMakeFiles/iotrace_static.dir/posix_io.c.o
In file included from /usr/include/stdio.h:864,
from /mnt/hgfs/fpj/fsprj2/libiotrace/src/posix_io.c:11:
/mnt/hgfs/fpj/fsprj2/libiotrace/src/posix_io.c:5362:13: error: expected identifier or ‘(’ before ‘__extension__’
5362 | size_t WRAP(fwrite_unlocked)(const void *data, size_t size, size_t count,
| ^~~~~~~~~~~~~~~
/mnt/hgfs/fpj/fsprj2/libiotrace/src/wrapper_defines.h:76:30: note: in expansion of macro ‘__WRAP’
76 | #define WRAP(function_macro) __WRAP(function_macro)
| ^~~~~~
/mnt/hgfs/fpj/fsprj2/libiotrace/src/posix_io.c:5362:8: note: in expansion of macro ‘WRAP’
5362 | size_t WRAP(fwrite_unlocked)(const void *data, size_t size, size_t count,
| ^~~~
make[2]: *** [src/CMakeFiles/iotrace_static.dir/build.make:154: src/CMakeFiles/iotrace_static.dir/posix_io.c.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:388: src/CMakeFiles/iotrace_static.dir/all] Error 2
make: *** [Makefile:163: all] Error 2
Failed build due to warnings in Murmur3 hash-fct
❯ rm -rf build && mkdir build && (cd build && cmake DCMAKE_BUILD_TYPE=Debug -DENABLE_OUTPUT=LOGFILE -DWITH_MPI_IO=OFF -DENABLE_REMOTE_CONTROL=OFF -DFILENAME_RESOLUTION_ENABLED=ON -DHASH_FUNCTION=MD5HASH .. && cmake --build .)
-- The C compiler identification is GNU 11.3.0
-- The CXX compiler identification is GNU 11.3.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Performing Test HAVE_ATTRIBUTE_NONNULL
-- Performing Test HAVE_ATTRIBUTE_NONNULL - Success
-- Performing Test HAVE_ATTRIBUTE_ALIGNED
-- Performing Test HAVE_ATTRIBUTE_ALIGNED - Success
-- Performing Test HAVE_DECLSPEC_ALIGNED
-- Performing Test HAVE_DECLSPEC_ALIGNED - Failed
-- Performing Test HAVE_ATTRIBUTE_UNUSED
-- Performing Test HAVE_ATTRIBUTE_UNUSED - Success
-- Performing Test HAVE_ATTRIBUTE_CONSTRUCTOR
-- Performing Test HAVE_ATTRIBUTE_CONSTRUCTOR - Success
-- Performing Test HAVE_ATTRIBUTE_DESTRUCTOR
-- Performing Test HAVE_ATTRIBUTE_DESTRUCTOR - Success
-- Performing Test HAVE_ATTRIBUTE_THREAD
-- Performing Test HAVE_ATTRIBUTE_THREAD - Success
-- Performing Test HAVE_DECLSPEC_THREAD
-- Performing Test HAVE_DECLSPEC_THREAD - Failed
-- Performing Test HAVE_RECVMMSG_CONST_TIMESPEC
-- Performing Test HAVE_RECVMMSG_CONST_TIMESPEC - Failed
-- Performing Test HAVE_TIME_RDTSC
-- Performing Test HAVE_TIME_RDTSC - Success
-- Looking for include file stdbool.h
-- Looking for include file stdbool.h - found
-- Looking for include file stdlib.h
-- Looking for include file stdlib.h - found
-- Looking for include file stdint.h
-- Looking for include file stdint.h - found
-- Looking for include file sys/fcntl.h
-- Looking for include file sys/fcntl.h - found
-- Looking for include file sys/time.h
-- Looking for include file sys/time.h - found
-- Looking for include file sys/types.h
-- Looking for include file sys/types.h - found
-- Looking for include file sys/stat.h
-- Looking for include file sys/stat.h - found
-- Looking for include file sys/syscall.h
-- Looking for include file sys/syscall.h - found
-- Looking for include file unistd.h
-- Looking for include file unistd.h - found
-- Looking for include file pthread.h
-- Looking for include file pthread.h - found
-- Looking for include file aio.h
-- Looking for include file aio.h - found
-- Looking for include file getmnt.h
-- Looking for include file getmnt.h - not found
-- Looking for include file sched.h
-- Looking for include file sched.h - found
-- Looking for include file mntent.h
-- Looking for include file mntent.h - found
-- Looking for include file utmp.h
-- Looking for include file utmp.h - found
-- Looking for include file utmpx.h
-- Looking for include file utmpx.h - found
-- Looking for __open
-- Looking for __open - not found
-- Looking for __open64
-- Looking for __open64 - not found
-- Looking for __open_2
-- Looking for __open_2 - found
-- Looking for __open64_2
-- Looking for __open64_2 - found
-- Looking for open64
-- Looking for open64 - found
-- Looking for openat
-- Looking for openat - found
-- Looking for creat64
-- Looking for creat64 - found
-- Looking for pread
-- Looking for pread - found
-- Looking for pread64
-- Looking for pread64 - found
-- Looking for pwrite
-- Looking for pwrite - found
-- Looking for pwrite64
-- Looking for pwrite64 - found
-- Looking for lseek64
-- Looking for lseek64 - found
-- Looking for readv
-- Looking for readv - found
-- Looking for writev
-- Looking for writev - found
-- Looking for preadv
-- Looking for preadv - found
-- Looking for preadv64
-- Looking for preadv64 - found
-- Looking for pwritev
-- Looking for pwritev - found
-- Looking for pwritev64
-- Looking for pwritev64 - found
-- Looking for preadv2
-- Looking for preadv2 - found
-- Looking for preadv64v2
-- Looking for preadv64v2 - found
-- Looking for pwritev2
-- Looking for pwritev2 - found
-- Looking for pwritev64v2
-- Looking for pwritev64v2 - found
-- Looking for copy_file_range
-- Looking for copy_file_range - found
-- Looking for mmap
-- Looking for mmap - found
-- Looking for mmap64
-- Looking for mmap64 - found
-- Looking for munmap
-- Looking for munmap - found
-- Looking for msync
-- Looking for msync - found
-- Looking for mremap
-- Looking for mremap - found
-- Looking for fopen64
-- Looking for fopen64 - found
-- Looking for freopen64
-- Looking for freopen64 - found
-- Looking for fdopen
-- Looking for fdopen - found
-- Looking for fcloseall
-- Looking for fcloseall - found
-- Looking for flockfile
-- Looking for flockfile - found
-- Looking for ftrylockfile
-- Looking for ftrylockfile - found
-- Looking for funlockfile
-- Looking for funlockfile - found
-- Looking for fwide
-- Looking for fwide - found
-- Looking for fputc_unlocked
-- Looking for fputc_unlocked - found
-- Looking for fputwc_unlocked
-- Looking for fputwc_unlocked - found
-- Looking for putc_unlocked
-- Looking for putc_unlocked - found
-- Looking for putwc_unlocked
-- Looking for putwc_unlocked - found
-- Looking for fputs_unlocked
-- Looking for fputs_unlocked - found
-- Looking for fputws_unlocked
-- Looking for fputws_unlocked - found
-- Looking for putw
-- Looking for putw - found
-- Looking for fgetc_unlocked
-- Looking for fgetc_unlocked - found
-- Looking for fgetwc_unlocked
-- Looking for fgetwc_unlocked - found
-- Looking for getc_unlocked
-- Looking for getc_unlocked - found
-- Looking for getwc_unlocked
-- Looking for getwc_unlocked - found
-- Looking for getw
-- Looking for getw - found
-- Looking for getline
-- Looking for getline - found
-- Looking for getdelim
-- Looking for getdelim - found
-- Looking for fgets_unlocked
-- Looking for fgets_unlocked - found
-- Looking for fgetws_unlocked
-- Looking for fgetws_unlocked - found
-- Looking for fread_unlocked
-- Looking for fread_unlocked - found
-- Looking for fwrite_unlocked
-- Looking for fwrite_unlocked - found
-- Looking for fwprintf
-- Looking for fwprintf - found
-- Looking for vfwprintf
-- Looking for vfwprintf - found
-- Looking for fwscanf
-- Looking for fwscanf - found
-- Looking for vfscanf
-- Looking for vfscanf - found
-- Looking for vfwscanf
-- Looking for vfwscanf - found
-- Looking for feof_unlocked
-- Looking for feof_unlocked - found
-- Looking for ferror_unlocked
-- Looking for ferror_unlocked - found
-- Looking for clearerr_unlocked
-- Looking for clearerr_unlocked - found
-- Looking for ftello
-- Looking for ftello - found
-- Looking for ftello64
-- Looking for ftello64 - found
-- Looking for fseeko
-- Looking for fseeko - found
-- Looking for fseeko64
-- Looking for fseeko64 - found
-- Looking for fgetpos64
-- Looking for fgetpos64 - found
-- Looking for fsetpos64
-- Looking for fsetpos64 - found
-- Looking for fflush_unlocked
-- Looking for fflush_unlocked - found
-- Looking for setbuffer
-- Looking for setbuffer - found
-- Looking for setlinebuf
-- Looking for setlinebuf - found
-- Looking for madvise
-- Looking for madvise - found
-- Looking for posix_madvise
-- Looking for posix_madvise - found
-- Looking for sync
-- Looking for sync - found
-- Looking for syncfs
-- Looking for syncfs - found
-- Looking for fsync
-- Looking for fsync - found
-- Looking for fdatasync
-- Looking for fdatasync - found
-- Looking for fileno
-- Looking for fileno - found
-- Looking for dup3
-- Looking for dup3 - found
-- Looking for accept4
-- Looking for accept4 - found
-- Looking for pipe2
-- Looking for pipe2 - found
-- Looking for memfd_create
-- Looking for memfd_create - not found
-- Looking for tmpfile64
-- Looking for tmpfile64 - found
-- Looking for epoll_create
-- Looking for epoll_create - found
-- Looking for epoll_create1
-- Looking for epoll_create1 - found
-- Looking for mkstemp
-- Looking for mkstemp - found
-- Looking for mkostemp
-- Looking for mkostemp - found
-- Looking for mkstemps
-- Looking for mkstemps - found
-- Looking for mkostemps
-- Looking for mkostemps - found
-- Looking for eventfd
-- Looking for eventfd - found
-- Looking for inotify_init
-- Looking for inotify_init - found
-- Looking for inotify_init1
-- Looking for inotify_init1 - found
-- Looking for vfork
-- Looking for vfork - found
-- Looking for clone
-- Looking for clone - found
-- Looking for dirfd
-- Looking for dirfd - found
-- Looking for sendmmsg
-- Looking for sendmmsg - found
-- Looking for recvmmsg
-- Looking for recvmmsg - found
-- Looking for memrchr
-- Looking for memrchr - found
-- Looking for execvpe
-- Looking for execvpe - found
-- Looking for aio_read64
-- Looking for aio_read64 - found
-- Looking for aio_write64
-- Looking for aio_write64 - found
-- Looking for aio_init
-- Looking for aio_init - found
-- Looking for lio_listio64
-- Looking for lio_listio64 - found
-- Looking for aio_error64
-- Looking for aio_error64 - found
-- Looking for aio_return64
-- Looking for aio_return64 - found
-- Looking for aio_fsync64
-- Looking for aio_fsync64 - found
-- Looking for aio_suspend64
-- Looking for aio_suspend64 - found
-- Looking for aio_cancel64
-- Looking for aio_cancel64 - found
-- Looking for dlmopen
-- Looking for dlmopen - found
-- Looking for getmntent_r
-- Looking for getmntent_r - found
-- Looking for reallocarray
-- Looking for reallocarray - found
-- Looking for off64_t
-- Looking for off64_t - not found
-- Looking for O_CLOEXEC
-- Looking for O_CLOEXEC - found
-- Looking for O_DIRECTORY
-- Looking for O_DIRECTORY - found
-- Looking for O_NOFOLLOW
-- Looking for O_NOFOLLOW - found
-- Looking for O_TMPFILE
-- Looking for O_TMPFILE - found
-- Looking for O_DIRECT
-- Looking for O_DIRECT - found
-- Looking for O_NOATIME
-- Looking for O_NOATIME - found
-- Looking for O_PATH
-- Looking for O_PATH - found
-- Looking for O_LARGEFILE
-- Looking for O_LARGEFILE - found
-- Looking for RWF_APPEND
-- Looking for RWF_APPEND - found
-- Looking for MAP_32BIT
-- Looking for MAP_32BIT - found
-- Looking for MAP_ANONYMOUS
-- Looking for MAP_ANONYMOUS - found
-- Looking for MAP_DENYWRITE
-- Looking for MAP_DENYWRITE - found
-- Looking for MAP_EXECUTABLE
-- Looking for MAP_EXECUTABLE - found
-- Looking for MAP_FILE
-- Looking for MAP_FILE - found
-- Looking for MAP_GROWSDOWN
-- Looking for MAP_GROWSDOWN - found
-- Looking for MAP_HUGETLB
-- Looking for MAP_HUGETLB - found
-- Looking for MAP_HUGE_2MB
-- Looking for MAP_HUGE_2MB - not found
-- Looking for MAP_HUGE_1GB
-- Looking for MAP_HUGE_1GB - not found
-- Looking for MAP_LOCKED
-- Looking for MAP_LOCKED - found
-- Looking for MAP_NONBLOCK
-- Looking for MAP_NONBLOCK - found
-- Looking for MAP_NORESERVE
-- Looking for MAP_NORESERVE - found
-- Looking for MAP_POPULATE
-- Looking for MAP_POPULATE - found
-- Looking for MAP_STACK
-- Looking for MAP_STACK - found
-- Looking for MAP_UNINITIALIZED
-- Looking for MAP_UNINITIALIZED - not found
-- Looking for MADV_REMOVE
-- Looking for MADV_REMOVE - found
-- Looking for MADV_DONTFORK
-- Looking for MADV_DONTFORK - found
-- Looking for MADV_DOFORK
-- Looking for MADV_DOFORK - found
-- Looking for MADV_HWPOISON
-- Looking for MADV_HWPOISON - found
-- Looking for MADV_MERGEABLE
-- Looking for MADV_MERGEABLE - found
-- Looking for MADV_UNMERGEABLE
-- Looking for MADV_UNMERGEABLE - found
-- Looking for MADV_SOFT_OFFLINE
-- Looking for MADV_SOFT_OFFLINE - not found
-- Looking for MADV_HUGEPAGE
-- Looking for MADV_HUGEPAGE - found
-- Looking for MADV_NOHUGEPAGE
-- Looking for MADV_NOHUGEPAGE - found
-- Looking for MADV_DONTDUMP
-- Looking for MADV_DONTDUMP - found
-- Looking for MADV_DODUMP
-- Looking for MADV_DODUMP - found
-- Looking for MADV_FREE
-- Looking for MADV_FREE - found
-- Looking for MADV_WIPEONFORK
-- Looking for MADV_WIPEONFORK - found
-- Looking for MADV_KEEPONFORK
-- Looking for MADV_KEEPONFORK - found
-- Looking for SEEK_DATA
-- Looking for SEEK_DATA - found
-- Looking for SEEK_HOLE
-- Looking for SEEK_HOLE - found
-- Looking for HOST_NAME_MAX
-- Looking for HOST_NAME_MAX - found
-- Looking for _POSIX_HOST_NAME_MAX
-- Looking for _POSIX_HOST_NAME_MAX - found
-- Found PkgConfig: /usr/bin/pkg-config (found version "0.29.2")
-- Checking for module 'openssl'
-- Found openssl, version 3.0.2
-- Found Git: /usr/bin/git (found version "2.34.1")
-- Looking for include file linux/kcmp.h
-- Looking for include file linux/kcmp.h - found
-- Found OpenMP_C: -fopenmp (found version "4.5")
-- Found OpenMP_CXX: -fopenmp (found version "4.5")
-- Found OpenMP: TRUE (found version "4.5")
-- Found OpenMP_C: -fopenmp (found version "4.5")
-- Found OpenMP_CXX: -fopenmp (found version "4.5")
-- Configuring done
-- Generating done
-- Build files have been written to: /home/user/fsprj2/libiotrace/build
[ 1%] Building C object src/libs/atomic_hash/CMakeFiles/atomic_hash_object.dir/atomic_hash.c.o
[ 1%] Building C object src/libs/atomic_hash/CMakeFiles/atomic_hash_object.dir/hash_functions/hash_md5.c.o
/home/user/fsprj2/libiotrace/src/libs/atomic_hash/hash_functions/hash_md5.c: In function ‘md5hash’:
/home/user/fsprj2/libiotrace/src/libs/atomic_hash/hash_functions/hash_md5.c:8:3: error: ‘MD5’ is deprecated: Since OpenSSL 3.0 [-Werror=deprecated-declarations]
8 | MD5((unsigned char *)s, (size_t)len, (unsigned char *)r);
| ^~~
In file included from /home/user/fsprj2/libiotrace/src/libs/atomic_hash/hash_functions/hash_md5.c:2:
/usr/include/openssl/md5.h:52:38: note: declared here
52 | OSSL_DEPRECATEDIN_3_0 unsigned char *MD5(const unsigned char *d, size_t n,
| ^~~
cc1: all warnings being treated as errors
gmake[2]: *** [src/libs/atomic_hash/CMakeFiles/atomic_hash_object.dir/build.make:90: src/libs/atomic_hash/CMakeFiles/atomic_hash_object.dir/hash_functions/hash_md5.c.o] Error 1
gmake[1]: *** [CMakeFiles/Makefile2:465: src/libs/atomic_hash/CMakeFiles/atomic_hash_object.dir/all] Error 2
gmake: *** [Makefile:166: all] Error 2Share IDE settings in repository?
E.g. settings to comply with the style guide could be shared in the repository.
get details from MPI_Comm
Get rank and size (and group?) from MPI_Comm and write these values as JSON for further analysis.
Currently unknown (not traced) open function(s)
Compilation flags/command: (cd build && cmake -DCMAKE_BUILD_TYPE=Debug -DWITH_FILENAME_RESOLUTION=ON -DENABLE_OUTPUT=LOGFILE -DWITH_MPI_IO=ON -DWITH_DL_IO=ON .. && cmake --build .)
Test command: (cd build/test && strace -f mpirun -np 1 -x IOTRACE_LOG_NAME=MPI_read_test1 -x LD_PRELOAD=../src/libiotrace_shared.so ./MPI_read 2> strace_MPI_read.txt ; code ../../test/MPI_read.c MPI_read_test1_iotrace.log strace_MPI_read.txt)
libiotrace (MPI_read_test1_iotrace.log) log excerpt:
{"traced_filename":"/usr/lib/x86_64-linux-gnu/pmix/share/pmix-mca-params.conf","hostname":"fpj-vm","process_id":16215,"thread_id":16215,"function_name":"close","time_start":1632320981264436116,"time_end":1632320981265070605,"return_state":"ok","wrapper":{"time_start":1632320981264435012,"time_end":1632320981271215912},"file_type":{"descriptor":11}}
{"traced_filename":"_ NOT FOUND _","hostname":"fpj-vm","process_id":16215,"thread_id":16215,"function_name":"mmap","time_start":1632320981271772410,"time_end":1632320981272323069,"return_state":"ok","wrapper":{"time_start":1632320981271771384,"time_end":1632320981277274017},"file_type":{"descriptor":11},"function_data":{"address":"0x7f5a3f43c000","length":4096,"offset":0,"protection_flags":["read"],"map_flags":["shared"]}}
{"traced_filename":"_ NOT FOUND _","hostname":"fpj-vm","process_id":16215,"thread_id":16215,"function_name":"close","time_start":1632320981277276042,"time_end":1632320981277934664,"return_state":"ok","wrapper":{"time_start":1632320981277275520,"time_end":1632320981284268631},"file_type":{"descriptor":11}}
{"traced_filename":"_ NOT FOUND _","hostname":"fpj-vm","process_id":16215,"thread_id":16215,"function_name":"mmap","time_start":1632320981286066275,"time_end":1632320981286571993,"return_state":"ok","wrapper":{"time_start":1632320981286064570,"time_end":1632320981291210756},"file_type":{"descriptor":11},"function_data":{"address":"0x7f5a3f43b000","length":4096,"offset":0,"protection_flags":["read","written"],"map_flags":["shared"]}}
{"traced_filename":"_ NOT FOUND _","hostname":"fpj-vm","process_id":16215,"thread_id":16215,"function_name":"close","time_start":1632320981291213039,"time_end":1632320981292019567,"return_state":"ok","wrapper":{"time_start":1632320981291212594,"time_end":1632320981298055931},"file_type":{"descriptor":11}}
{"traced_filename":"_ NOT FOUND _","hostname":"fpj-vm","process_id":16215,"thread_id":16215,"function_name":"mmap","time_start":1632320981298580093,"time_end":1632320981299359030,"return_state":"ok","wrapper":{"time_start":1632320981298579177,"time_end":1632320981304699822},"file_type":{"descriptor":11},"function_data":{"address":"0x7f5a3f43a000","length":4096,"offset":0,"protection_flags":["read"],"map_flags":["shared"]}}
{"traced_filename":"_ NOT FOUND _","hostname":"fpj-vm","process_id":16215,"thread_id":16215,"function_name":"close","time_start":1632320981304702107,"time_end":1632320981305226969,"return_state":"ok","wrapper":{"time_start":1632320981304701660,"time_end":1632320981310364369},"file_type":{"descriptor":11}}
strace (strace_MPI_read.txt) log excerpt:
[pid 16215] openat(AT_FDCWD, "/tmp/ompi.fpj-vm.1000/pid.16211/pmix_dstor_ds12_16211/dstore_sm.lock", O_RDWR <unfinished ...>
[pid 16211] write(1, "<<libiotrace>> [pid=16215] [DEBU"..., 392 <unfinished ...>
[pid 16215] <... openat resumed>) = 11
[pid 16211] <... write resumed>) = 392
[pid 16215] mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_SHARED, 11, 0 <unfinished ...>
[pid 16211] poll([{fd=5, events=POLLIN}, {fd=3, events=POLLIN}, {fd=7, events=POLLIN}, {fd=20, events=POLLIN}, {fd=0, events=POLLIN}, {fd=25, events=POLLIN}, {fd=21, events=POLLIN}], 7, -1 <unfinished ...>
[pid 16215] <... mmap resumed>) = 0x7f5a40579000
[pid 16215] getpid() = 16215
[pid 16215] write(1, "\n", 1) = 1
[pid 16211] <... poll resumed>) = 1 ([{fd=21, revents=POLLIN}])
[pid 16215] write(1, "[pid = 16215] CALLED W/ mmap\n", 29 <unfinished ...>
[pid 16211] read(21, <unfinished ...>
[pid 16215] <... write resumed>) = 29
[pid 16211] <... read resumed>"\n[pid = 16215] CALLED W/ mmap\n", 4096) = 30
[pid 16215] getpid( <unfinished ...>
[pid 16211] poll([{fd=5, events=POLLIN}, {fd=3, events=POLLIN}, {fd=7, events=POLLIN}, {fd=20, events=POLLIN}, {fd=0, events=POLLIN}, {fd=25, events=POLLIN}, {fd=1, events=POLLOUT}, {fd=21, events=POLLIN}], 8, -1 <unfinished ...>
Note: This tracing issue occured most likely during MPI_init.
Fildes 11 gets close'd (line 1 in libiotrace log) and subsequently mmap'ed (line 2), which doesn't make any sense.
As shown in line 3 of the strace log, a openat syscall has been invoked, indicating that there's an unknown open function (as part of MPI_init), which currently isn't covered by our tracing.
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.