Comments (50)
Thanks - I thought it was getting further, but is still failing with the capstone library error, and also seems to be stubbornly building the debug app.
I really appreciate your attention to this, btw.
from die-engine.
Success! That seems to have done the trick - thanks again for your assistance :)
Maybe consider adding release/ debug as a parameter to the build script? Not sure how much work that would be, but just adding a hard-coded CONFIG+=release might be enough. Anyway, really appreciate all your help.
from die-engine.
Well.. spoke too soon.
I'm unable to open the DiE.app - Mac whines about error -10810. I think there may still be something wrong with the build. I can upload the whole build.log if needed, since I'm not really sure what's going wrong, but here's the end of it:
Check file /srv/repos/detect-it-easy/die-source/build/release/diec: TRUE
Check file /srv/repos/detect-it-easy/die-source/build/release/die.app/Contents/MacOS/die: TRUE
error: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/install_name_tool: can't open file: /srv/repos/detect-it-easy/die-source/build/release/die.app/Contents/MacOS/diec (No such file or directory)
error: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/install_name_tool: can't open file: /srv/repos/detect-it-easy/die-source/build/release/die.app/Contents/MacOS/diec (No such file or directory)
error: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/install_name_tool: can't open file: /srv/repos/detect-it-easy/die-source/build/release/die.app/Contents/MacOS/diec (No such file or directory)
error: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/install_name_tool: can't open file: /srv/repos/detect-it-easy/die-source/build/release/die.app/Contents/MacOS/diec (No such file or directory)
error: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/install_name_tool: can't open file: /srv/repos/detect-it-easy/die-source/build/release/die.app/Contents/MacOS/diec (No such file or directory)
error: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/install_name_tool: can't open file: /srv/repos/detect-it-easy/die-source/build/release/die.app/Contents/MacOS/diec (No such file or directory)
error: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/install_name_tool: can't open file: /srv/repos/detect-it-easy/die-source/build/release/die.app/Contents/MacOS/diec (No such file or directory)
error: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/install_name_tool: can't open file: /srv/repos/detect-it-easy/die-source/build/release/die.app/Contents/MacOS/diec (No such file or directory)
error: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/install_name_tool: can't open file: /srv/repos/detect-it-easy/die-source/build/release/die.app/Contents/MacOS/diec (No such file or directory)
error: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/install_name_tool: can't open file: /srv/repos/detect-it-easy/die-source/build/release/die.app/Contents/MacOS/diec (No such file or directory)
error: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/install_name_tool: can't open file: /srv/repos/detect-it-easy/die-source/build/release/die.app/Contents/MacOS/diec (No such file or directory)
mkdir: /srv/repos/detect-it-easy/die-source/release/die_mac_portable/die.app/Contents/Frameworks: File exists
mkdir: /srv/repos/detect-it-easy/die-source/release/die_mac_portable/die.app/Contents/Frameworks: File exists
mkdir: /srv/repos/detect-it-easy/die-source/release/die_mac_portable/die.app/Contents/Frameworks: File exists
mkdir: /srv/repos/detect-it-easy/die-source/release/die_mac_portable/die.app/Contents/Frameworks: File exists
mkdir: /srv/repos/detect-it-easy/die-source/release/die_mac_portable/die.app/Contents/Frameworks: File exists
mkdir: /srv/repos/detect-it-easy/die-source/release/die_mac_portable/die.app/Contents/Frameworks: File exists
mkdir: /srv/repos/detect-it-easy/die-source/release/die_mac_portable/die.app/Contents/Frameworks: File exists
mkdir: /srv/repos/detect-it-easy/die-source/release/die_mac_portable/die.app/Contents/Frameworks: File exists
mkdir: /srv/repos/detect-it-easy/die-source/release/die_mac_portable/die.app/Contents/Frameworks: File exists
mkdir: /srv/repos/detect-it-easy/die-source/release/die_mac_portable/die.app/Contents/Frameworks: File exists
mkdir: /srv/repos/detect-it-easy/die-source/release/die_mac_portable/die.app/Contents/PlugIns/: File exists
mkdir: /srv/repos/detect-it-easy/die-source/release/die_mac_portable/die.app/Contents/PlugIns/platforms/: File exists
mkdir: /srv/repos/detect-it-easy/die-source/release/die_mac_portable/die.app/Contents/PlugIns/: File exists
mkdir: /srv/repos/detect-it-easy/die-source/release/die_mac_portable/die.app/Contents/PlugIns/platforms/: File exists
mkdir: /srv/repos/detect-it-easy/die-source/release/die_mac_portable/die.app/Contents/PlugIns/: File exists
mkdir: /srv/repos/detect-it-easy/die-source/release/die_mac_portable/die.app/Contents/PlugIns/: File exists
mkdir: /srv/repos/detect-it-easy/die-source/release/die_mac_portable/die.app/Contents/PlugIns/imageformats/: File exists
mkdir: /srv/repos/detect-it-easy/die-source/release/die_mac_portable/die.app/Contents/PlugIns/: File exists
mkdir: /srv/repos/detect-it-easy/die-source/release/die_mac_portable/die.app/Contents/PlugIns/imageformats/: File exists
mkdir: /srv/repos/detect-it-easy/die-source/release/die_mac_portable/die.app/Contents/PlugIns/: File exists
mkdir: /srv/repos/detect-it-easy/die-source/release/die_mac_portable/die.app/Contents/PlugIns/imageformats/: File exists
ditto: can't get real path for source '/srv/repos/detect-it-easy/die-source/release/die_mac_portable/.app'
zip warning: /srv/repos/detect-it-easy/die-source/release/die_mac_portable_portable_3.04.zip not found or empty
zip warning: name not matched: __MACOSX/*
zip error: Nothing to do! (/srv/repos/detect-it-easy/die-source/release/die_mac_portable_portable_3.04.zip)
This is odd since there's nothing under release/, and the entries inside build/release have a different case from what seems to be expected.
from die-engine.
from die-engine.
Yes! It built successfully, although there was the same zip error at the end, however I'm unable to open the app:
LSOpenURLsWithRole() failed with error -10810 for the file /System/Volumes/Data/srv/repos/detect-it-easy/die-source/build/release/DiE.app.
I've seen this before with various permission issues but as far as I can tell, the files are set up correctly - executable permissions, no quarantine attributes etc.
Oddly if I unzip the die_mac_portable_portable_3.04.zip file I'm able to successfully run the die.app contained there. It's just the build/release/DiE.app that won't open.
from die-engine.
This is actually running inside a linux docker container - and if I remove the call to ::convert (and all the subsequent code), everything works fine - and works up to that point in any case.
from die-engine.
Here's the line that fails:
record.colText=QApplication::palette().text().color();
I'm guessing something to do with not running in the GUI maybe? Or I haven't initialized the application correctly?
from die-engine.
Great! Just remove this line. It is only for GUI, color for detect.
from die-engine.
Fixed
horsicq/Formats@73692c8
horsicq/SpecAbstract@7e1d444
from die-engine.
Thanks again for all your hard work! I think I have enough to move forward now.
from die-engine.
Hello! Thanks a lot for your report! If you need debug version just open .pro file in qt creator or use cmake.
cmake -DCMAKE_BUILD_TYPE=Debug ..
from die-engine.
I could create mac_build_debug.sh for debug bundle but I do not know if it really necessary.
from die-engine.
Thanks for the (quick!) response. My question is more how can I build the release versions?
from die-engine.
Hmm. I always thought that build_mac makes release version. :) Are you sure?
You could use cmake here too. Just change Debug to Release
from die-engine.
I can upload the full build log but it's pretty huge. Here's the relevant bit though:
ack debug build.log
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang++ -stdlib=libc++ -headerpad_max_install_names -arch x86_64 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk -mmacosx-version-min=10.12 -Wl,-rpath,@executable_path/../Frameworks -Wl,-rpath,/usr/local/Qt-5.12.10/lib -o ../build/debug/diec consoleoutput.o main_console.o binary_script.o com_script.o die_script.o die_scriptengine.o global_script.o msdos_script.o pe_script.o ne_script.o le_script.o lx_script.o mach_script.o elf_script.o xformats.o subdevice.o xbinary.o xcom.o xmsdos.o xne.o xle.o xpe.o xmach.o xelf.o xandroidbinary.o xdex.o xarchives.o xarchive.o xmachofat.o xsevenzip.o xzip.o xcab.o xrar.o scanitem.o scanitemmodel.o xcapstone.o entropyprocess.o xoptions.o moc_consoleoutput.o moc_binary_script.o moc_com_script.o moc_die_script.o moc_die_scriptengine.o moc_global_script.o moc_msdos_script.o moc_pe_script.o moc_ne_script.o moc_le_script.o moc_lx_script.o moc_mach_script.o moc_elf_script.o moc_xformats.o moc_subdevice.o moc_xbinary.o moc_xcom.o moc_xmsdos.o moc_xne.o moc_xle.o moc_xpe.o moc_xmach.o moc_xelf.o moc_xandroidbinary.o moc_xdex.o moc_xarchives.o moc_xarchive.o moc_xmachofat.o moc_xsevenzip.o moc_xzip.o moc_xcab.o moc_xrar.o moc_scanitemmodel.o moc_xcapstone.o moc_entropyprocess.o moc_xoptions.o -F/usr/local/Qt-5.12.10/lib /System/Volumes/Data/srv/repos/detect-it-easy/die-source/XArchive/3rdparty/zlib/libs/mac/libzlib.a /System/Volumes/Data/srv/repos/detect-it-easy/die-source/XArchive/3rdparty/bzip2/libs/mac/libbzip2.a /System/Volumes/Data/srv/repos/detect-it-easy/die-source/XArchive/3rdparty/lzma/libs/mac/liblzma.a /System/Volumes/Data/srv/repos/detect-it-easy/die-source/XCapstone/3rdparty/Capstone/libs/mac/libcapstone_x86.a -framework QtScript -framework QtCore -framework DiskArbitration -framework IOKit
make[1]: *** [../build/debug/diec] Error 1
With cmake I get:
srv/repos/detect-it-easy/die-source/XCapstone
CMake Error at gui_source/CMakeLists.txt:21 (find_package):
Could not find a package configuration file provided by "QT" with any of
the following names:
Qt5Config.cmake
qt5-config.cmake
Add the installation prefix of "QT" to CMAKE_PREFIX_PATH or set "QT_DIR" to
a directory containing one of the above files. If "QT" provides a separate
development package or SDK, be sure it has been installed.
from die-engine.
Error 1 is a common error. I cannot say anything. Is there errors before the error?
It seems cmake cannot find qt5 on your machine. Can you open qt creator, create a simple project from the first template and run it or debug from qt creator IDE?
from die-engine.
I'm not using qtcreator, is the build_mac script not enough? I believe the errors are because the script has built this:
XCapstone/3rdparty/Capstone/libs/mac/libcapstone_x86d.a # x86d
which seems like it should be
XCapstone/3rdparty/Capstone/libs/mac/libcapstone_x86.a # x86
...but I just want to build the release app in any case.
from die-engine.
Yes. build_mac script should be enough
// which seems like it should be
Hmm. It is really interesting.
Could you please try to compile on your machine this project. https://github.com/horsicq/XOpcodeCalc
It uses build_mac too but does use libcapstone_x86
from die-engine.
$ find build -type f
build/debug/XOcalc.app/Contents/MacOS/XOcalc
build/debug/XOcalc.app/Contents/Resources/empty.lproj
build/debug/XOcalc.app/Contents/Resources/main.icns
build/debug/XOcalc.app/Contents/Info.plist
build/debug/XOcalc.app/Contents/PkgInfo
from die-engine.
Great! Thanks you a lot for testing. At least You have qt5 but qt5 compiles debug builds.
I will try to research it.
from die-engine.
I guess it should work if you will add to build_tools/mac.sh line 30
$QMAKE_PATH "$1" -spec $X_QMAKE_SPEC CONFIG+=x86_64
-->
$QMAKE_PATH "$1" -spec $X_QMAKE_SPEC CONFIG+=x86_64 CONFIG+=release
from die-engine.
How did you install Qt on your machine? With brew or Qt online installer?
from die-engine.
These are the steps that worked:
git clone git://code.qt.io/qt/qt5.git
cd qt5
git checkout 5.12.10
./init-repository
# create a build directory, configure qt with specific modules, and build/ install
cd ..
mkdir -p qt5.12.10
../qt5/configure -opensource -skip qtwebengine -nomake tests -nomake examples -prefix /usr/local/Qt-5.12.10
make # Note this may take the better part of a day
sudo make install # Also takes a good while
from die-engine.
I just saw this in the build log for XOpcodeCalc:
cd gui_source/ && ( test -e Makefile || /usr/local/Qt-5.12.10/bin/qmake -o Makefile /System/Volumes/Data/srv/repos/detect-it-easy/XOpcodeCalc/gui_source/gui_source.pro -spec macx-clang CONFIG+=x86_64 ) && /Applications/Xcode.app/Contents/Developer/usr/bin/make -f Makefile clean
/usr/local/Qt-5.12.10/bin/qmake -o Makefile gui_source.pro -spec macx-clang CONFIG+=x86_64
it's not passing the CONFIG+=release flag to make?
from die-engine.
Aha! Had to make clean AND get rid of the Makefiles. Seems a little happier now, will update when things finish building.
from die-engine.
Thanks a lot for your solution! BtW I added hardcoded release flag horsicq/build_tools@e517981
from die-engine.
The whole build log would be nice.
from die-engine.
Thanks a lot for the build log! Qt compiled the project successful and it is a good news. But there is errors in build_mac.sh
// This is odd since there's nothing under release/, and the entries inside build/release have a different case from what seems to be expected.
Yes. You are right. I changed name from die to DiE on macOS but forgot to fix all macOS scripts. :(
from die-engine.
I fixed it: 517e555
After scripts' work in release folder should be file die_mac_portable_portable_3.04.zip
from die-engine.
Great! Thanks a lot for the bugreport and testing. :)
from die-engine.
No problem - thanks for jumping on this!
While you're here, any idea how I can get the output as seen in the attached screenshot directly from SpecAbstract? Basically I have a JNI wrapper around StaticScan and it looks like getMsRichString() might be helpful but I have no idea what to pass in.
Thanks!
from die-engine.
SpecAbstract::SCAN_OPTIONS scanOptions={0};
SpecAbstract::SCAN_RESULT scanResult=StaticScan::processFile(sFileName,&scanOptions);
static QListXBinary::SCANSTRUCT _listRecords=SpecAbstract::convert(&(scanResult.listRecords));
ScanItemModel model(&_listRecords);
XBinary::FORMATTYPE formatType=XBinary::FORMATTYPE_TEXT;
printf("%s\n",model.toString(formatType).toLatin1().data());
from die-engine.
SpecAbstract::SCAN_OPTIONS scanOptions={0};
scanOptions.bDeepScan=true;
scanOptions.bRecursiveScan=true;
SpecAbstract::SCAN_RESULT scanResult=StaticScan::processFile(sFileName,&scanOptions);
from die-engine.
XBinary::FORMATTYPE_CSV;
XBinary::FORMATTYPE_JSON;
XBinary::FORMATTYPE_TSV;
XBinary::FORMATTYPE_XML;
If you need another formats
from die-engine.
You can create a topic here: https://github.com/horsicq/DIE-engine/discussions
If you will have any questions in the future. :)
from die-engine.
Unfortunately this line just crashes with no information:
static QListXBinary::SCANSTRUCT _listRecords=SpecAbstract::convert(&(scanResult.listRecords));
I can open a discussion/ other ticket if needed.
from die-engine.
Just remove static.
use:
QList<XBinary::SCANSTRUCT> _listRecords=SpecAbstract::convert(&(scanResult.listRecords));
from die-engine.
Same behaviour.
from die-engine.
It is code from console version of nfd https://github.com/horsicq/Nauz-File-Detector/blob/c596263386067a4d7d7c98c7e1272896b7bd817d/console_source/main_console.cpp#L57
Could you try to compile this project an check behaviour https://github.com/horsicq/Nauz-File-Detector
from die-engine.
That app seems to work fine. It seems like the convert() method isn't even being called from my code - I just get a SIGSEGV and the entire app bombs out.
SpecAbstract::SCAN_OPTIONS scanOptions = {};
SpecAbstract::SCAN_RESULT scanResult = {};
scanOptions.bRecursiveScan = true;
scanOptions.bDeepScan = true;
scanOptions.bHeuristicScan = true;
scanOptions.bAllTypesScan = true;
StaticScan staticScan;
staticScan.setData(buf, len, &scanOptions, &scanResult);
staticScan.process();
staticScan.stop();
cerr << "scanned\n";
QList<XBinary::SCANSTRUCT> _listRecords = SpecAbstract::convert(&(scanResult.listRecords));
cerr << "converted\n";
I have some diagnostics in the convert() method but they don't get output.
from die-engine.
// That app seems to work fine.
// SIGSEGV
Can you try to copy libs folder(XArchive,XCapstone) from this app build folder to build folder where SIGSEGV is.
from die-engine.
Same thing - output is below. I should explain I've got a JNI wrapper around your code, but all of this is happening inside C++, not java, so I don't think it's related to the JNI code.
[thread 140490899154688 also had an error][thread 140490898102016 also had an error]
#
# A fatal error has been detected by the Java Runtime Environment:
#
#
SIGSEGV (0xb) at pc=0x00007fc6b655884c, pid=99, tid=0x00007fc696034700
#
# JRE version: OpenJDK Runtime Environment (8.0_312-b07) (build 1.8.0_312-b07)
# Java VM: OpenJDK 64-Bit Server VM (25.312-b07 mixed mode linux-amd64 compressed oops)
# Problematic frame:
# C [libQt5Gui.so.5+0x10884c]
#
# Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#
# An error report file with more information is saved as:
# /tmp/hs_err_pid99.log
#
# If you would like to submit a bug report, please visit:
# https://bugzilla.redhat.com/enter_bug.cgi?product=Red%20Hat%20Enterprise%20Linux%208&component=java-1.8.0-openjdk
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
Here's the (AFAICT) relevant bit from the error log, but it's on the java side so might not be super useful.
Event: 0.037 Thread 0x00007fa82804c800 Exception <a 'java/lang/NoSuchMethodError': Method sun.misc.Unsafe.defineClass(Ljava/lang/String;[BII)Ljava/lang/Class; name or signature does not match> (0x000000076c987cc8) thrown at [/builddir/build/BUILD/java-1.8.0-openjdk-1.8.0.312.b0
7-2.el8_5.x86_64
Event: 0.037 Thread 0x00007fa82804c800 Exception <a 'java/lang/NoSuchMethodError': Method sun.misc.Unsafe.prefetchRead(Ljava/lang/Object;J)V name or signature does not match> (0x000000076c987fb0) thrown at [/builddir/build/BUILD/java-1.8.0-openjdk-1.8.0.312.b07-2.el8_5.x86_64/o
penjdk/hotspot/s
from die-engine.
// Basically I have a JNI wrapper
// That app seems to work fine.
Hmm. Mb just use modified console version of Nauz-File-Detector with xml or json output?
Call it from Java code https://dzone.com/articles/execute-shell-command-java and handle output?
from die-engine.
That's not a bad idea, but the reason for doing all this is for the best performance possible - I think starting a separate process may degrade performance somewhat.
from die-engine.
It may be security limits with java in macOS.
from die-engine.
Hmm. Interesting.
from die-engine.
I've been able to get some diagnostics to display, will let you know when I figure out exactly what's causing the crash.
from die-engine.
ok!
from die-engine.
I will think about it how fix it for java bind.
from die-engine.
That helped - getting further at least.
from die-engine.
Related Issues (20)
- Text very badly readable in Dark Mode (Fusion) (QT6 Version) HOT 4
- Zig binaries are unknown HOT 3
- Suggest to support calling a third-party program to open the target file HOT 2
- Compiling DIE GUI fails on Ubuntu 20.4.5 ARM64 HOT 4
- Wrong Windows 10 Version HOT 1
- Suggestion: Make ressources information accessable via commandline HOT 5
- a release zip package with only the console version HOT 2
- DLL version? HOT 2
- DIE-engine Crashed when I try to dump the TTF format HOT 1
- Suggestion: Exclude unpacked/unprotected files from directory scan results HOT 3
- submodule XTranslation access 404 HOT 3
- i cannot execute Example B1 in Geant4 iam using Ubuntu 20 :Available UI session types: [ Qt, GAG, tcsh, csh ] qt.qpa.xcb: could not connect to display :0 qt.qpa.plugin: Could not load the Qt platform plugin "xcb" in "" even though it was found. This application failed to start because no Qt platform plugin could be initialized. Reinstalling the application may fix this problem. Available platform plugins are: eglfs, linuxfb, minimal, minimalegl, offscreen, vnc, xcb. Aborted (core dumped) HOT 1
- Apple Silicon (ARM64) Mac OS Build HOT 1
- It is recommended to add a window top selection box HOT 2
- [Idea] initial signature scanning speed up HOT 1
- Automation to update the winget package? HOT 1
- Ошибка интерфейса HOT 1
- Unable to detect the specific installer for jre-8u371-windows-x64.exe. HOT 1
- CVE-2023-51714 Vulnerability HOT 4
- Does it support for arm Ubuntu HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from die-engine.