armijnhemel / binaryanalysis Goto Github PK

This is the output when I try to extract the firmware DVRF_v03.bin.
It works until it finishes yaffs2 unpacking. I guess after that exe unpack starts, right? what goes wrong? are there any debug info, because multiprocess totally masks error messages?

DVRF_v03.bin -o wasd
COPYING BEGIN 2016-11-30T20:22:22.677171
COPYING END 2016-11-30T20:22:22.982368
PRERUN UNPACK BEGIN 2016-11-30T20:22:26.223154
bat.prerun verifyELF /vagrant/tmp/unpack/tmpSMyDEz/data/DVRF_v03.bin 2016-11-30T20:22:26.460714
bat.prerun searchXML /vagrant/tmp/unpack/tmpSMyDEz/data/DVRF_v03.bin 2016-11-30T20:22:26.463020
bat.prerun verifyRSACertificate /vagrant/tmp/unpack/tmpSMyDEz/data/DVRF_v03.bin 2016-11-30T20:22:26.465210
bat.prerun verifyTZ /vagrant/tmp/unpack/tmpSMyDEz/data/DVRF_v03.bin 2016-11-30T20:22:26.465367
bat.prerun verifyChromePak /vagrant/tmp/unpack/tmpSMyDEz/data/DVRF_v03.bin 2016-11-30T20:22:26.468490
bat.prerun verifyVimSwap /vagrant/tmp/unpack/tmpSMyDEz/data/DVRF_v03.bin 2016-11-30T20:22:26.468603
bat.prerun verifyAndroidXML /vagrant/tmp/unpack/tmpSMyDEz/data/DVRF_v03.bin 2016-11-30T20:22:26.468688
bat.prerun verifyIco /vagrant/tmp/unpack/tmpSMyDEz/data/DVRF_v03.bin 2016-11-30T20:22:26.468782
bat.prerun verifyMessageCatalog /vagrant/tmp/unpack/tmpSMyDEz/data/DVRF_v03.bin 2016-11-30T20:22:26.468909
bat.prerun verifyCertificate /vagrant/tmp/unpack/tmpSMyDEz/data/DVRF_v03.bin 2016-11-30T20:22:26.469004
bat.prerun verifyAndroidResource /vagrant/tmp/unpack/tmpSMyDEz/data/DVRF_v03.bin 2016-11-30T20:22:26.469154
bat.prerun verifyTerminfo /vagrant/tmp/unpack/tmpSMyDEz/data/DVRF_v03.bin 2016-11-30T20:22:26.469247
bat.fwunpack searchUnpackByteSwap /vagrant/tmp/unpack/tmpSMyDEz/data/DVRF_v03.bin 2016-11-30T20:22:26.471940
bat.fwunpack searchUnpackAndroidSparseDataImage /vagrant/tmp/unpack/tmpSMyDEz/data/DVRF_v03.bin 2016-11-30T20:22:26.580553
bat.fwunpack searchUnpackExt2fs /vagrant/tmp/unpack/tmpSMyDEz/data/DVRF_v03.bin 2016-11-30T20:22:26.580788
bat.fwunpack searchUnpackYaffs2 /vagrant/tmp/unpack/tmpSMyDEz/data/DVRF_v03.bin 2016-11-30T20:22:26.604661
Process Process-3:
Traceback (most recent call last):
File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
self._target(*self._args, **self.kwargs)
File "/usr/local/lib/python2.7/dist-packages/bat/bruteforcescan.py", line 650, in scan
scanres = locals()["bat%s" % method](filetoscan, tempdir, blacklist, offsets, newenv, debug=debug)
File "/usr/local/lib/python2.7/dist-packages/bat/fwunpack.py", line 1902, in searchUnpackYaffs2
yaffsres = unpackYaffs(filename, offset, tmpdir)
File "/usr/local/lib/python2.7/dist-packages/bat/fwunpack.py", line 1929, in unpackYaffs
p = subprocess.Popen(['bat-unyaffs', '-b', filename, '-d', tmpdir, '-j', '-n', '%d' % offset], stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
File "/usr/lib/python2.7/subprocess.py", line 710, in init
errread, errwrite)
File "/usr/lib/python2.7/subprocess.py", line 1327, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory

Im getting and error on ubuntu 16.04

$ bat-scan -c bat-scan.config -b ~/Downloads/T2/U/usb_backup_upgrade.bin -o TEST
output file already exists
slobodan@G31M-ES2L:~/binaryanalysis/src$ bat-scan -c bat-scan.config -b ~/Downloads/T2/U/usb_backup_upgrade.bin -o Test
Process Process-3:
Traceback (most recent call last):
  File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/lib/python2.7/dist-packages/bat/bruteforcescan.py", line 650, in scan
    scanres = locals()["bat_%s" % method](filetoscan, tempdir, blacklist, offsets, newenv, debug=debug)
  File "/usr/local/lib/python2.7/dist-packages/bat/fwunpack.py", line 1902, in searchUnpackYaffs2
    yaffsres = unpackYaffs(filename, offset, tmpdir)
  File "/usr/local/lib/python2.7/dist-packages/bat/fwunpack.py", line 1929, in unpackYaffs
    p = subprocess.Popen(['bat-unyaffs', '-b', filename, '-d', tmpdir, '-j', '-n', '%d' % offset], stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
  File "/usr/lib/python2.7/subprocess.py", line 711, in __init__
    errread, errwrite)
  File "/usr/lib/python2.7/subprocess.py", line 1343, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory

Hi,

when I run bat-scan -b test.bin I get the following error message

usage: bat-scan [options]
bat-scan: error: Need configuration file

what is the root cause for that?

There is a note in the code suggesting their are many incompatible magic module versions. I could not find documentation anywhere in the package or project website about requirements, or which version of this module may be required. Could someone suggest which version works?

I am trying to run some of the tools in the bat suite, but I ran into an issue with the python-magic/libmagic library.

$ bat-scan
Traceback (most recent call last):
File "/usr/local/bin/bat-scan", line 16, in
import bat.bruteforcescan
File "/usr/local/lib/python2.7/dist-packages/bat/bruteforcescan.py", line 68, in
ms = magic.open(magic.MAGIC_NONE)
AttributeError: 'module' object has no attribute 'open'

the python-magic api specify from_file function to read a file instead of the open function, which is used in this case. What version of libmagic/python-magic is this tool dependent on?

thanks.

I tried to build bat 36.0 to apply the fixed file but the following error occurred. :

$ git clone https://github.com/armijnhemel/binaryanalysis.git
Cloning into 'binaryanalysis'...
remote: Counting objects: 26603, done.
remote: Compressing objects: 100% (156/156), done.
remote: Total 26603 (delta 101), reused 0 (delta 0), pack-reused 26447
Receiving objects: 100% (26603/26603), 4.07 MiB | 728.00 KiB/s, done.
Resolving deltas: 100% (15107/15107), done.
Checking connectivity... done.
$ cd binaryanalysis/src
binaryanalysis/src$ debuild -uc -us
dpkg-buildpackage -rfakeroot -D -us -uc
dpkg-buildpackage: export CFLAGS from dpkg-buildflags (origin: vendor): -g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Wformat-security
dpkg-buildpackage: export CPPFLAGS from dpkg-buildflags (origin: vendor): -D_FORTIFY_SOURCE=2
dpkg-buildpackage: export CXXFLAGS from dpkg-buildflags (origin: vendor): -g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Wformat-security
dpkg-buildpackage: export FFLAGS from dpkg-buildflags (origin: vendor): -g -O2
dpkg-buildpackage: export LDFLAGS from dpkg-buildflags (origin: vendor): -Wl,-Bsymbolic-functions -Wl,-z,relro
dpkg-buildpackage: source package bat
dpkg-buildpackage: source version 36.0
dpkg-buildpackage: source changed by Armijn Hemel [email protected]
dpkg-source --before-build src
dpkg-buildpackage: host architecture amd64
fakeroot debian/rules clean
dh clean
dh_testdir
dh_auto_clean
running clean
'build/lib.linux-x86_64-2.7' does not exist – can't clean it
'build/bdist.linux-x86_64' does not exist – can't clean it
'build/scripts-2.7' does not exist – can't clean it
dh_clean
dpkg-source -b src
dpkg-source: warning: no source format specified in debian/source/format, see dpkg-source(1)
dpkg-source: warning: source directory 'src' is not - 'bat-36.0'
dpkg-source: info: using source format `1.0'
dpkg-source: info: building bat in bat_36.0.tar.gz
dpkg-source: info: building bat in bat_36.0.dsc
debian/rules build
dh build
dh_testdir
dh_auto_configure
dh_auto_build
running build
running build_py
creating build
creating build/lib.linux-x86_64-2.7
creating build/lib.linux-x86_64-2.7/bat
copying bat/bruteforcescan.py -> build/lib.linux-x86_64-2.7/bat
copying bat/unpackrpm.py -> build/lib.linux-x86_64-2.7/bat
copying bat/fwunpack.py -> build/lib.linux-x86_64-2.7/bat
copying bat/busyboxversion.py -> build/lib.linux-x86_64-2.7/bat
copying bat/file2package.py -> build/lib.linux-x86_64-2.7/bat
copying bat/findduplicates.py -> build/lib.linux-x86_64-2.7/bat
copying bat/prerun.py -> build/lib.linux-x86_64-2.7/bat
copying bat/fixduplicates.py -> build/lib.linux-x86_64-2.7/bat
copying bat/licenseversion.py -> build/lib.linux-x86_64-2.7/bat
copying bat/jffs2.py -> build/lib.linux-x86_64-2.7/bat
copying bat/ext2.py -> build/lib.linux-x86_64-2.7/bat
copying bat/reportcopyright.py -> build/lib.linux-x86_64-2.7/bat
copying bat/extractor.py -> build/lib.linux-x86_64-2.7/bat
copying bat/piecharts.py -> build/lib.linux-x86_64-2.7/bat
copying bat/generatejson.py -> build/lib.linux-x86_64-2.7/bat
copying bat/fsmagic.py -> build/lib.linux-x86_64-2.7/bat
copying bat/busybox.py -> build/lib.linux-x86_64-2.7/bat
copying bat/identifier.py -> build/lib.linux-x86_64-2.7/bat
copying bat/kernelanalysis.py -> build/lib.linux-x86_64-2.7/bat
copying bat/generatehexdump.py -> build/lib.linux-x86_64-2.7/bat
copying bat/renamefiles.py -> build/lib.linux-x86_64-2.7/bat
copying bat/kernelsymbols.py -> build/lib.linux-x86_64-2.7/bat
copying bat/generatereports.py -> build/lib.linux-x86_64-2.7/bat
copying bat/fssearch.py -> build/lib.linux-x86_64-2.7/bat
copying bat/init.py -> build/lib.linux-x86_64-2.7/bat
copying bat/elfcheck.py -> build/lib.linux-x86_64-2.7/bat
copying bat/security.py -> build/lib.linux-x86_64-2.7/bat
copying bat/guireport.py -> build/lib.linux-x86_64-2.7/bat
copying bat/javacheck.py -> build/lib.linux-x86_64-2.7/bat
copying bat/findlibs.py -> build/lib.linux-x86_64-2.7/bat
copying bat/checks.py -> build/lib.linux-x86_64-2.7/bat
copying bat/images.py -> build/lib.linux-x86_64-2.7/bat
copying bat/prunefiles.py -> build/lib.linux-x86_64-2.7/bat
copying bat/batxor.py -> build/lib.linux-x86_64-2.7/bat
copying bat/derivekernelconfig.py -> build/lib.linux-x86_64-2.7/bat
copying bat/generateimages.py -> build/lib.linux-x86_64-2.7/bat
copying bat/interfaces.py -> build/lib.linux-x86_64-2.7/bat
running build_scripts
creating build/scripts-2.7
error: maintenance/batconfigcheck.py: No such file or directory
dh_auto_build: python setup.py build --force returned exit code 1
make: *** [build] Error 1
dpkg-buildpackage: error: debian/rules build gave error exit status 2
debuild: fatal error at line 1350:
dpkg-buildpackage -rfakeroot -D -us -uc failed

this is info when i tried to debuild the deb:
darcy@darcy-pc:~/binaryanalysis/src$ sudo debuild -uc -us
dpkg-buildpackage -rfakeroot -D -us -uc
dpkg-buildpackage: warning: using a gain-root-command while being root
dpkg-buildpackage: info: source package bat
dpkg-buildpackage: info: source version 36.0
dpkg-buildpackage: info: source distribution unstable
dpkg-buildpackage: info: source changed by Armijn Hemel [email protected]
dpkg-source --before-build src
dpkg-buildpackage: info: host architecture amd64
fakeroot debian/rules clean
dh clean --with python2
dh: Compatibility levels before 9 are deprecated (level 7 in use)
dh_testdir
dh_auto_clean
dh_auto_clean: Compatibility levels before 9 are deprecated (level 7 in use)
dh_auto_clean: pyversions -d failed [1]
debian/rules:13: recipe for target 'clean' failed
make: *** [clean] Error 1
dpkg-buildpackage: error: fakeroot debian/rules clean gave error exit status 2
debuild: fatal error at line 1376:
dpkg-buildpackage -rfakeroot -D -us -uc failed