dorigom / yaffs2utils Goto Github PK

yaffs2utils_endian.c loads byteorder.h, which is linux only, and prevents from 
compiling.


ignoring the header seems to work on mac


#ifdef __MACH__ 
#else 
#include <asm/byteorder.h>
#endif /* __MACH__ */ 

I believe the equivalent functionality can be found in 

#include <libkern/OSByteOrder.h> 

I recently had to extract the contents of a yaffs2 image file and found that 
none of the three revisions of yaffs2utils available could extract the image. 
This was caused a great distress with me as I've previously used an older 
revision of the software to extract an image. I tried all three "current 
downloads" versions (the initial release wouldn't compile) as both a regular 
user and as root. None extracted the file and all displayed error messages 
varying from a segmentation fault to almost no output. 

I was fortunate to find a binary of unyaffs that I had saved previously and 
could use it to successfully extract the image. The version that works is so 
old it doesn't take any option other than the path to the image file and 
extracts to the current working directory. Unfortunately I'm unable to find the 
version of the binary but I last modified it on 2010-12-03.

I should expect that the newer versions of yaffs2utils should be able to 
extract the image file where a previous version could. Please let me know what 
information I can provide to you in order to help troubleshoot this issue. Note 
that I won't provide the image file used as it contains private data but I'll 
be happy to provide any information about that file you wish.

Thanks,
Greg

i am doing some stuff extracting image file from a phone.
as i entered adb shell and use mount to find out the file system of /system 
partition is yaffs2   . so i tryed dd if=/dev/mtd/mtd4 of=/sdcard/system.img 
and cat /dev/mtd/mtd4 > /sdcard/system.img and pull to my computer(running 
fedora linux 16) .

but i can not extract the two image files .unyass2 output image file size is 
not a miltiple of 20048+32 (or other number i dont remember)

some says this file system built in my phone is modified base on yaffs2 file 
system.i wonder how to extract these image files.

What steps will reproduce the problem?
1. Make oob-file read-only
2. run unyaffs2 -o <oobfile> ...

What is the expected output? What do you see instead?
Instead of extracting the image, the command fails with "parse oob image failed"

What version of the product are you using? On what operating system?
Latest SVN revision (23)

Please provide any additional information below.
strace output:
open("/etc/yaffs2/oob", O_RDWR)     = -1 EROFS (Read-only file system)
write(2, "parse oob image failed\n", 23parse oob image failed
) = 23
write(1, "unyaffs2-0.2.0: image extracting"..., 49) = 49

The file is being opened with O_RDWR, while O_RDONLY is sufficient.

Steps to reproduce:
  unyaffs2 -f blah -p 2048 <image> <dir>
Expected:
  Unknown option: -f
Seen:
  <garbage in output>

Reason:
  if getopt_long encounters an unknown option, it will return '?' and the "default' clause of the switch() is hit. In that clause argv[option_index] is printed, rather than argv[optind].

$ gcc -v
...
Target: x86_64-alt-linux
...
gcc version 4.9.2 ...

$ make
...
gcc -I. -I./yaffs2 -O2 -Wall -Wshadow -Winline -Wpointer-arith -Wnested-externs 
-Wwrite-strings -Wstrict-prototypes -Wmissing-declarations -Wmissing-prototypes 
-Wredundant-decls -D_HAVE_MMAP   -c -o unyaffs2.o unyaffs2.c
unyaffs2.c: In function ‘unyaffs2_obj_chattr’:
unyaffs2.c:763:2: warning: ignoring return value of ‘lchown’, declared with 
attribute warn_unused_result [-Wunused-result]
  lchown(fpath, obj->uid, obj->gid);
  ^
In file included from /usr/include/string.h:654:0,
                 from unyaffs2.c:26:
In function ‘memcpy’,
    inlined from ‘unyaffs2_extract_hardlink.constprop’ at unyaffs2.c:1222:2,
    inlined from ‘unyaffs2_extract_obj’ at unyaffs2.c:1289:10:
/usr/include/bits/string3.h:51:3: error: call to __builtin___memcpy_chk will 
always overflow destination buffer
   return __builtin___memcpy_chk (__dest, __src, __len, __bos0 (__dest));
   ^
make: *** [unyaffs2.o] Ошибка 1

When using NAND controllers from various chip vendors (in our case a freescale 
processor) the free OOB bytes may differ from the standard linux ones.

If the layout could be made configurable (optionally with a contrib directory 
carrying various layouts in the sources) this would help in using mkyaffs2 for 
various other platforms.

What steps will reproduce the problem?

1. "mkyaffs2 " with -e [Rootfs Dir name] [Image Name]

  Note : 2k pages and spare size is 64 , this is the default for the 
  mkyaffs2  tool.

2. Flash them in NAND partion [/dev/mtdblockN] using nand erase 
   and nand write through uboot

3.try to boot the kernel by seeting the  U-Boot environment varibale 

bootargs = .. rootfstype=yaffs2 .. root=/dev/mtdblockN ...


What is the expected output? What do you see instead?

Succesful booting and getting "login" prompt is expected after mounting. But 
even mounting is also not done successfully.

What version of the product are you using? On what operating system?

Latest tool.

OS : Ubuntu SMP x86_64 Linux

Please provide any additional information below.
I am using 128 MB NAND, 2k pages and spare size is 64

1. Generelly we used to pass "dev" information file for the device node 
information with -dev kind of option. Is the reason to get the error

2. I tried to modify the nand_ecclayout also "nand_oob_64" according to my NAND 
ecc layout 
.oobfree        = {{.offset = 4, .length = 60}},

but  the image  is broken.

Attcahed the Mount error and addtional erros logs

What steps will reproduce the problem?
1.I tried all commands available in unyaffs no luck at all
2.The system.img is a stock firmware file from a Android media player with 
Android ICS 4.03
3.The header in the system.img says 
YFSS!V10��  ��#� �  �   ƒ 
yaffs2 file system: 4K 24bits/1K 
mkyaffs2image version: Linux-2.6.35 Nand Controller V504, build time: Tue Jan  
8 16:34:36 2013
.                                                                               


What is the expected output? What do you see instead?


What version of the product are you using? On what operating system?
Windows 7 and Ubuntu no luck on both

Please provide any additional information below.

D:\blaat>unyaffs2 system.img test
unyaffs2 0.2.9_20120815: image extracting tool for YAFFS2.
warning: image size (279506352)is NOT a multiple of (0 + 2048).

scanning image 'system.img'... [done]
scanning complete, total objects: 53

building fs tree ... [done]
building complete, total objects: 1

extracting image into 'test'
[======================================================================] 1/1 100
%

modify files attributes... [done]

operation complete,
files were extracted into 'test'.

D:\blaat> I really have no idea how i can determine the chunck and page size

Please help

The link to the firmware 
https://mega.co.nz/#!KMkHVTCR!XSLG0gNa-z20jDqYNQbNMy89qQZC1WIQpy5SmUGAcbk

Thank you for any help ;)

I'm building my own system.img file for Android Emulator. I am using 
yaffs2utils on Windows 7 and I can unyaffs system.img files but when I do my 
modifications to the files and try to mkyaffs the directory containing all the 
files it fails. My Emulator is not able to start with the system.img created.

Commands tried:

mkyaffs2.exe -p 2048 -s 64 [path_of_dir] [location_to_place_system.img]

mkyaffs2.exe --yaffs-ecclayout [path_of_dir] [location_to_place_system.img]

mkyaffs2.exe -p 2048 -s 64 --yaffs-ecclayout [path_of_dir] 
[location_to_place_system.img]

Questions:
Am I missing something on my command lines?

Hi,

I tried the latest yaffs2-utils from this project, seems like
they didn't provide any patches for Linux-3.5.

Does the patches available on upstream, if yes could you point me the 
link, if not please let me know the steps for doing these utils on Linux-3.5.

--
Thanks, Jagan.

What steps will reproduce the problem?
1. compile and run

What is the expected output? What do you see instead?
Expected: unyaffs2 extracts yaffs image
Result: First file OK, but filename for second file is "<first file>/<second 
file>", third file is named "<first file>/<second file>/<third file>" and so 
on. Of course the creation of these files fails.

What version of the product are you using? On what operating system?
unyaffs2 0.2.9, Mac OS X 10.7.5 (Lion)

Please provide any additional information below.

I analysed the unyaffs2 source code and found several bugs. After applying the 
attached patch file, the program works.

- Compile warning at all %lu printf formattings, because type off_t is long 
long on MAC OS X
   I found no really nice solution. I just casted all off_t arguments to long and %lu to %ld (off_t is signed at least on Linux and Mac OS X)

- In function unyaffs2_extract_hardlink the variable variant should'nt be a 
pointer
   The variable variant saves the contents of obj->variant. A pointer has no enough space for that, it must be the real structure/union.

- The program depends on a side effect of dirname, which modifies the argument 
of dirname.
   Mac OS X (and probably other BSD variants) don't have this side effect, they return a pointer to a static area. That is the main reason for the extraction issue on Mac OS X.
   To restore the variable unyaffs2_curfile, I save the string length into cur file_len and restore it on the end of the function.
   I didn't change mkyaffs2.

- Variable type not set in function unyaffs2_extract_objtree
   The variable type is only set to a reasonable value on a root object. I assume "type" should hold the information in obj->type, except on a root object, than it is a directory. So I changed the initialization of type and use type instead of obj->type in the rest of the function.

When running unyaffs2 (with file filter, although this is optional) on an 
embedded/slow system the command takes a long time to complete.

A quick profile run shows that the many bsearch and qsort calls take up a lot 
of CPU. Apart from the fact that bsearch and qsort don't allow inlining the 
comparison function (like C++ equivalents allow) the size of the structures, 
with char[NAME_MAX+1] taking up most of it, probably cause a lot of useless 
copying around.

Would "unyaffs2" extract a runtime image dumped from NAND device directly?

As I know at this moment,
T-node tree and YAFFS2 initial have to be implemented for this enhancement.

take the attached patch as a proposal for improving your project.

When using mmap(), the target file is opened using PROT_WRITE only.
On MIPS this crashes during the memcmp(), which reads back the data for 
verification.
The protection bits should be set to PROT_READ | PROT_WRITE instead.

I am getting following error when using mkyaffs2 command:

stage 1: scanning directory 'mtddir'... [done]
scanning complete, total objects: 5.

stage 2: creating image 'mtd'
[===========================                                         ] 2/5  40%
object 0: [????] 'mtddir/.DS_Store/mtd0' (FAILED).


What version of the product are you using? On what operating system?
 Using Mac OS 10.8 and using version latest.tar.gz  from downloads.

Please guide. 

Thanks

First, whether you do these defines, or not

 #DEFINE __APPLE__ 
 #DEFINE __MACH__

inside of configs.h

and then do a "make", you still get the same error message.

What is the expected output? What do you see instead?

In file included from yaffs2/yaffs_packedtags1.h:21,
                 from yaffs2/yaffs_packedtags1.c:16:
yaffs2/yaffs_guts.h:135: error: expected specifier-qualifier-list before 
‘loff_t’
make: *** [yaffs2/yaffs_packedtags1.o] Error 1

What version of the product are you using? On what operating system?
Mac OS X 10.7.3

Please provide any additional information below.

$ make -k
gcc -I. -I./yaffs2 -O2 -Wall -Wshadow -Wpointer-arith -Wwrite-strings 
-Wstrict-prototypes -Wmissing-declarations -Wmissing-prototypes 
-Wredundant-decls -Wnested-externs -Winline -D_HAVE_MMAP   -c -o 
yaffs2/yaffs_hweight.o yaffs2/yaffs_hweight.c
gcc -I. -I./yaffs2 -O2 -Wall -Wshadow -Wpointer-arith -Wwrite-strings 
-Wstrict-prototypes -Wmissing-declarations -Wmissing-prototypes 
-Wredundant-decls -Wnested-externs -Winline -D_HAVE_MMAP   -c -o 
yaffs2/yaffs_ecc.o yaffs2/yaffs_ecc.c
gcc -I. -I./yaffs2 -O2 -Wall -Wshadow -Wpointer-arith -Wwrite-strings 
-Wstrict-prototypes -Wmissing-declarations -Wmissing-prototypes 
-Wredundant-decls -Wnested-externs -Winline -D_HAVE_MMAP   -c -o 
yaffs2/yaffs_packedtags1.o yaffs2/yaffs_packedtags1.c
gcc -I. -I./yaffs2 -O2 -Wall -Wshadow -Wpointer-arith -Wwrite-strings 
-Wstrict-prototypes -Wmissing-declarations -Wmissing-prototypes 
-Wredundant-decls -Wnested-externs -Winline -D_HAVE_MMAP   -c -o 
yaffs2/yaffs_packedtags2.o yaffs2/yaffs_packedtags2.c
gcc -I. -I./yaffs2 -O2 -Wall -Wshadow -Wpointer-arith -Wwrite-strings 
-Wstrict-prototypes -Wmissing-declarations -Wmissing-prototypes 
-Wredundant-decls -Wnested-externs -Winline -D_HAVE_MMAP   -c -o 
yaffs2utils_io.o yaffs2utils_io.c
gcc -I. -I./yaffs2 -O2 -Wall -Wshadow -Wpointer-arith -Wwrite-strings 
-Wstrict-prototypes -Wmissing-declarations -Wmissing-prototypes 
-Wredundant-decls -Wnested-externs -Winline -D_HAVE_MMAP   -c -o 
yaffs2utils_endian.o yaffs2utils_endian.c
gcc -I. -I./yaffs2 -O2 -Wall -Wshadow -Wpointer-arith -Wwrite-strings 
-Wstrict-prototypes -Wmissing-declarations -Wmissing-prototypes 
-Wredundant-decls -Wnested-externs -Winline -D_HAVE_MMAP   -c -o 
yaffs2utils_progress.o yaffs2utils_progress.c
gcc -I. -I./yaffs2 -O2 -Wall -Wshadow -Wpointer-arith -Wwrite-strings 
-Wstrict-prototypes -Wmissing-declarations -Wmissing-prototypes 
-Wredundant-decls -Wnested-externs -Winline -D_HAVE_MMAP   -c -o mkyaffs2.o 
mkyaffs2.c
mkyaffs2.c: In function ‘mkyaffs2_tag2spare’:
mkyaffs2.c:309:49: error: dereferencing pointer to incomplete type
mkyaffs2.c: In function ‘mkyaffs2_load_spare’:
mkyaffs2.c:814:47: error: invalid application of ‘sizeof’ to incomplete 
type ‘struct nand_ecclayout’ 
mkyaffs2.c:815:22: error: invalid application of ‘sizeof’ to incomplete 
type ‘struct nand_ecclayout’ 
mkyaffs2.c: In function ‘main’:
mkyaffs2.c:974:21: warning: assignment from incompatible pointer type
mkyaffs2.c:980:21: warning: assignment from incompatible pointer type
mkyaffs2.c:997:20: warning: assignment from incompatible pointer type
mkyaffs2.c: In function ‘mkyaffs2_process_objtree’:
mkyaffs2.c:733:11: warning: ignoring return value of ‘readlink’, declared 
with attribute warn_unused_result
make: *** [mkyaffs2.o] Error 1
gcc -I. -I./yaffs2 -O2 -Wall -Wshadow -Wpointer-arith -Wwrite-strings 
-Wstrict-prototypes -Wmissing-declarations -Wmissing-prototypes 
-Wredundant-decls -Wnested-externs -Winline -D_HAVE_MMAP   -c -o unyaffs2.o 
unyaffs2.c
unyaffs2.c: In function ‘unyaffs2_spare2tag’:
unyaffs2.c:460:49: error: dereferencing pointer to incomplete type
unyaffs2.c: In function ‘unyaffs2_load_spare’:
unyaffs2.c:1288:47: error: invalid application of ‘sizeof’ to incomplete 
type ‘struct nand_ecclayout’ 
unyaffs2.c:1289:22: error: invalid application of ‘sizeof’ to incomplete 
type ‘struct nand_ecclayout’ 
unyaffs2.c: In function ‘main’:
unyaffs2.c:1534:21: warning: assignment from incompatible pointer type
unyaffs2.c:1538:21: warning: assignment from incompatible pointer type
unyaffs2.c:1554:20: warning: assignment from incompatible pointer type
unyaffs2.c: In function ‘unyaffs2_extract_hardlink’:
unyaffs2.c:1010:8: warning: ignoring return value of ‘link’, declared with 
attribute warn_unused_result
unyaffs2.c:1050:12: warning: ignoring return value of ‘symlink’, declared 
with attribute warn_unused_result
unyaffs2.c: In function ‘unyaffs2_obj_chattr’:
unyaffs2.c:587:8: warning: ignoring return value of ‘lchown’, declared with 
attribute warn_unused_result
make: *** [unyaffs2.o] Error 1
gcc -I. -I./yaffs2 -O2 -Wall -Wshadow -Wpointer-arith -Wwrite-strings 
-Wstrict-prototypes -Wmissing-declarations -Wmissing-prototypes 
-Wredundant-decls -Wnested-externs -Winline -D_HAVE_MMAP   -c -o unspare2.o 
unspare2.c
gcc -lm -o unspare2 yaffs2/yaffs_hweight.o yaffs2/yaffs_ecc.o 
yaffs2/yaffs_packedtags1.o yaffs2/yaffs_packedtags2.o yaffs2utils_io.o 
yaffs2utils_endian.o yaffs2utils_progress.o unspare2.o
make: Target `all' not remade because of errors.

The cause is struct nand_oobfree not being defined.  There is a conflict 
between the files included in this package and the developer files that come 
with Ubuntu - specifically the file /usr/include/mtd/mtd-abi.h - which is part 
of the linux-libc-dev package.

What steps will reproduce the problem?
1. make 

What is the expected output? What do you see instead?
mkyaffs2.c:111: error: ‘YAFFS_UNUSED_OBJECT_ID’ undeclared (first use in 
this function)

What version of the product are you using? On what operating system?
Ubuntu 10.04 - 2.6.32-23-generic

Please provide any additional information below.

What steps will reproduce the problem?

I compile yaffs2utils by running "make", I get binaries that I can run. When I 
try to use these binaries to unpack my yaffs2 img files, they tell me that the 
unpacking is complete, but they say that there's "total objects: 1" and no 
files show up. This happens also when I run the binaries as root.

What is the expected output? What do you see instead?

I expect to see a bunch of files and some extracting going on, but instead it 
completes instantly with the following -v log:

# ./unyaffs2 -v system.yaffs2.img system/
unyaffs2-0.2.8: image extracting tool for YAFFS2
scanning image 'system.yaffs2.img'... [done]
scanning complete, total objects: 1
building fs tree ... [done]
building complete, total valid objects: 1
extracting image into 'system/'

modify files attributes... [done]

operation complete
files are extracted into 'system/'

What version of the product are you using? On what operating system?

Latest snapshot, Apr 24, 2012, Ubuntu 12.04

Please provide any additional information below.

I haven't had the possibility to try other img files with yaffs2 than these 
ones (they are android backups made by clockworkmod recovery), but all three of 
them (cache, system and data) give the same error.

Any additional info on how to access these files (I can't seem to mount them 
either) would also be highly appreciated.

What steps will reproduce the problem?
1. # mkdir test
2. # cd test
3. # touch empty_file
4. # cd ..
5. # yaffs2utils-latest/src/mkyaffs2 test test_new.yaffs2 

What is the expected output? What do you see instead?

  mkyaffs2-0.1.3: image building tool for YAFFS2
  Processing directory test into image file test_new.yaffs2
  object 257, test/empty_file is a file
  error while parsing test/empty_file
  operation incomplete!


What version of the product are you using? On what operating system?
- mkyaffs2-0.1.3
- opensuse 11.2 
Linux linux-pdkc 2.6.31.12-0.2-desktop #1 SMP PREEMPT 2010-03-16 21:25:39 +0100 
i686 i686 i386 GNU/Linux

Please provide any additional information below.

After I've written one byte into the file empty_file,
the application works:

# yaffs2utils-latest/src/mkyaffs2 test test_new.yaffs2 
 mkyaffs2-0.1.3: image building tool for YAFFS2
 Processing directory test into image file test_new.yaffs2
 object 257, test/empty_file is a file
 operation complete.
 2 objects in 1 directories
 3 NAND pages

Best regards,
Markus

It would be nice to be able to extract a given file or fileset from a YAFFS2 
image. (e.g. a file carrying an image version number)

What steps will reproduce the problem?
1. Simply extract any yaffs2 image contains setuid/setgid files with unyaffs2.

What is the expected output? What do you see instead?
I'm testing it with android system.img from SDK.
Without the fix:
==============================
$ sudo ~/svn/yaffs2utils-read-only/src/unyaffs2 -v -p 2048 system.img.orig 
system.orig
$ ls -l system.orig/bin/ping
-rwxr-xr-x 1 root 3004 26768 Oct 13 05:09 system.orig/bin/ping*
==============================
After the fix:
==============================
$ sudo ~/svn/yaffs2utils-read-only/src/unyaffs2 -v -p 2048 system.img.orig 
system.orig
$ ls -l system.orig/bin/ping
-rwxr-sr-x 1 root 3004 26768 Oct 13 05:09 system.orig/bin/ping*
==============================

Following patch fixes this issue.
Index: unyaffs2.c
===================================================================
--- unyaffs2.c  (revision 26)
+++ unyaffs2.c  (working copy)
@@ -581,11 +581,11 @@
        utime(fpath, &ftime);
 #endif

-       /* mode */
-       chmod(fpath, obj->mode);
-
        /* owner */
        lchown(fpath, obj->uid, obj->gid);
+
+       /* mode */
+       chmod(fpath, obj->mode & 07777);
 }

 static int

I created a filesystem with one file 'hello' with content 'hello'

I can't extract with unyaffs2. Ofcourse i changed the oob layout in 
yaffs2utils_ecc.h:

static struct nand_ecclayout nand_oob_64 = {
        .eccbytes = 0,
        .oobfree = {{.offset = 30, .length = 16}},
};

Unyaffs2 create a hello file but the content of those is bad.

Could you debug it, please?

What steps will reproduce the problem?
1. run unspare2 /dev/mtdX /tmp/test1
2. run unspare2 -e /dev/mtdX /tmp/test2
3. diff /tmp/test1 /tmp/test2

What is the expected output? What do you see instead?
Files should differ

What version of the product are you using? On what operating system?
SVN version r23

After parsing the '-e' parameter the following is done:
unspare2_flags &= UNSPARE2_FLAGS_ENDIAN
instead of
unspare2_flags |= UNSPARE2_FLAGS_ENDIAN

The latest trunk version contains the following line in unyaffs2
===
static int
unyaffs2_extract_hardlink (const char *fpath, struct unyaffs2_obj *obj)
{
[...]
union unyaffs2_file_variant *variant;
[...]
memcpy(&variant, &obj->variant, sizeof(obj->variant));
===

Since variant is an uninitialised pointer to a structure, memcpy() will try to 
write to a random location.  A quick fix would be to define variant as a static 
variable (i.e. remove '*'), or you need to initialise the variable before using 
it :)

I've created some example images (see attached) using the "official" yaffs2 
tool to create images (mkyaffs2image) as well as using the android fork. Code 
is from

git clone ssh://www.aleph1.co.uk/home/aleph1/git/yaffs2
git clone https://android.googlesource.com/platform/external/yaffs2

Creating images using e.g.

$ mkyaffs2image /tmp/data/ /tmp/official_mkyaffs2.img

However, none of these images can be extracted using unyaffs2:

$ unyaffs2 official_mkyaffs2.img .
unyaffs2 0.2.9: image extracting tool for YAFFS2.
warning: non-root users.

scanning image '../official_mkyaffs2.img'... [done]
scanning complete, total objects: 1

building fs tree ... [done]
building complete, total objects: 1

extracting image into '.'
[======================================================================] 1/1 
100%

modify files attributes... [done]

operation complete,
files were extracted into '.'.


However, it has extracted nothing.

What steps will reproduce the problem?
1. Download the latest tarball
2. Extract it, enter the directory
3. Run 'make'

What is the expected output? What do you see instead?

I would expect to see it compile, instead it says:
cc: yaffs_ecc.c: Bestand of map bestaat niet
and I notice that it's create a symlink:
lrwxrwxrwx 1 robin robin    12 2010-08-08 18:09 yaffs_ecc.c -> /yaffs_ecc.c

What version of the product are you using? On what operating system?

The -lastest tarball on Ubuntu 10.04

If there are no other concerns, I will discontinue the YAFFS1 image supports 
and remove the related codes in the future release.

Any suggestion?

What steps will reproduce the problem?

I use newest svn version of unyaffs on a android image,but failed:
$ sudo unyaffs2 system.img android
unyaffs2-0.2.4: image extracting tool for YAFFS2
image size (318665764) is NOT a mutiple of 2048 + 64

operation incomplete
files contents may be broken

What is the expected output? What do you see instead?


What version of the product are you using? On what operating system?


Please provide any additional information below.

Enhancement: Progress bar displayed while making/extracting a image.

This enhancement will add a progress bar while making/extracting a image.

A bar will be used to displayed by default to show the current status of
image creation/extraction. An new option "--verbose(-v)" will also be add
to verbose the details information instead of the progress bar.

In order to implement this feature, the 2-pass processing will be applied.
In first stage, the directory/image should to scanned to determine the
numbers of objects(files), and the basic information of these files. 
Then parsing/extracting the file in the second stage according to the 
database from the previous stage.

I hope I have time to finish it.

When I do a build of the Android 2.3.7b1 with make sdk I get a system.img for 
the emulator.

yaffs2utils command unyaffs sees 0 objects in it.

The unyaffs project, however, will correctly extract all the files and folders. 
 http://code.google.com/p/unyaffs/

yaffs2utils should also support the given system.img.

What steps will reproduce the problem?

$ make
gcc -I. -I./yaffs2 -O2 -Wall -Wshadow -Winline -Wpointer-arith -Wnested-externs 
-Wwrite-strings -Wstrict-prototypes -Wmissing-declarations -Wmissing-prototypes 
-Wredundant-decls  -D_HAVE_MMAP   -c -o mkyaffs2.o mkyaffs2.c
mkyaffs2.c:19:21: error: configs.h: 没有那个文件或目录
mkyaffs2.c: In function ‘mkyaffs2_process_objtree’:
mkyaffs2.c:740: warning: ignoring return value of ‘readlink’, declared with 
attribute warn_unused_result
make: *** [mkyaffs2.o] 错误 1


What is the expected output? What do you see instead?


What version of the product are you using? On what operating system?
svn rev27.I build it on ubuntu 10.04

Please provide any additional information below.

What steps will reproduce the problem?

a.manjunatha@a-manjunatha~$# cd ~/myrootfs
a.manjunatha@a-manjunatha~$# ls 
bin  boot  csa  dev  etc  home  lib  lost+found  manju  media  mnt  opt  proc  
root  run  sbin  sdcard  smack  srv  sys  tmp  usr  var
a.manjunatha@a-manjunatha~$# cd ..
bash# sudo attr -s "Key001" -V "9999" ./myrootfs/usr/bin/file 
Attribute "Key001" set to a 4 byte value for ./myrootfs/usr/bin/file:
9999
a.manjunatha@a-manjunatha~$# getfattr -d ./myrootfs/usr/bin/file 
# file: myrootfs/usr/bin/file
user.Key001="9999"

a.manjunatha@a-manjunatha:~$# sudo mkyaffs2 --yaffs-ecclayout ./myrootfs/ 
yaffsrootfs.img 
mkyaffs2 0.2.9_20120815: image building tool for YAFFS2.

stage 1: scanning directory './myrootfs/'... [done]
scanning complete, total objects: 7918.

stage 2: creating image 'yaffsrootfs.img'
[===============================================================================
================================================================================
================================================================================
========] 7918/7918 100%

operation complete,
7918 objects in 90015 NAND pages.

a.manjunatha@a-manjunatha:~/$# sudo unyaffs2 --yaffs-ecclayout 
./yaffsrootfs.img  extracted_folder
unyaffs2 0.2.9_20120815: image extracting tool for YAFFS2.

scanning image './yaffsrootfs.img'... [done]
scanning complete, total objects: 7918

building fs tree ... [done]
building complete, total objects: 7918

extracting image into 'extracted_folder'
[===============================================================================
================================================================================
================================================================================
========] 7918/7918 100%

modify files attributes... [done]

operation complete,
files were extracted into 'extracted_folder'.
a.manjunatha@a-manjunatha:~/$# getfattr -d ./extracted_folder/usr/bin/file
<Extended attribute is missing after mkyaff2 and unyaff2>

Is there is Way to Fix this issue?
http://www.yaffs.net/documents/faq
in the above link it is mentioned it supports the extended attributes.


What is the expected output? What do you see instead?
It should show extended attributes after mkyaffs2 and unyaff2

What version of the product are you using? On what operating system?
mkyaffs2 0.2.9_20120815 - A utility to make the yaffs2 image.

Please provide any additional information below.
Is there is Way to Fix this issue?
http://www.yaffs.net/documents/faq
in the above link it is mentioned it supports the extended attributes.

I traced the issue back to closedir() being called with the 'path' variable
rather than the 'dir' variable, as returned from opendir().

Applying the following patch fixes the issue:
Index: mkyaffs2.c
===================================================================
--- mkyaffs2.c  (revision 10)
+++ mkyaffs2.c  (working copy)
@@ -584,12 +584,12 @@
 error:
                if (retval) {
                        fprintf(stderr, "error while parsing %s\n", fpath);
-                       closedir(path);
+                       closedir(dir);
                        return -1;
                }
        }

-       closedir(path);
+       closedir(dir);

        return 0;
 }

What steps will reproduce the problem?
1. When I try extracting a embedded image which contain busybox by unyaffs2.,
2. The attribute of the busybox is NOT as same as the original one.

Please use labels and text to provide additional information.

The chmod() should NOT apply on a symbol link.
It will modify the attribute of the target. 

What steps will reproduce the problem?
1. call unyaffs2 source.img destination.folder

What is the expected output? What do you see instead?
I expect to find output in the folder, but its empty. The process runs without 
complain.

What version of the product are you using? On what operating system?
trunk

Please provide any additional information below.
tried it with sudo as well. and with full path without substitution as ~ or .

+++
x@xyz:~$ ~/dev/build/yaffs2utils/unyaffs2 
/media/x/6631-6634/clockworkmod/backup/2012-12-18.19.21.33/system.yaffs2.img .
unyaffs2-0.2.9: image extracting tool for YAFFS2.
warning: non-root users.

scanning image 
'/media/x/6631-6634/clockworkmod/backup/2012-12-18.19.21.33/system.yaffs2.img'..
. [done]
scanning complete, total objects: 1

building fs tree ... [done]
building complete, total objects: 1

extracting image into '.'
[======================================================================] 1/1 
100%

modify files attributes... [done]
operation complete,
files were extracted into '.'.
++++

Hello,

Why can't i decrease the length to 16 in oobfree?

If i know well we need only 16 bytes area to store four unsigned number in oob 
as struct yaffs_packed_tags2_tags_only.

static struct nand_ecclayout nand_oob_64 = {
        .eccbytes = 0,
        .oobfree = {{.offset = 30, .length = 16}},
};

I used msm nand chip with this layout:

static struct nand_ecclayout msm_nand_oob_64 = {
        .eccbytes       = 40,
        .eccpos         = {
                0,  1,  2,  3,  4,  5,  6,  7,  8,  9,
                10, 11, 12, 13, 14, 15, 16, 17, 18, 19,
                20, 21, 22, 23, 24, 25, 26, 27, 28, 29,
                46, 47, 48, 49, 50, 51, 52, 53, 54, 55,
                },
        .oobavail       = 16,
        .oobfree        = {
                {30, 16},
        }
};

Thank you.

What steps will reproduce the problem?
1. I downloaded the binary from 
https://dl.google.com/dl/android/aosp/tungsten-ian67k-factory-468d9865.tgz
2. Extracted the update.zip and unzipped it to get a system.img
3. Used the following commandline 2 commandlines to try and get the contents of 
the binary.

./unyaffs2 --verbose --yaffs-ecclayout ../system.img test-yaffs2
./unyaffs2 --verbose -e --yaffs-ecclayout ../system.img test-yaffs2 


What is the expected output? What do you see instead?
both fail. The second one does not give me any errors but no contents in the 
destination directory.


What version of the product are you using? On what operating system?
2.9 with the patch that was there in issue 28.

Please provide any additional information below.
It may be just that these binaries have somekind of header. I think this would 
be the case since I am able to use the binary that I built using the AOSP setup.

Please feel free to get more info if needed.

BR
Kozak

I observed a couple of spurious errors when running the unyaffs2 tools on 
target:

1) spurious crashes that disappeared when running under strace/gdb.
Using a coredump and valgrind I traced these back to the use  of 
list_for_each_entry() where list_for_each_entry_safe is required (see 
unyaffs2_objtable_exit and unyaffs2_specfile_exit() )

2) when running from a script using 2 '-f <file>' options the program would 
indicate that it did not find the 2 files, while running the exact same command 
from the shell ran fine.
It wasn't untill I checked compiler warnings that I found that a variable is 
not being initialized before use: see "off_t offset" in unyaffs2_scan_img() in 
case MMAP is used.

After I update my Debian system to Squeeze, I will migrate the svn repository 
to git.

What steps will reproduce the problem?
Try to run the makefile under Ubuntu 11.10 with gcc 4.6.1. Error messages 
appear due to improperly linked math functions.

The ordering of linker flags prevents compiling on my system. Putting 
$(LDFLAGS) last on the cc lines fixes the error for me.

What steps will reproduce the problem?
1. Download the latest yaffs2utils
2. make it
3. error

What is the expected output? What do you see instead?
It cannot be compiled with the latest yaffs2 source.

Please use labels and text to provide additional information.
Because of modification of the latest yaffs2 source, the yaffs2utils
cannot be compiled with the latest version of yaffs2.

I have downloaded the image directly from device, just to examine the contents.
The problem is that the image does not contain extra +64 bytes at the end of 
every chunk.

Is it possible to extract contents while skipping OOB area? Or maybe generate 
OOB from the downloaded image?

I try to extract system.img ，this file is created by mkyaffs2image when I 
build android source. But it doesn't work, nothing was extracted, did i miss 
something?