Coda mount has 4 times the specified amount of 1K-blocks about coda

[jaharkes]

I think a cache block is 1k instead of 4. Jan

…

On July 14, 2018 11:53:12 AM EDT, Pedro Cuadra ***@***.***> wrote: When running `df` the ouput shows 4x`cacheblocks`. I tested it with the value specified in `/etc/coda/venus.conf` and with `venus -c <n>`. In both cases the same behaviour is seen. I'm running kernel version `4.15.0-24-generic` on `Ubuntu 18.04`. Seen in `master` branch. I'm investigating if it's also present in previous commits/versions.

[pjcuadra]

Yes. What I mean is that if I run venus -c 100000 I'll get a size of 400000 in the coda mount. Bellow the output;

pjcuadra@pjcuadra-vaio:/$ sudo venus -init -c 100000

Date: Sat 07/14/2018

13:58:22 Coda Venus, version 6.14.0
13:58:22 /var/lib/coda/LOG size is 19780620 bytes
13:58:22 /var/lib/coda/DATA size is 79122480 bytes
13:58:22 /var/lib/coda/DATA size is 79122480 bytes
13:58:22 Initializing RVM data...
13:58:22 ...done
13:58:22 Loading RVM data
13:58:22 Starting RealmDB scan
13:58:22 	Found 1 realms
13:58:22 starting VDB scan
13:58:22 	0 volume replicas
13:58:22 	0 replicated volumes
13:58:22 	0 CML entries allocated
13:58:22 	0 CML entries on free-list
13:58:22 starting FSDB scan (4166, 100000) (25, 75, 4)
13:58:22 	0 cache files in table (0 blocks)
13:58:22 	4166 cache files on free-list
13:58:23 starting HDB scan
13:58:23 	0 hdb entries in table
13:58:23 	0 hdb entries on free-list
13:58:23 Kernel version ioctl failed.
13:58:23 Mounting root volume...
13:58:23 Venus starting...
13:58:23 /coda now mounted.

pjcuadra@pjcuadra-vaio:/$ df
Filesystem     1K-blocks     Used Available Use% Mounted on
udev             3989196        0   3989196   0% /dev
...
coda              400000        0    375000   0% /coda

What happens is that; in my file system, the block size is in fact 4096. Output below;

pjcuadra@pjcuadra-vaio:/coda$ stat -f .
  File: "."
    ID: 0        Namelen: 255     Type: coda
Block size: 4096       Fundamental block size: 4096
Blocks: Total: 100000     Free: 100000     Available: 93750
Inodes: Total: 4166       Free: 4163

Aparently, the size of coda's root volume is calculated assuming 1K block size.

[jaharkes]

That actually changed over time. I think statvfs reports filesystem block size and number of blocks. My guess is that we're reporting the right number of blocks, but the block size of the file system which holds the cached files, because that value is also a hint for applications about optimal io and alignment size. So we may have to do a division or multiplication in the data we return to statvfs to compensate. Jan

…

On July 14, 2018 2:01:59 PM EDT, Pedro Cuadra ***@***.***> wrote: Yes. What I mean is that if I run `venus -c 100000` I'll get a size of 400000 in the coda mount. Bellow the output; ``` ***@***.***:/$ sudo venus -init -c 100000 Date: Sat 07/14/2018 13:58:22 Coda Venus, version 6.14.0 13:58:22 /var/lib/coda/LOG size is 19780620 bytes 13:58:22 /var/lib/coda/DATA size is 79122480 bytes 13:58:22 /var/lib/coda/DATA size is 79122480 bytes 13:58:22 Initializing RVM data... 13:58:22 ...done 13:58:22 Loading RVM data 13:58:22 Starting RealmDB scan 13:58:22 Found 1 realms 13:58:22 starting VDB scan 13:58:22 0 volume replicas 13:58:22 0 replicated volumes 13:58:22 0 CML entries allocated 13:58:22 0 CML entries on free-list 13:58:22 starting FSDB scan (4166, 100000) (25, 75, 4) 13:58:22 0 cache files in table (0 blocks) 13:58:22 4166 cache files on free-list 13:58:23 starting HDB scan 13:58:23 0 hdb entries in table 13:58:23 0 hdb entries on free-list 13:58:23 Kernel version ioctl failed. 13:58:23 Mounting root volume... 13:58:23 Venus starting... 13:58:23 /coda now mounted. ***@***.***:/$ df Filesystem 1K-blocks Used Available Use% Mounted on udev 3989196 0 3989196 0% /dev ... coda 400000 0 375000 0% /coda ``` What happens is that; in my file system, the block size is in fact 4096. Output below; ``` ***@***.***:/coda$ stat -f . File: "." ID: 0 Namelen: 255 Type: coda Block size: 4096 Fundamental block size: 4096 Blocks: Total: 100000 Free: 100000 Available: 93750 Inodes: Total: 4166 Free: 4163 ``` Aparently, the size of the mount is calculated assuming 1K block size.

[pjcuadra]

You mean something like editing vproc_vfscalls.cc in this way?

#include <sys/statvfs.h>
...
#define CODA_BLOCK_SIZE 1024
...
void vproc::statfs(struct coda_statfs *sfs) {
    struct statvfs stat;

    LOG(1, ("vproc::statfs\n"));

    sfs->f_blocks  = CacheBlocks;
    sfs->f_bfree   = (CacheBlocks - FSDB->DirtyBlockCount());
    sfs->f_bavail  = FSDB->FreeBlockCount() - FSDB->FreeBlockMargin;
    sfs->f_files   = CacheFiles;
    sfs->f_ffree   = FSDB->FreeFsoCount();

    /* Get stats from the FS where the cache is stored */
    statvfs(CacheDir, &stat);

    /* Compensate block size */
    sfs->f_blocks  *= CODA_BLOCK_SIZE;
    sfs->f_blocks  /= stat.f_bsize;
    sfs->f_bfree   *= CODA_BLOCK_SIZE;
    sfs->f_bfree   /= stat.f_bsize;
    sfs->f_bavail  *= CODA_BLOCK_SIZE;
    sfs->f_bavail  /= stat.f_bsize;
}

With this change I have the correct output as follows;

pjcuadra@pjcuadra-vaio:/coda$ df
Filesystem     1K-blocks     Used Available Use% Mounted on
udev             3989196        0   3989196   0% /dev
...
coda              100000        0     93748   0% /coda
pjcuadra@pjcuadra-vaio:/coda$ stat -f .
  File: "."
    ID: 0        Namelen: 255     Type: coda
Block size: 4096       Fundamental block size: 4096
Blocks: Total: 25000      Free: 25000      Available: 23437
Inodes: Total: 4166       Free: 4163

Shouldn't we maybe make coda kernel "aware" of block size?

[pjcuadra]

You can have a look at the change here pjcuadra/coda/tree/fix/block_size_missmatch

[jaharkes]

I just checked and the statfs->f_bsize value is not taken from the underlying filesystem block size, it is hardcoded as 4096 in the Linux kernel module. So doing a simple sfs->f_blocks /= 4; probably is good enough.

I just went to check what the FBSD kernel module is doing and discovered that they dropped the Coda kernel module with release 10, either because of how it handled locking or GPL contamination. Either way we don't have to worry about how it used to handle file system block size.