c-fs / cfs Goto Github PK

use o-direct for write and create user level cache

we want to have better control of buffer caching. for example, we can get hit rate and control the max memory we can use (each client can use).

Changes may need:
https://github.com/google/cadvisor/blob/master/docs/running.md#centos-fedora-and-rhel

We need to test them to ensure that they work.

currently disk.Write will save the data to the current dir. we need to make a disk struct that represent a root path.

use bolt? https://github.com/boltdb/bolt

ReadDir should return . as current directory and .. as parent directory

/cc @yichengq

try cfsctl write --name="/cfs0/foodir/foo" --data="bar"

通过 writeBlock(ws io.WriteSeeker, index, bs int64, p []byte)，更新一个block。
当len(p)小于block size时，是什么语义？

1. 算错误，直接返回
2. 更新前len(p)字节，并重新计算整个block的crc

Currently docker container can only read default configs. We need a way to let client of the docker container pass in their own configuration.

I don't think any modern servers are using int32 anymore. I propose we change all int64 types to int so that we don't need explicit type conversions.

run go test under /erasure

# github.com/c-fs/cfs/erasure
./erasure.go:45:31: warning: unknown attribute '__gcc_struct__' ignored [-Wattributes]
        } __attribute__((__packed__, __gcc_struct__)) *a = v;
                                     ^
./erasure.go:65:31: warning: unknown attribute '__gcc_struct__' ignored [-Wattributes]
        } __attribute__((__packed__, __gcc_struct__)) *a = v;
                                     ^
./erasure.go:79:31: warning: unknown attribute '__gcc_struct__' ignored [-Wattributes]
        } __attribute__((__packed__, __gcc_struct__)) *a = v;
                                     ^
./erasure.go:100:31: warning: unknown attribute '__gcc_struct__' ignored [-Wattributes]
        } __attribute__((__packed__, __gcc_struct__)) *a = v;
                                     ^
./erasure.go:117:31: warning: unknown attribute '__gcc_struct__' ignored [-Wattributes]
        } __attribute__((__packed__, __gcc_struct__)) *a = v;
                                     ^
5 warnings generated.
# github.com/c-fs/cfs/erasure
clang: warning: argument unused during compilation: '-pthread'
# github.com/c-fs/cfs/erasure
clang: warning: argument unused during compilation: '-pthread'

@yunxing could you take a look?
And close this issue if we don't need to worry about it.

on Mac with:

Apple LLVM version 6.0 (clang-600.0.54) (based on LLVM 3.5svn)
Target: x86_64-apple-darwin14.3.0
Thread model: posix

make server and ctl under bin

use https://github.com/tsuraan/Jerasure/blob/master/src/jerasure.c#L1453.

This may include a local cluster running way, and a multiple-machine cluster running way.

https://github.com/dvyukov/go-fuzz

https://github.com/constabulary/gb

readBlock(rs io.ReadSeeker, p []byte, index, bs int64), should return err when len(p) < the block size ?

disk.WriteAt("cfs0/newfile", []byte("123"), 0)

buf := make([]byte, 3)
disk.ReadAt("cfs0/newfile", buf, 0)  return 3, nil
disk.ReadAt("cfs0/newfile", buf, 3)  return 0, nil   // actual expected 0, io.EOF?

When disk.Readat wrap to io.Reader interface, then as a read param for io.Copy, it will dead loop.

maybe influxdb

from #20

should be a very thin wrapper around grpc client