Comments (4)
To Meta team: could you please share what is the rationale behind fetching the latest dependencies on every build? Did you consider setting some specific version of each dependency in build-package.sh (e.g. to the latest release) and just updating them when necessary?
from cachelib.
Hello @igchor ,
We can divide the dependencies into two groups: everything before folly
(i.e. packages which do not depend on folly: zstd
, googlelog
, googletest
, googleflags
, fmt
, sparsemap
), then folly
and all packages that do depend on it (folly
, fizz
, wangle
, fbthrift
) .
folly
itself does not have a "release" or "stable" version. It is always being updated.
folly
is added as git submodule
to cachelib
(like in many other projects which use folly). The difference is: other projects can decide to pin the folly git-submodule to a specific commit, and leave it as-is for a long time (effectively making it "stable").
But cachelib
is a Facebook/Meta project, and the internal code base always tracks the latest folly
build (observe all the commits at https://github.com/facebook/CacheLib/commits/main which just say "updating submodules" - those are automatic).
And so, when you update cachelib (with git pull
) you also update the folly submodule commit version. Then you need to rebuild folly. and then you need to rebuild the downstream packages (fizz/wangle/fbthrift) which depend on folly.
For the other packages, you'll notice we do use a specific git tag or branch ( e.g https://github.com/facebook/CacheLib/blob/main/contrib/build-package.sh#L120 ). So those, even though we do a "git pull", it should remain in the same commit, and a make
should be very fast.
I notice fmt
is an exception - not pinned to a specific git tag or revision - likely an omission that can be fixed.
Hope this answered your question.
regards,
-assaf
from cachelib.
For fmt, the build error is being caused by, this commit: fmtlib/fmt@3a951a6
I think there needs to be either changes made in Cachelib to fix this, or make it point to one of the previous version of fmt.
from cachelib.
Not sure if this was already cleared up (I'm assuming not since I ran into the same issue when reinstalling Cachelib today), but I managed to get around the folly
build issue by doing ./contrib/build-package.sh -d
. Definitely doesn't address the issue within the build script though...
Still in the midst of installation, so there may or may not be further issues down the line that I haven't encountered yet doing it this way.
from cachelib.
Related Issues (20)
- Questions about trace files when running cachebench HOT 2
- Running simple-cache-example gives an error, flag 'v' was defined more than once HOT 6
- OSS build broken as of May, 2024 -> PRs are all blocked HOT 8
- No build support for Fedora37 OS HOT 4
- failed to build CacheLib following document HOT 2
- Build fails on debian-10 HOT 2
- Segmentation fault while fetching refcount HOT 2
- Minimum Limit For Cache Allocation? HOT 1
- build failed when building dependency 'fbthrift' HOT 3
- Build issue with CacheLib with missing source files HOT 3
- [Seeking Volunteers] Add new builds to CacheLib HOT 2
- Build is failing with error: ‘fmt::v10::detail::type_is_unformattable_for<const facebook::cachelib::navy::Status, char> _’ has incomplete type 1600 | type_is_unformattable_for<T, typename Context::char_type> _; | ^ HOT 4
- NavySetup should not involve the MockDevice HOT 2
- How to configure folly for logging CacheLib
- Build failed with error message "cachelib/navy/common/FdpNvme.h:143:28: error: expected ‘)’ before ‘&’ token 143 | explicit FdpNvme(NvmeData& data, struct nvme_fdp_ruh_status* ruh_status); | ~ ^ | ) make[2]: *** [navy/CMakeFiles/cachelib_navy.dir/build.make:328: navy/CMakeFiles/cachelib_navy.dir/common/Device.cpp.o] Error 1 make[1]: *** [CMakeFiles/Makefile2:395: navy/CMakeFiles/cachelib_navy.dir/all] Error 2 make: *** [Makefile:130: all] Error 2 build-package.sh: error: make failed" HOT 4
- Are there instructions on how to build and install in Amazon Linux? HOT 1
- Is nvmLookupLatency just calculates job scheduling time? HOT 1
- Are admission policies applied to objects (value changes in DRAM) even if they are read from the NVM cache? HOT 3
- Some of shared libraries are not linked to unit test binaries
- cachebench can not execute
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from cachelib.