Comments (7)
The float version was mainly introduced for the bslmm model. I tried the float version a while ago on our previous cluster (pps cluster), and found that it only uses half the memory required by the double version (if I remember correctly, for 5k idv and 0.5million snps, the float version uses 8gb while the double version uses 16gb). But that was about 5 years ago. I am not sure either whether there is any benefit nowadays. It would be great if you could do a quick test on this, and we can certainly remove this option if there is no longer benefits.
from gemma.
@xiangzhou I will test to see if FORCE_FLOAT
reduces memory usage in 64-bit systems.
from gemma.
Don't use float if you have double. You have to really know what you are doing.
https://wiki.haskell.org/Performance/Floating_point
from gemma.
@xiangzhou An update on this:
I wasn't able to build the binary executable using FORCE_FLOAT = 1
; it generated several errors which did not seem straightforward to resolve.
Upon further experimentation in MATLAB, I think it is possible to reduce memory requirements with single-precision. I wasn't able to find any documentation about this in MATLAB, but this webpage briefly discusses storage of <64-bit numbers in Julia.
In short, there is a potential benefit to keeping the FORCE_FLOAT
option.
However, perhaps a larger issue is that debugging & maintaining single-precision and double-precision versions of GEMMA will involve substantially more effort, and I'm not sure if the effort is justified by the benefit (which, in my opinion, is relatively small---there are other more effective ways to reduce memory, e.g., by LD pruning).
I've created a new branch force-float that preserves the state of the current repository with the FORCE_FORCE
option, and my plan is to remove this option in the master
branch unless there are any objections.
from gemma.
This is one of the best talks I ever heard. If you want to know about float/double, spend half an hour here. It is worth it. Entertaining too.
http://dconf.org/2016/talks/clugston
from gemma.
I also had issues with FORCE_FLOAT
in Linux.
from gemma.
@tseemann Thanks for sharing.
from gemma.
Related Issues (20)
- Parsing input file '' failed in function AnalyzeBimbamGXE
- Enforce failed for number of analyzed individuals equals 0. in src/param.cpp at line 2073 in ProcessCvtPhen HOT 2
- long haplotype as population structure
- basic usage doesn't work
- Gemma run-time for Estimate Relatedness Matrix calculation
- very low number of analyzed SNPs/var
- The input files includes 26 linkage groups but the output files only includes 22 linkage group
- Categorical covariates
- Pve is .999 HOT 1
- gemma-wrapper:279:in `<main>': Expected GEMMA -a genotype file switch (RuntimeError)
- Got a nan in the p-value results HOT 1
- gsl: newton.c:88: ERROR: function value is not finite 3.08%
- GSL ERROR: function value is not finite in brent.c at line 58 errno 9
- WARNING: Brent did not converge
- Why the diagonal of relatedness matrix is not equal to 1 or a constant value? HOT 1
- The problem of filtering snp parameters
- ERROR: Enforce failed for Problem reading FAM file (phenotypes do not match geno file) in src/gemma_io.cpp at line 600 in ReadFile_fam GEMMA 0.98.3 (2020-11-28) by Xiang Zhou and team (C) 2012-2020
- Identical P-Values at end of run HOT 1
- BIMBAM format (preferred)
- ##ERROR: FAILED: Parsing input file 'mikk_small_hapVariants_updated_noheaders.csv' failed in function ReadFile_geno in src/gemma_io.cpp at line 744, for GEMMA 0.98.5 (2021-08-25) by Xiang Zhou, Pjotr Prins and team (C) 2012-2021
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from gemma.