Comments (2)
That is probably a lmfit
issue. I don't know for sure if parallelization is an option for the Trust Region Reflective method used here, but I doubt it (IIRC it's an iterative process). Same goes for a progress bar, because the algorithm doesn't know in advance how long it will be looking for a local minimum.
Remember that the difficulty of fitting the model increases dramatically with the number of parameters to fit, not so much with the number of analyses. I usually process datasets with 3-6 sessions and 20-30 unknown samples. That is quasi-instantaneous. When processing the more demanding dataset of Anderson et al. (2021), it took something like 10-20 seconds because of the large number of sessions (each batch of a few tens of replicates was treated as a new session). Is this what you're doing? It might be possible to relax the convergence conditions (see here and here, ftol
and xtol
parameters).
My personal approach is not to process a year's worth of data at once, but usually rather to process together only the sessions related to a given project.
from d47crunch.
Thanks! Yep I didn't realize the slowness came because of all the unique Sample names. After redefining sample to mean something like "period of time for which we calculate one temperature" it is now running within several seconds, depending on the dataset.
FTR:
- "sample" = an amount of presumably homogeneous carbonate material. Each sample should be uniquely identified by a sample name (field
Sample
in the csv file).- "analysis" or "replicate" = corresponds to a single acid reaction followed by purification of the evolved CO2 and by a series of dual-inlet IRMS measurements. Each analysis is identified by a unique identifier (field
UID
in the csv file, but if it's missing a default series of UIDs will be generated).
Originally posted by @mdaeron in #7 (comment)
from d47crunch.
Related Issues (15)
- decide on how to make plotting interfaces consistent HOT 5
- add a **see also** section to readme HOT 2
- clumpycrunch formatting
- issue with trying to set parameter b to 0 in standardize(constraints = ...) call HOT 7
- D47crunch name HOT 2
- missing depency: rich? HOT 1
- UTF-8 encoding of csv HOT 4
- Output temperature in Table of Samples? HOT 2
- `indep_sessions` standardization is not working HOT 1
- Update module-level table functions for Δ49
- Implementation of Δ48 standardization HOT 6
- crunch throws an error with failed d45 type HOT 11
- standardize throws an error because of package lmfit HOT 5
- Improve `plot_distribution_of_analyses` or throw warnings in case there are many sessions/samples HOT 7
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from d47crunch.