epfl-enac / ccfatigueplatform

ccfatigue ccfatigue.epfl.ch CCLAB

CCFatigue Platform

CCFatiguePlatform is an initiative from CCLab that aims to develop a web application to faciltate manipulation and harmonized storage of composite materials testing datasets.

Demo

How to use on your desktop

Secrets

mkdir -p secrets && cat <<EOF > secrets/.env
POSTGRES_USER=ccfatigue
POSTGRES_PASSWORD=change-it!
EOF

First time setup

make install

Run Backend

make dev-database
make dev-backend

Run Frontend

make dev-frontend

Dev tools

Run Preprocessing

make preprocessing

Re-generate API bindings btw backend and frontend

make api

Server provisioning

make run

App architecture

Original analysis fortran modules

The modules have been removed from the project but can be viewed here.

License

MIT

Contributors

Charlotte Weil, Samuel Bancal, David Tang, Nicolas Dubois, Pierre Guilbert, Yannick Marcon Scott M. Salmon, Anastasios Vassilopoulos, Shayan Khalooei

ccfatigueplatform's Issues

CCFatigue analysis dashboard: S-N Curve: Display parameters next to plot

fetch automatically parameters from (ideally reformatted - see issue #14 ) S-N curve outputs and display as described in design doc (slides)

Makefile - run all in one docker-compose

Instead of several steps :

make install
make run-preprocessing
make run-db
make run-backend
cd backend
make init-db  # 1st time init
cd ..
make run-frontend

Have a single :

make run

(data preprocessing): TST_Srinivasan_2021-11_QS

Hi,
Your data didn't successfully passed convention tests.
Please check the following validation output.
In order to integrate this dataset, you'll need to fix it and/or provide an update for the data convention (in agreement with all the colleagues involved).

Data convention is defined here :

• https://github.com/EPFL-ENAC/CCFatiguePlatform/blob/feat/data-validation/Data/TST_Data_Convention.md
• https://github.com/EPFL-ENAC/CCFatiguePlatform/blob/feat/data-validation/Data/TST_Data_Template.xls

(i): Parsing experiment /home/sbancal/IT4R/CCFatiguePlatform/Data/raw/TST_Srinivasan_2021-11_QS
(i): Read /home/sbancal/IT4R/CCFatiguePlatform/Data/raw/TST_Srinivasan_2021-11_QS/TST_2021-11_QS_metadata.xlsx Experiment tab
(i): Cleanup experiment
(i): Validate experiment
ERROR: missing mandatory column : Geometry>Length
ERROR: missing mandatory column : Geometry>Width
ERROR: missing mandatory column : Geometry>Thickness
ERROR: unrecognized value for column General>Control mode = 'Displacement controlled'
(i): Read /home/sbancal/IT4R/CCFatiguePlatform/Data/raw/TST_Srinivasan_2021-11_QS/TST_2021-11_QS_metadata.xlsx Tests tab
(i): Cleanup tests
Warning: drop unexpected column initial crack length
Warning: drop unexpected column Specimen name
Warning: drop unexpected column displacement loading rate
(i): Validate tests
ERROR: mandatory column 'Stress Ratio' not found
ERROR: mandatory column 'Maximum Stress' not found
ERROR: mandatory column 'Loading rate' not found
ERROR: mandatory column 'Run-out' not found
[ cut ---- 8X ---- ]
Warning: Experiment encountered errors -> cannot save any preprocessed data

Variables consistency validation

@nicdub - Now with Tassos' answer, can you update the variables dictionary with the comments below, as well as verify that all Data conventions are full and consistent with variable dictionary ?

Makefile revised

Have a Makefile at the root of the project that offers :

• make install that runs all necessary installations
• make dev-* that runs everything for dev environment
• make run that runs all stuffs in docker services

Test Dashboard: polish CSS Grid to avoid scrolling

as much as possible...

AGG Data convention

description of "stress_cluster_number" only used for Whitney, according to Tassos- but apparently used in other methods ?!

stress_cluster_number used in the following modules to get corresponding ASTM value:

• LinLog fortran line 167, python line 231
• LogLog fortran line 167, python line 247
• Sendeckyj fortran [not used], python (because we add stress upper and lower bounds calculation) line 257

And also in Whitney (but not with ASTM) fortran line 66

(TST Preprocessing)

Preprocessing pipeline

• tst_data_quality_control.py
• delete TST_data_harmonization.py if obsolete
• Validate/convert new TST datasets (pull requests)
  • TST_Khalooei_2021-10_FA
  • TST_Srinivasan_2021-10_FA
  • #45
  • #46
  • #47
  • #48
  • #49
  • #50 ➡️ impossible as is
  • #51 ➡️ impossible as is
• implement constraint checks on xls/json files
• Code review

Tests injection to DB

As Experiments are injected in the DB, inject the corresponding Tests.

CCFatigue: Re-format S-N curve output

Turn the output file into a json with 2 tables:
1- data (should only be the rows where 3 coluns are full)
2 - parameters : a table with a column for each of the 11 rows of parameters, as described in CCFatigue readme. Header should have column names. note that these are different for each method.

Would be great to do this one while we have it in mind...

(DB: Search DB screen)

• Integrate new datasets

(data preprocessing): TST_Srinivasan_2021-10_FA

Hi,
Your data didn't successfully passed convention tests.
Please check the following validation output.
In order to integrate this dataset, you'll need to fix it and/or provide an update for the data convention (in agreement with all the colleagues involved).

Data convention is defined here :

• https://github.com/EPFL-ENAC/CCFatiguePlatform/blob/feat/data-validation/Data/TST_Data_Convention.md
• https://github.com/EPFL-ENAC/CCFatiguePlatform/blob/feat/data-validation/Data/TST_Data_Template.xls

(i): Parsing experiment /home/sbancal/IT4R/CCFatiguePlatform/Data/raw/TST_Srinivasan_2021-10_FA
(i): Read /home/sbancal/IT4R/CCFatiguePlatform/Data/raw/TST_Srinivasan_2021-10_FA/TST_2021-10_FA_metadata.xlsx Experiment tab
(i): Cleanup experiment
(i): Validate experiment
ERROR: missing mandatory column : Geometry>Length
ERROR: missing mandatory column : Geometry>Width
ERROR: missing mandatory column : Geometry>Thickness
ERROR: unrecognized value for column General>Control mode = 'Load controlled'
(i): Read /home/sbancal/IT4R/CCFatiguePlatform/Data/raw/TST_Srinivasan_2021-10_FA/TST_2021-10_FA_metadata.xlsx Tests tab
(i): Cleanup tests
Warning: drop unexpected column Load controlled loading rate
Warning: drop unexpected column Thickness
Warning: drop unexpected column x corrdinate of measuring point #
Warning: drop unexpected column Displacement controlled loading rate
Warning: drop unexpected column Specimen name
Warning: drop unexpected column Length
Warning: drop unexpected column Temperature
Warning: drop unexpected column initial crack length
Warning: drop unexpected column y corrdinate of measuring point #
Warning: drop unexpected column Humidity
Warning: drop unexpected column Width
Warning: drop unexpected column Frequency
(i): Validate tests
ERROR: mandatory column 'Loading rate' not found
(i): Read measures /home/sbancal/IT4R/CCFatiguePlatform/Data/raw/TST_Srinivasan_2021-10_FA/TST_2021-10_FA_001.csv
(i): Read measures /home/sbancal/IT4R/CCFatiguePlatform/Data/raw/TST_Srinivasan_2021-10_FA/TST_2021-10_FA_002.csv
(i): Cleanup measures /home/sbancal/IT4R/CCFatiguePlatform/Data/raw/TST_Srinivasan_2021-10_FA/TST_2021-10_FA_001.csv
(i): Cleanup measures /home/sbancal/IT4R/CCFatiguePlatform/Data/raw/TST_Srinivasan_2021-10_FA/TST_2021-10_FA_002.csv
(i): Validate measures /home/sbancal/IT4R/CCFatiguePlatform/Data/raw/TST_Srinivasan_2021-10_FA/TST_2021-10_FA_001.csv
(i): optional column not found: 'MD_Displacement--\d'
(i): optional column not found: 'T--\d'
(i): optional column not found: 'Storage_modulus'
(i): optional column not found: 'Tan_delta'
(i): Validate measures /home/sbancal/IT4R/CCFatiguePlatform/Data/raw/TST_Srinivasan_2021-10_FA/TST_2021-10_FA_002.csv
(i): optional column not found: 'MD_Displacement--\d'
(i): optional column not found: 'T--\d'
(i): optional column not found: 'Storage_modulus'
(i): optional column not found: 'Tan_delta'
Warning: Experiment encountered errors -> cannot save any preprocessed data

(data preprocessing): TST_Liu_2020-09_FA

Hi,
Your data didn't successfully passed convention tests.
Please check the following validation output.
In order to integrate this dataset, you'll need to fix it and/or provide an update for the data convention (in agreement with all the colleagues involved).

Data convention is defined here :

• https://github.com/EPFL-ENAC/CCFatiguePlatform/blob/feat/data-validation/Data/TST_Data_Convention.md
• https://github.com/EPFL-ENAC/CCFatiguePlatform/blob/feat/data-validation/Data/TST_Data_Template.xls

(i): Parsing experiment /home/sbancal/IT4R/CCFatiguePlatform/Data/raw/TST_Liu_2020-09_FA
(i): Read /home/sbancal/IT4R/CCFatiguePlatform/Data/raw/TST_Liu_2020-09_FA/TST_2020-09_FA_metadata.xlsx Experiment tab
(i): Cleanup experiment
(i): Validate experiment
ERROR: missing mandatory column : Geometry>Length
ERROR: missing mandatory column : Geometry>Width
ERROR: missing mandatory column : Geometry>Thickness
ERROR: unrecognized value for column General>Fatigue Test Type = 'Tension'
ERROR: unrecognized value for column General>Control mode = 'Load controlled'
(i): Read /home/sbancal/IT4R/CCFatiguePlatform/Data/raw/TST_Liu_2020-09_FA/TST_2020-09_FA_metadata.xlsx Tests tab
(i): Cleanup tests
Warning: drop unexpected column Maximum load
Warning: drop unexpected column Load controlled loading rate
Warning: drop unexpected column Thickness
Warning: drop unexpected column x corrdinate of measuring point #
Warning: drop unexpected column Displacement controlled loading rate
Warning: drop unexpected column Specimen name
Warning: drop unexpected column Length
Warning: drop unexpected column Temperature
Warning: drop unexpected column initial crack length
Warning: drop unexpected column y corrdinate of measuring point #
Warning: drop unexpected column Humidity
Warning: drop unexpected column Width
Warning: drop unexpected column Frequency
(i): Validate tests
ERROR: mandatory column 'Maximum Stress' not found
ERROR: mandatory column 'Loading rate' not found
[ cut ---- 8X ---- ]
Warning: Experiment encountered errors -> cannot save any preprocessed data

(Fortran2Py) > Module 2 LinLog > Average problem for LNb

• Error ? LNb (average of all Log(N)) is summed but not reset to 0 between stress ratios, the average cannot be correct..
  https://github.com/EPFL-ENAC/CCFatiguePlatform/blob/develop/CCFatigue_modules/2_S-NCurves/S-N-Curve-LinLog.for#L297
  https://github.com/EPFL-ENAC/CCFatiguePlatform/blob/develop/CCFatigue_modules/2_S-NCurves/S-N-Curve-LinLog.for#L302

• Question :
  Is it on purpose that some variables are cumulative sum between stress ratios: LSST, LSSE

Make init-db may be misleading

Hi ! I ran into a "kind-of" bug when trying to populate the db, if we run make init-db and then make init-db --with-fake-data the only real experiment won't geet populated with fake tests.
Maybe we could add a test : if experiment already exists then only populate with fake tests. Because right now I think it's not working because trying to add an already existent experiment will stop the SQL query.
Or, maybe simpler, we could add explicit warning in the documentation to prevent people (like me 😛) from running the two commands one after the other.

(DB : Test Selection screen)

New issue template request : BUG

See Pierre's templates.

(Add new Datasets)

• Describe what's missing
• New TST Datasets
• New AGG Datasets

(data preprocessing): TST_Liu_2021-12_QS

Hi,
Your data didn't successfully passed convention tests.
Please check the following validation output.
In order to integrate this dataset, you'll need to fix it and/or provide an update for the data convention (in agreement with all the colleagues involved).

Data convention is defined here :

• https://github.com/EPFL-ENAC/CCFatiguePlatform/blob/feat/data-validation/Data/TST_Data_Convention.md
• https://github.com/EPFL-ENAC/CCFatiguePlatform/blob/feat/data-validation/Data/TST_Data_Template.xls

(i): Parsing experiment /home/sbancal/IT4R/CCFatiguePlatform/Data/raw/TST_Liu_2021-12_QS
(i): Read /home/sbancal/IT4R/CCFatiguePlatform/Data/raw/TST_Liu_2021-12_QS/TST_2021-12_QS_metadata.xlsx Experiment tab
(i): Cleanup experiment
(i): Validate experiment
ERROR: missing mandatory column : Geometry>Length
ERROR: missing mandatory column : Geometry>Width
ERROR: missing mandatory column : Geometry>Thickness
ERROR: unrecognized value for column General>Control mode = 'Load controlled'
(i): Read /home/sbancal/IT4R/CCFatiguePlatform/Data/raw/TST_Liu_2021-12_QS/TST_2021-12_QS_metadata.xlsx Tests tab
(i): Cleanup tests
Warning: drop unexpected column Loading rate (kN/s)
Warning: drop unexpected column Specimen name
(i): Validate tests
ERROR: mandatory column 'Stress Ratio' not found
ERROR: mandatory column 'Maximum Stress' not found
ERROR: mandatory column 'Loading rate' not found
ERROR: mandatory column 'Run-out' not found
(i): Read measures /home/sbancal/IT4R/CCFatiguePlatform/Data/raw/TST_Liu_2021-12_QS/TST_2021-12_QS_001.csv
(i): Read measures /home/sbancal/IT4R/CCFatiguePlatform/Data/raw/TST_Liu_2021-12_QS/TST_2021-12_QS_002.csv
(i): Read measures /home/sbancal/IT4R/CCFatiguePlatform/Data/raw/TST_Liu_2021-12_QS/TST_2021-12_QS_003.csv
(i): Read measures /home/sbancal/IT4R/CCFatiguePlatform/Data/raw/TST_Liu_2021-12_QS/TST_2021-12_QS_004.csv
(i): Read measures /home/sbancal/IT4R/CCFatiguePlatform/Data/raw/TST_Liu_2021-12_QS/TST_2021-12_QS_005.csv
(i): Read measures /home/sbancal/IT4R/CCFatiguePlatform/Data/raw/TST_Liu_2021-12_QS/TST_2021-12_QS_006.csv
(i): Cleanup measures /home/sbancal/IT4R/CCFatiguePlatform/Data/raw/TST_Liu_2021-12_QS/TST_2021-12_QS_001.csv
(i): Cleanup measures /home/sbancal/IT4R/CCFatiguePlatform/Data/raw/TST_Liu_2021-12_QS/TST_2021-12_QS_002.csv
(i): Cleanup measures /home/sbancal/IT4R/CCFatiguePlatform/Data/raw/TST_Liu_2021-12_QS/TST_2021-12_QS_003.csv
(i): Cleanup measures /home/sbancal/IT4R/CCFatiguePlatform/Data/raw/TST_Liu_2021-12_QS/TST_2021-12_QS_004.csv
(i): Cleanup measures /home/sbancal/IT4R/CCFatiguePlatform/Data/raw/TST_Liu_2021-12_QS/TST_2021-12_QS_005.csv
(i): Cleanup measures /home/sbancal/IT4R/CCFatiguePlatform/Data/raw/TST_Liu_2021-12_QS/TST_2021-12_QS_006.csv
(i): Validate measures /home/sbancal/IT4R/CCFatiguePlatform/Data/raw/TST_Liu_2021-12_QS/TST_2021-12_QS_001.csv
(i): optional column not found: 'T--\d'
(i): optional column not found: 'Storage_modulus'
(i): optional column not found: 'Tan_delta'
(i): Validate measures /home/sbancal/IT4R/CCFatiguePlatform/Data/raw/TST_Liu_2021-12_QS/TST_2021-12_QS_002.csv
(i): optional column not found: 'T--\d'
(i): optional column not found: 'Storage_modulus'
(i): optional column not found: 'Tan_delta'
(i): Validate measures /home/sbancal/IT4R/CCFatiguePlatform/Data/raw/TST_Liu_2021-12_QS/TST_2021-12_QS_003.csv
(i): optional column not found: 'T--\d'
(i): optional column not found: 'Storage_modulus'
(i): optional column not found: 'Tan_delta'
(i): Validate measures /home/sbancal/IT4R/CCFatiguePlatform/Data/raw/TST_Liu_2021-12_QS/TST_2021-12_QS_004.csv
(i): optional column not found: 'T--\d'
(i): optional column not found: 'Storage_modulus'
(i): optional column not found: 'Tan_delta'
(i): Validate measures /home/sbancal/IT4R/CCFatiguePlatform/Data/raw/TST_Liu_2021-12_QS/TST_2021-12_QS_005.csv
(i): optional column not found: 'T--\d'
(i): optional column not found: 'Storage_modulus'
(i): optional column not found: 'Tan_delta'
(i): Validate measures /home/sbancal/IT4R/CCFatiguePlatform/Data/raw/TST_Liu_2021-12_QS/TST_2021-12_QS_006.csv
(i): optional column not found: 'T--\d'
(i): optional column not found: 'Storage_modulus'
(i): optional column not found: 'Tan_delta'
Warning: Experiment encountered errors -> cannot save any preprocessed data

[Dataset to fix] : TST_Srinivasan_2021-10_QS

Dear @vsdharun ,

I realize your Dataset TST_Srinivasan_2021-10_QS that you can find here has also an inconsistency regarding the date.

• Folder says : 2021-10
• XLS file says : 2020-10

Please let me know what is the real date.
Thank you

ACCESS Issues

Within EPFL network

Security Warning
e.g., on Firefox :

Attention : risque probable de sécurité

Firefox a détecté un problème et a interrompu le chargement de ccfatigue-test.epfl.ch. Soit le site est mal configuré, soit l’horloge de votre ordinateur est réglée à la mauvaise heure.

Le certificat du site a probablement expiré, ce qui empêche Firefox d’établir une connexion sécurisée. Si vous visitez ce site, des attaquants pourraient dérober des informations telles que vos mots de passe, vos adresses électroniques ou vos informations de carte bancaire.

Que pouvez-vous faire ?

Le problème vient probablement du site web, vous ne pouvez donc pas y remédier. Vous pouvez le signaler aux personnes qui administrent le site.

En savoir plus…

Outside of EPFL network:

=> Not open to the world yet, so that normal :

This site can’t be reached. Check if there is a typo in ccfatigue-test.epfl.ch.
DNS_PROBE_FINISHED_NXDOMAIN

(DB: Migration noDB?)

• Fix DB bug
• Migration noDB ?

(Analysis: Mod2.Sendeckyj) Equation check

Sendeckyj > Equation 17 (p. 259)

Equation looks like exp is applied to first part [1/(m-k)] but in fortran it is applied to {[(1 / (m - k)] * sum of log(sigma_e))}, is it correct?

See in S-N-Curve-Sendeckyj.for, line 122

(Fortran2Py) > Command-line parsing of input parameters

In the modules already translated, transform all inputs (currently constants) as parameters
https://docs.python.org/3/library/argparse.html

Perhaps @blueur already plans to do that (or do you @nicdub ?)

Je pense qu'on pourrait garder en CONSTANTS un DEFAULT_INPUT avec des jeux de données par défault pour tester l'algo, qu'en penses-tu ? C'est une partie de réponse à ta question hier sur l'roganisation des données de test dans notre dossier Data ..

Search database - adjust columns for experiments

Visible columns are not optimal ... and even contain erased columns.
Fix it !

(Fortran2Py) > Module 2 Sendeckyj

• replace functions eq 17+ 19 by one function def estimate_maximum_likelihood (maximum likelihood is beta), so g becomes only an in-function parameters.

• Algo optimization decision (finding max alpha based on C and S) > run the 7 ones and select the best results

Migration eCharts

• Test Dashboard
• CCFatigue Analysis (Results)

CCFatigue analysis dashboard: S-N Plot missing experimental data

@scottausor Can you upgrade the function that plots S-N curve to integrate the experimental data points, as asked by Tassos in last meeting and displayed in design document slides?

TST_Data_Convention review

Hello,

I didn't notice the data type. But the Machine_Time and Th_time should be either Double or in hh:mm:ss.00 (ss in Double precision). Also we missed a column MD_time, whose type also be Double or hh:mm:ss.00.
Could you update these changes?
Seems there were some mistakes from our side. I am sorry for that. But these changes have been agreed among us for a long time. I don't know why we didn't include these in the last convention file...

Originally posted by @johnzer123 in #50 (comment)

preprocessing: validate metadata from file/folder naming

Validate that metadata read from file / folder naming matches the data found in the files.

See #58 and #60

New data convention report to DB + API+ Frontend

Report TST data convention struccture to :

• DB
• API
• Front

(data preprocessing): TST_Khalooei_2021-10_FA

With new data convention, we get following output :

(i): Parsing experiment /home/sbancal/IT4R/CCFatiguePlatform/Data/raw/TST_Khalooei_2021-10_FA
(i): Read /home/sbancal/IT4R/CCFatiguePlatform/Data/raw/TST_Khalooei_2021-10_FA/TST_2021-10_FA_metadata.xls Experiment tab
(i): Cleanup experiment
(i): Validate experiment
ERROR: Wrong type for column material type>area density: 425.0
(i): Read /home/sbancal/IT4R/CCFatiguePlatform/Data/raw/TST_Khalooei_2021-10_FA/TST_2021-10_FA_metadata.xls Tests tab
(i): Cleanup tests
Warning: drop unexpected column loading rate
Warning: drop unexpected column run-out
(i): Validate tests
ERROR: missing mandatory column when experiment type is 'FA': frequency
ERROR: missing mandatory column when experiment type is 'FA': run out
[ cut ---- 8X ---- ]
Warning: Experiment encountered errors -> cannot save any preprocessed data

(TST > AGG) Aggregate pipeline

[Doc] Missing doc on compilation pipeline

Is there an existing issue for this?

• I have searched the existing issues

Current Behavior

On branch develop, I run make api ... and get :

make -C backend api
make[1]: Entering directory '/home/sbancal/IT4R/CCFatiguePlatform/backend'
POSTGRES_PASSWORD="" poetry run python api.py ../openapi.json
Traceback (most recent call last):
File "/home/sbancal/IT4R/CCFatiguePlatform/backend/api.py", line 8, in
from ccfatigue.main import app
File "/home/sbancal/IT4R/CCFatiguePlatform/backend/ccfatigue/main.py", line 8, in
from ccfatigue.routers import root, experiments, tests, analysis
File "/home/sbancal/IT4R/CCFatiguePlatform/backend/ccfatigue/routers/analysis.py", line 7, in
from ccfatigue.analyzer import run_sn_curve
File "/home/sbancal/IT4R/CCFatiguePlatform/backend/ccfatigue/analyzer.py", line 15, in
import ccfatigue.modules.sn_curve_sendeckyj as sn_curve_sendeckyj
File "/home/sbancal/IT4R/CCFatiguePlatform/backend/ccfatigue/modules/sn_curve_sendeckyj.py", line 30, in
import ccfatigue.fortran.sendeckyj as sendeckyj # Fortran optimized functions
ModuleNotFoundError: No module named 'ccfatigue.fortran'
make[1]: *** [Makefile:20: api] Error 1
make[1]: Leaving directory '/home/sbancal/IT4R/CCFatiguePlatform/backend'
make: *** [Makefile:26: api] Error 2

Apparently I was missing the make compile inside backend folder.

Expected Behavior

• README.md file documents how to setup the env so that we can run anything.
• the step to setup should be launchable from the root of the project

Steps To Reproduce

make api

Environment

- OS: U22.04

Anything else?

thks : )

CYC Data convention

Do you have documentation for that module - as well as other examples of inputs and outputs to help us ?

Load spectrum input (example input) is the peak ?

Outputs (example output) should be, according to our last discussion, the 5 columns described here in the CYC Data convention. However,

• column 3 doesn’t look like R ratios at all !! What is it?
• Columns 4 should be “number_of_cycles” or “number_of_cycles_to_failure” ?
• Column 5 is “cumulative_number_of_cycles" you said, but as a percentage, right?

Detailed documentation on module 1’s methods would really help !

Test Dashboard: (i) buttons

Split (i) buttons per graph, next to Graph title, instead of a giant one! (May keep one geenral if deemed necesary)

(data preprocessing): TST_Srinivasan_2021-10_QS

Hi,
Your data didn't successfully passed convention tests.
Please check the following validation output.
In order to integrate this dataset, you'll need to fix it and/or provide an update for the data convention (in agreement with all the colleagues involved).

Data convention is defined here :

• https://github.com/EPFL-ENAC/CCFatiguePlatform/blob/feat/data-validation/Data/TST_Data_Convention.md
• https://github.com/EPFL-ENAC/CCFatiguePlatform/blob/feat/data-validation/Data/TST_Data_Template.xls

(i): Parsing experiment /home/sbancal/IT4R/CCFatiguePlatform/Data/raw/TST_Srinivasan_2021-10_QS
(i): Read /home/sbancal/IT4R/CCFatiguePlatform/Data/raw/TST_Srinivasan_2021-10_QS/TST_2021-10_QS_metadata.xlsx Experiment tab
(i): Cleanup experiment
(i): Validate experiment
ERROR: missing mandatory column : Geometry>Length
ERROR: missing mandatory column : Geometry>Width
ERROR: missing mandatory column : Geometry>Thickness
ERROR: unrecognized value for column General>Control mode = 'Displacement controlled'
(i): Read /home/sbancal/IT4R/CCFatiguePlatform/Data/raw/TST_Srinivasan_2021-10_QS/TST_2021-10_QS_metadata.xlsx Tests tab
(i): Cleanup tests
Warning: drop unexpected column Specimen name
Warning: drop unexpected column displacement loading rate
(i): Validate tests
ERROR: mandatory column 'Stress Ratio' not found
ERROR: mandatory column 'Maximum Stress' not found
ERROR: mandatory column 'Loading rate' not found
ERROR: mandatory column 'Run-out' not found
(i): Read measures /home/sbancal/IT4R/CCFatiguePlatform/Data/raw/TST_Srinivasan_2021-10_QS/TST_2021-10_QS_001.csv
(i): Read measures /home/sbancal/IT4R/CCFatiguePlatform/Data/raw/TST_Srinivasan_2021-10_QS/TST_2021-10_QS_002.csv
(i): Read measures /home/sbancal/IT4R/CCFatiguePlatform/Data/raw/TST_Srinivasan_2021-10_QS/TST_2021-10_QS_003.csv
(i): Read measures /home/sbancal/IT4R/CCFatiguePlatform/Data/raw/TST_Srinivasan_2021-10_QS/TST_2021-10_QS_004.csv
(i): Cleanup measures /home/sbancal/IT4R/CCFatiguePlatform/Data/raw/TST_Srinivasan_2021-10_QS/TST_2021-10_QS_001.csv
(i): Cleanup measures /home/sbancal/IT4R/CCFatiguePlatform/Data/raw/TST_Srinivasan_2021-10_QS/TST_2021-10_QS_002.csv
(i): Cleanup measures /home/sbancal/IT4R/CCFatiguePlatform/Data/raw/TST_Srinivasan_2021-10_QS/TST_2021-10_QS_003.csv
(i): Cleanup measures /home/sbancal/IT4R/CCFatiguePlatform/Data/raw/TST_Srinivasan_2021-10_QS/TST_2021-10_QS_004.csv
(i): Validate measures /home/sbancal/IT4R/CCFatiguePlatform/Data/raw/TST_Srinivasan_2021-10_QS/TST_2021-10_QS_001.csv
(i): optional column not found: 'T--\d'
(i): optional column not found: 'Storage_modulus'
(i): optional column not found: 'Tan_delta'
(i): Validate measures /home/sbancal/IT4R/CCFatiguePlatform/Data/raw/TST_Srinivasan_2021-10_QS/TST_2021-10_QS_002.csv
(i): optional column not found: 'T--\d'
(i): optional column not found: 'Storage_modulus'
(i): optional column not found: 'Tan_delta'
(i): Validate measures /home/sbancal/IT4R/CCFatiguePlatform/Data/raw/TST_Srinivasan_2021-10_QS/TST_2021-10_QS_003.csv
(i): optional column not found: 'T--\d'
(i): optional column not found: 'Storage_modulus'
(i): optional column not found: 'Tan_delta'
(i): Validate measures /home/sbancal/IT4R/CCFatiguePlatform/Data/raw/TST_Srinivasan_2021-10_QS/TST_2021-10_QS_004.csv
(i): optional column not found: 'T--\d'
(i): optional column not found: 'Storage_modulus'
(i): optional column not found: 'Tan_delta'
Warning: Experiment encountered errors -> cannot save any preprocessed data

[DOCS] Update Flowchart for Analysis part

todos @nicdub :

• Update FLowchart with detailed part for CCFatigue Analysis
• Check that variables dictionary is up to date - or update if necessary !

Questions for Tassos :
You said the only input to Module 3 (e.g Piecewise linear method) is the output of module 2 - SNC Data. However it seems like we need the stress amplitudes at known R ratios to be able to run Module 3. Module 2 does not output the stress amplitude, so where do we find these stress amplitudes ? In the output of module 1? Is it the same as stress_range ?

Also can you describe to us what the columns means in CCinput, a secondary input to module 5 ?

Preprocessing : integration into backend

Preprocessing pipeline has to be run on the backend -> so integrate it here !

(DB : Test Dashboard)

• Migration eCharts ?

CCFatigue analysis dashboard: S-N Curve - Allow for several Methods and several R-ratios selection

Either through checkboxes (on/off) in the dropdown, or through an additional option "All" in each dropdown.

Dataset entrypoint on Github issues

Perpare one Github Issue template that has an entry for "I want my dataset to be injected"

DB: Data checker (in "Data Upload" tab)

Data Upload tab :

• Users load zip of folder TST_... contining data and metadata
• Preprocessing scripts run and returns error log or a message "Your data is valid!" (then "Load yr data on Github" (link to New issue)"

Leave for later :

• Integration à la DB

Create "Feature request" issue template

For the researcher to be able to request some new features.
Split into

• Database feature
• Analysis feature
• other

[New Feature] : Results dashboard S-N curve plot X axis

Not a duplicate

• I have checked that this feature is not already present in the open issues

Feature description

In the Fatigue analysis :

X axis label is expected in the Scientific notation : 10^3 for 1'000.

(DB) Error in column name for TST_Khalooei_2021-10_FA

All 23 csv files from dataset TST_Khalooei_2021-10_FA have a column named Camera_N_cycles which is not matching current TST_DATA_Convention as discussed here #38 .

Should it be renamed to MD_N_cycles--1 or Crack_N_cycles ? Or even something else ?

CCFatiguePlatform/Data/TST_Khalooei_2021-10_FA/TST_2021-10_FA_15.csv

Line 1 in 09853c0

Machine_N_cycles,Machine_Load,Machine_Displacement,index,Camera_N_cycles,exx,eyy,exy,crack_length,Th_time,Th_N_cycles,Th_specimen_max,Th_specimen_mean,Th_chamber,Th_uppergrips,Th_lowergrips

FIX in TST_Data_Template.xls convention

Hi,
As I implement the specs in https://github.com/EPFL-ENAC/CCFatiguePlatform/blob/feat/data-validation/Data/TST_Data_Template.xls I realize you set Publication Year to be a string. Shouldn't that be an Int ?

deprecation to be removed

Surpris de voir le @deprecated() ... De ce que je vois, cet appel n'est plus fait nul part, donc c'est du code mort.

Originally posted by @sbancal in #114 (review)

(Fortran2Py) > Remove references to Fortran code once modules validated

Many py modules have links to Fortran version (in functions docstrings). To be removed once finalized.

(Fortran2Py) > Mod3 Piecewise-Linear

status : to clean up