The implementation of a 'Undo Processing' button in the 'File' menu would be very beneficial to users to judge if a certain processing step improves their data. However, this will require that HappyTools will store the data at every processing step in memmory.

This leads to the case where HappyTools can open data that is encoded in the US style, but needs a lot of messy logic to deal with each country. The program must be swapped to use a method that works regardless of what country the data is taken from (specifically the numerical formatting).

After navigating to the program location and typing "HappyTools.py" I get the following error message:

C:\mypath\HappyTools> python HappyTools.py
Traceback (most recent call last):
File "HappyTools.py", line 43, in
from HappyTools.util.peak_detection import PeakDetection
File "C:\mypath\HappyTools\HappyTools\util\peak_detection.py", line 3, in
from HappyTools.bin.peak import Peak
File "C:\mypath\HappyTools\HappyTools\bin\peak.py", line 5, in
from scipy.interpolate import InterpolatedUnivariateSpline
File "C:\mypath\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.7_qbz5n2kfra8p0\LocalCache\local-packages\Python37\site-packages\scipy\interpolate_init_.py", line 167, in
from .interpolate import *
File "C:\mypath\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.7_qbz5n2kfra8p0\LocalCache\local-packages\Python37\site-packages\scipy\interpolate\interpolate.py", line 15, in
import scipy.special as spec
File "C:\mypath\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.7_qbz5n2kfra8p0\LocalCache\local-packages\Python37\site-packages\scipy\special_init_.py", line 634, in
from . import _ufuncs
ImportError: DLL load failed: The specified module could not be found.

Describe the bug
When opening a standard mix for HPLC calibration and smoothing and baseline correcting the data followed by "Peak detection", no peaks are detected.
Since this procedure works for the supplied example data, I suspect the problem with my own supplied files.

Settings are as followed:

Can someone please take a look and explain if I am doing something wrong?

Build: 109718a
OS: Win 10 64x
Python 3.7.4

Thanks in advance for your help,

Kind regards!

Refactor HappyTools to be object orientated, mimicing early work done by DavidJRobertson (https://github.com/DavidJRobertson/HappyTools) and also as indicated by Bryan Oakley on SO (https://stackoverflow.com/a/17470842/1093485).

Describe the bug
HappyTools currently does not work properly on Mac OS (or posix) systems due to issues with Tcl and matplotlib backends.

To Reproduce
Steps to reproduce the behavior:

1. Run HappyTools on Mac OS X.

Expected behavior
HappyTools should start as expected.

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

• OS: Mac OS

Additional context
traceback.txt
call_stack.txt

This class will contain all information about a Peak, such as area, S/N, Gaussian peak area etc. Furthermore, this class should contain the methods to do all the peak quantitations (migrating them from the current Functions class.

Describe the bug
The "Save Chromatogram" button doesn't work, the function call is wrong, see log message below:

2019-12-09 14:03 __main__ ERROR save_chromatogram() takes 1 positional argument but 2 were given

To Reproduce
Steps to reproduce the behavior:

1. Open example data
2. Perform "Baseline Correction"
3. Click "Save Chromatogram"
4. Open 'saved' chromatogram

Expected behavior
The newly opened chromatogram (after step 4) should be baseline corrected, but it is the original chromatogram

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

• Version 0.1-beta1
• Build 190718a

Additional context
Add any other context about the problem here.

This was caused by the i['Peak'] value no longer being in the middle of i['Data'] after
the border was changed. The fix was to always compare to relevant border of i['Data'] to
i['Peak'] rather than the two opposite ends of i['Data'].

Detailed Explanation:

--- ORIGINAL ---
Peak 1: 15.00-15.50 - Intensity: 1
Peak 2: 15.25-15.75 - Intensity: 1
Peak 3: 15.70-16.20 - Intensity: 1

--- PROCESSED --
Peak 1: 15.000-15.375 - Intensity: 1
Peak 2: 15.375-15.750 - Intensity: 1
Peak 3: 15.700-16.200 - Intensity: 1

--- CAUSE ---

15.750 - 15.375 = 0.375 (down from 0.5)
15.50 (Center) + 0.5*0.375 = 15.6875
Peak 2 Border (Expected) = 15.6875 - Peak 2 Border (Actual) = 15.750
No overlap between Peak 3 (Lower) and Peak 2 (Expected)
No adjustment made between peak 2 and 3

A request was made if HappyTools can also generated a PDF report for chromatograms that were not calibrated due to the chromatogram not having the required number of calibrant peaks above the specified signal-to-noise ratio.

HappyTools will not work properly if it doesn't have read/write access to required folders. We will implement a function that checks if HappyTools has the necessary rights and if not, warn the user.

Hi! I was taking a look at your repo after following the link from your paper and noted that the readme refers to a CONTRIBUTING file, but I couldn't actually find the file...

The current implementation of gui.Settings is overly complex and would benefit from simplifcation by using a standard library such as json or configparser

The current parameter that specifies the threshold for peak detection is hidden within the code and can't be changed from within the program, which has to be addressed.

HappyTools attempts to cast numbers into floats, with a (power)-user defined precision, however it is possible that the intensity contains a number using scientific notation that causes HappyTools to crash.

Describe the bug
HappyTools will not start if there is no 'HappyTools.ini' (settings.settings) file present in the HappyTools base folder.

To Reproduce
Steps to reproduce the behavior:

1. Download b180913a
2. Run 'HappyTools.py'

Expected behavior
The HappyTools GUI should appear.

Desktop (please complete the following information):

• OS: Win7
• Version: 0.1-alpha1
• Build: 180913a

Hi,
I'm trying to detect peaks in my GPC (gel permeation chromatography) data.
However, no peak was detected in HappyTools. Does anyone know the reason or the proper setting?
Screenshot and data are attached.
Thanks!

GPC chromatogram.txt

The current settings window has no clear structure/logic anymore due to addition of more settings. This should be addressed by restructuring.

import numpy as np imports more than is actually needed by the program, causing the Windows binary to be bigger than it has to be. This can be reduced by importing only the specific parts of numpy by using from numpy import foo.

Describe the bug
The matplotlib canvas seems to resize randomly

To Reproduce

1. Goto 'File'
2. Click on 'Open Chromatogram'
3. Select 'IgG Vtag 1_ACQUITY FLR ChA.txt' from 'Example Data'
4. Click 'Open'
5. Goto 'File'
6. Click on 'Normalize Chromatogram'
7. See bug (canvas completely different size)

Expected behavior
The canvas size should never change as it is set at the start of the program and the program uses the axes object to change what is shown on the canvas.

Screenshots
After step 4

After step 6

Desktop (please complete the following information):

• OS: Win10
• Version: 0.0.2
• Build: 20180905a

Only the python source files (.py) should be in the repository - the compiled (.pyc) files are not guaranteed to be portable between systems.

I can see that the .gitignore file contains an entry for *.pyc, however this must have been added after the the .pyc files were already in the repository, so they would have to be removed manually.

The 180815b build introduced a bug where the residual is always listed as 100%.

Describe the bug
The figure window is resized when more legends are shown than fit on the screen.

To Reproduce
Steps to reproduce the behavior:

1. Open more than 50 chromatograms.

Expected behavior
The image size should have remained unaffected, with the legend going 'off-frame' (for which a different solution has to be created in the future).

Screenshots
See the stackoverflow question link for full details -> https://stackoverflow.com/questions/53081084/matplotlib-axes-legend-changes-image-size-image-aspect?noredirect=1#comment93059351_53081084

Desktop (please complete the following information):

• Version 0.1-alpha1
• Build b181015a

The tracebuffer.txt file remains after closing HappyTools.

This class should contain all the aspects needed to both initialize the output summary and add the desired output blocks to the summary file.

Combining the overal chromatogram PDF and all the individual peaks into a single PDF file would provide a significant quality of life improvement. Ideally, this would not require any additional library besides what comes bundled with Anaconda (meaning reportlab is not a valid option).

The create figure setting opens a pdf object, when this setting is false the program will still try to close the non existing pdf object and thus crash.

Describe the bug
HappyTools throws an Exception if a Peak instance is outside the Trace time window set in the Settings instance (during background and noise determination).

To Reproduce

Expected behavior
The Peak shouldn't be considered or processed if it falls outside the Trace time window set in the Settings instance (at all).

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

Additional context
Bug uncovered by Matthew Choo

Is your feature request related to a problem? Please describe.
The user currently is unable to tell if certain processing steps were performed during the GUI based processing, such as baseline correction. Therefore, it would be a nice feature if in the summary file at the top it also lists if baseline correction (and other similar steps) were performed.

Describe the solution you'd like
See above.

Describe alternatives you've considered
N/A

Additional context
N/A

plasma-control_37_Emission_1.txt

Performing automatic peak detection on the attached sample (with a Minimum Intensity of 0.01) crashes HappyTools with an empty i['Data'] array. The cause was that the 'peak' value was 25.65 while the data covered the 24.52 to 24.75 range, this causes the entire data set to be discared during overlap removal.

Describe the bug
HappyTools will attempt to quantify all files of the allowed filetypes in the folder selected as 'Batch Directory' during batch process, which includes files that are not chromatograms such as the LICENSE.txt file.

To Reproduce
Steps to reproduce the behavior:

1. Copy the example data and files to the base HappyTools directory.
2. Run HappyTools.
3. Select Batch Process
4. Select the calibration file in the HappyTools base directory
5. Select the HappyTools base directory as the 'Batch Directory'
6. Run

Expected behavior
HappyTools should ignore non chromatogram files.

Screenshots

Desktop (please complete the following information):

• Version 0.1-alpha
• Build 181005b

Happytools does not entirely function with Python 3.8. Even with Python 3.7.0 (32 bit) the program has similar issues.
The chromatograms can be opened, baseline correction works, calibration also works. However, the program does not work for the integration.
In case of Python 3.7.0, (32 bit) on uninstalling the numpy, scipy and matplotlib libraries and reinstalling the recommended version of numpy 1.15.0 , scipy 1.1.0 and matplotlib 2.2.3; the integration function of happytools works.

Describe the bug
HappyTools' Batchprocess can take various file formats as an input for calibration (txt, arw and csv). However, the quantitation expects a 'txt' file but the calibration did not change the extension. Therefore, Batchprocess didn't quantify arw or csv files.

To Reproduce
Steps to reproduce the behavior:

1. Goto Batchprocess
2. Perform calibration on folder with acccepted non-txt files
3. Perform quantitation on the same folder
4. Note that there are calibrated files with the original extension, yet the summary is empty

Expected behavior
HappyTools should have changed the file format of the calibrated files to 'txt'.

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

• OS: [e.g. iOS]
• Browser [e.g. chrome, safari]
• Version [e.g. 22]

Additional context
Bug uncovered by glycoaddict (Matthew Choo)

I am unable to install the dependencies easily because the pip freeze requirements.txt file is missing.

HappyTools should support as many data formats as possible, however this requires input from users. Therefore, if you have any chromatographic data in a format that we do not currently support, please send it to us!

Describe the bug
The old legend is visible under the new legend once additional/different chromatograms are loaded after a previous plot.

To Reproduce
Steps to reproduce the behavior:

1. Open 2 of the test chromatograms
2. Open 1 different test chromatogram
3. Notice the old legend still being visible

Expected behavior
The old label should have been removed/cleared

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

• Version 0.1-alpha1
• Build 181005c

Python2 has EOL planned for 2020.

Hi, I'm very interested in implementing HappyTools in my lab.
The tutorial PDF in the package is for v0.02.
Some screenshots are different from current 0.1 -beta version.
I have some problems in following tutorial. (failed batch processing, unable to Save Calibrants)
Could you please update the tutorial so that I can find which is the problem, my procedure or code itself.
Thank you!

Hi,

I found that my workspace (Python3.7, anaconda distribution, windows10 64bit) have a problem in opening example chromatogram.
The issue would be the default text encoding (CP932) in my windows workspace.
Modifying the file_parser.py line 47
from
__ with Path(master.filename).open() as fr: __
to
__ with Path(master.filename).open(encoding="utf-8_sig") as fr: __
solved my problem.

If this modification makes no problem in other workspaces, could you consider setting this as default code?
Thank you,

HappyTools generates values that are written in scientific notation to calibrated files. The quantitation aspect of the program can not read numbers in scientific notation.

HappyTools has trouble processing analyte for which the time region does not contain a local maxima or minima. The cause for this is that the program goes to the except IndexError clause in the following bit from the batchFunctions.py file:

    try:
        if max(newY[0:breaks[0]]) > maxPoint:
            maxPoint = max(newY[0:breaks[0]])
            xData = newX[0:breaks[0]]
            yData = [x - NOBAN['Background'] for x in newY[0:breaks[0]]]
        for index,j in enumerate(breaks):
            try:
                if max(newY[breaks[index]:breaks[index+1]]) > maxPoint:
                    maxPoint = max(newY[breaks[index]:breaks[index+1]])
                    xData = newX[breaks[index]:breaks[index+1]]
                    yData = [x - NOBAN['Background'] for x in newY[breaks[index]:breaks[index+1]]]
            except:
                if max(newY[breaks[index]:-1]) > maxPoint:
                    maxPoint = max(newY[breaks[index]:-1])
                    xData = newX[breaks[index]:-1]
                    yData = [x - NOBAN['Background'] for x in newY[breaks[index]:-1]]
                pass
    **except IndexError:
        pass**

The result is that xData and yData are empty, thus crashing the next segment of the code.

Describe the bug
Saving of calibrants after Peak Detection in Example File is not working.
Return text in py console:
Exception in Tkinter callback Traceback (most recent call last): File "C:\Users\kaihi\AppData\Local\Programs\Python\Python37\lib\tkinter\__init__.py", line 1705, in __call__ return self.func(*args) File "C:\Users\kaihi\Desktop\HappyTools-master\HappyTools.py", line 388, in save_calibrants save_calibrants(self) File "C:\Users\kaihi\Desktop\HappyTools-master\HappyTools\util\functions.py", line 42, in save_calibrants raise NotImplementedError('This feature is not implemented in the ' + NotImplementedError: This feature is not implemented in the refactor yet.

To Reproduce
Steps to reproduce the behavior:

1. Go to 'File\OpenChromatogram\IgG Vtag 1_ACQUITY FLR ChA...'
2. Perform Smoothing and Baseline Correction as in Tutorial.
3. Click on 'Advanced\Peak Identification'
4. See peaks being identified.
5. Click on 'Advanced\SaveCalibrant'
6. See error in console.

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots

Desktop (please complete the following information):

• OS: Windows 10 64 x bit
• Browser Google Chrome 76.0.3809.100
• Version HappyTools - 0.1-beta1 Build 190718a

The attached python code illustrates the 'issue' and also shows how it can lead to certain peaks being missed.

#! /usr/bin/env python
from scipy.optimize import curve_fit
import bisect
import numpy as np

X = [16.4697402328,16.4701402404,16.4705402481,16.4709402557,16.4713402633,16.4717402709,16.4721402785,16.4725402862,16.4729402938,16.4733403014,16.473740309,16.4741403166,16.4745403243,16.4749403319,16.4753403395,16.4757403471,16.4761403547,16.4765403623,16.47694037,16.4773403776,16.4777403852,16.4781403928,16.4785404004,16.4789404081,16.4793404157,16.4797404233,16.4801404309,16.4805404385,16.4809404462,16.4813404538,16.4817404614,16.482140469,16.4825404766,16.4829404843,16.4833404919,16.4837404995,16.4841405071,16.4845405147,16.4849405224,16.48534053,16.4857405376,16.4861405452,16.4865405528,16.4869405604,16.4873405681,16.4877405757,16.4881405833,16.4885405909,16.4889405985,16.4893406062,16.4897406138,16.4901406214,16.490540629,16.4909406366,16.4913406443,16.4917406519,16.4921406595,16.4925406671,16.4929406747,16.4933406824,16.49374069,16.4941406976,16.4945407052,16.4949407128,16.4953407205,16.4957407281,16.4961407357,16.4965407433,16.4969407509,16.4973407585,16.4977407662,16.4981407738,16.4985407814,16.498940789,16.4993407966,16.4997408043,16.5001408119,16.5005408195,16.5009408271,16.5013408347,16.5017408424,16.50214085,16.5025408576,16.5029408652,16.5033408728,16.5037408805,16.5041408881,16.5045408957,16.5049409033,16.5053409109,16.5057409186,16.5061409262,16.5065409338,16.5069409414,16.507340949,16.5077409566,16.5081409643,16.5085409719,16.5089409795,16.5093409871,16.5097409947,16.5101410024,16.51054101,16.5109410176,16.5113410252,16.5117410328,16.5121410405,16.5125410481,16.5129410557,16.5133410633,16.5137410709,16.5141410786,16.5145410862,16.5149410938,16.5153411014,16.515741109,16.5161411166,16.5165411243,16.5169411319,16.5173411395,16.5177411471,16.5181411547,16.5185411624,16.51894117,16.5193411776,16.5197411852,16.5201411928,16.5205412005,16.5209412081,16.5213412157,16.5217412233,16.5221412309,16.5225412386,16.5229412462,16.5233412538,16.5237412614,16.524141269,16.5245412767,16.5249412843,16.5253412919,16.5257412995,16.5261413071,16.5265413147,16.5269413224,16.52734133,16.5277413376,16.5281413452,16.5285413528,16.5289413605,16.5293413681,16.5297413757,16.5301413833,16.5305413909,16.5309413986,16.5313414062,16.5317414138,16.5321414214,16.532541429,16.5329414367,16.5333414443,16.5337414519,16.5341414595,16.5345414671,16.5349414748,16.5353414824,16.53574149,16.5361414976,16.5365415052,16.5369415128,16.5373415205,16.5377415281,16.5381415357,16.5385415433,16.5389415509,16.5393415586,16.5397415662,16.5401415738,16.5405415814,16.540941589,16.5413415967,16.5417416043,16.5421416119,16.5425416195,16.5429416271,16.5433416348,16.5437416424,16.54414165,16.5445416576,16.5449416652,16.5453416729,16.5457416805,16.5461416881,16.5465416957,16.5469417033,16.5473417109,16.5477417186,16.5481417262,16.5485417338,16.5489417414,16.549341749,16.5497417567,16.5501417643,16.5505417719,16.5509417795,16.5513417871,16.5517417948,16.5521418024,16.55254181,16.5529418176,16.5533418252,16.5537418329,16.5541418405,16.5545418481,16.5549418557,16.5553418633,16.5557418709,16.5561418786,16.5565418862,16.5569418938,16.5573419014,16.557741909,16.5581419167,16.5585419243,16.5589419319,16.5593419395,16.5597419471,16.5601419548,16.5605419624,16.56094197,16.5613419776,16.5617419852,16.5621419929,16.5625420005,16.5629420081,16.5633420157,16.5637420233,16.564142031]
Y = [11579127.8554,11671781.7263,11764419.0191,11857026.0444,11949589.1124,12042094.5338,12134528.6188,12226877.6781,12319128.0219,12411265.9609,12503277.8053,12595149.8657,12686868.4525,12778419.8762,12869790.334,12960965.209,13051929.5278,13142668.3154,13233166.5969,13323409.3973,13413381.7417,13503068.6552,13592455.1627,13681526.2894,13770267.0602,13858662.5004,13946697.6348,14034357.4886,14121627.0868,14208491.4544,14294935.6166,14380944.5984,14466503.4248,14551597.1208,14636210.7116,14720329.3102,14803938.4081,14887023.5981,14969570.4732,15051564.6263,15132991.6503,15213837.1383,15294086.683,15373725.8775,15452740.3147,15531115.5875,15608837.2888,15685891.0116,15762262.3488,15837936.8934,15912900.2382,15987137.9762,16060635.7004,16133379.0036,16205353.4789,16276544.72,16346938.7731,16416522.8674,16485284.4226,16553210.8587,16620289.5956,16686508.0531,16751853.6511,16816313.8096,16879875.9485,16942527.4876,17004255.8468,17065048.446,17124892.7052,17183776.0442,17241685.8829,17298609.6412,17354534.739,17409448.5962,17463338.6327,17516192.2683,17567996.9463,17618741.7702,17668418.588,17717019.5043,17764536.6238,17810962.0514,17856287.8916,17900506.2493,17943609.2292,17985588.936,18026437.4744,18066146.9493,18104709.4653,18142117.1271,18178362.0396,18213436.3074,18247332.0352,18280041.3279,18311556.2901,18341869.0265,18370971.642,18398856.332,18425517.6188,18450952.493,18475158.064,18498131.4412,18519869.7341,18540370.0523,18559629.505,18577645.202,18594414.2525,18609933.7661,18624200.8523,18637212.6205,18648966.1802,18659458.6408,18668687.1119,18676648.7029,18683340.5233,18688759.6825,18692903.29,18695768.4553,18697352.5327,18697655.9558,18696681.2608,18694431.0245,18690907.8241,18686114.2363,18680052.838,18672726.2063,18664136.918,18654287.5501,18643180.6795,18630818.883,18617204.7377,18602340.8204,18586229.7081,18568873.9777,18550276.2061,18530438.9703,18509364.8471,18487056.4135,18463516.2464,18438747.4526,18412756.9228,18385553.1936,18357144.808,18327540.3094,18296748.2409,18264777.1456,18231635.5669,18197332.0479,18161875.1318,18125273.3619,18087535.2812,18048669.4331,18008684.3606,17967588.6071,17925390.7158,17882099.2297,17837722.6922,17792269.6464,17745748.6355,17698168.2027,17649537.512,17599868.3744,17549173.3069,17497464.8262,17444755.4492,17391057.6927,17336384.0736,17280747.1087,17224159.3148,17166633.2088,17108181.3075,17048816.1277,16988550.1864,16927396.0002,16865366.0862,16802472.961,16738729.1416,16674147.1447,16608739.4873,16542518.6861,16475497.2591,16407688.2541,16339106.0951,16269765.4262,16199680.8916,16128867.1358,16057338.8029,15985110.5372,15912196.9829,15838612.7844,15764372.5859,15689491.0316,15613982.7659,15537862.4329,15461144.6771,15383844.1425,15305975.4735,15227553.3143,15148592.3093,15069107.1026,14989112.3386,14908622.6595,14827652.5673,14746216.3337,14664328.209,14582002.4435,14499253.2874,14416094.9911,14332541.8049,14248607.9791,14164307.764,14079655.4098,13994665.1668,13909351.2855,13823728.016,13737809.6086,13651610.3137,13565144.3816,13478426.0625,13391469.6068,13304289.2646,13216899.2865,13129313.8865,13041546.3657,12953609.0623,12865514.2686,12777274.277,12688901.3798,12600407.8693,12511806.0378,12423108.1777,12334326.5812,12245473.5407,12156561.3486,12067602.297,11978608.6785,11889592.7852]

def gaussFunction(x, *p):
"""Define and return a Gaussian function.

This function returns the value of a Gaussian function, using the
A, mu and sigma value that is provided as *p.

Keyword arguments:
x -- number
p -- A, mu and sigma numbers
"""
A, mu, sigma = p
return A*np.exp(-(x-mu)**2/(2.*sigma**2))

newGaussX = np.linspace(10, 25, 2500*(X[-1]-X[0]))
p0 = [np.max(Y), X[np.argmax(Y)],0.1]

coeff, var_matrix = curve_fit(gaussFunction, X, Y, p0)
newGaussY = gaussFunction(newGaussX, *coeff)

print "Sigma is "+str(coeff[2])

Original

low = bisect.bisect_left(newGaussX,coeff[1]-2coeff[2])
high = bisect.bisect_right(newGaussX,coeff[1]+2coeff[2])
print newGaussX[low], newGaussX[high]

Absolute

low = bisect.bisect_left(newGaussX,coeff[1]-2abs(coeff[2]))
high = bisect.bisect_right(newGaussX,coeff[1]+2abs(coeff[2]))
print newGaussX[low], newGaussX[high]