haascp / mocca

Open-source Python project to analyze HPLC–DAD raw data.

MOCCA (Multivariate Online Contextual Chromatographic Analysis) is an open-source Python project to analyze HPLC–DAD raw data.

Automation and digitalization solutions in the field of small molecule synthesis face new challenges for chemical reaction analysis, especially in the field of high-performance liquid chromatography (HPLC). Chromatographic data remains locked in vendors’ hardware and software components limiting their potential in automated workflows and contradicting to FAIR data principles (findability, accessibility, interoperability, reuse), which enable chemometrics and data science applications. In this work, we present an open-source Python project called MOCCA (Multivariate Online Contextual Chromatographic Analysis) for the analysis of open-format HPLC–DAD (photodiode array detector) raw data. MOCCA provides a comprehensive set of data analysis features including a peak deconvolution routine which allows for automated deconvolution of known signals even if overlapped with signals of unexpected impurities or side products. By publishing MOCCA as a Python package, we envision an open-source community project for chromatographic data analysis with the potential of further advancing its scope and capabilities.

Open-source project: https://github.com/HaasCP/mocca

Documentation: https://mocca.readthedocs.io/en/latest/

Corresponding scientific publication (open access): ACS Central Science, 2023, https://doi.org/10.1021/acscentsci.2c01042.

Installation

1. We recommend creating an isolated conda environment to avoid any problems with your installed Python packages:

   conda create -n mocca python=3.9
   conda activate mocca
   

2. Install mocca and its dependencies:

   pip install mocca
   

3. If you want to use mocca's reporting functionality:

   pip3 install -U datapane==0.14
   

4. If you want to use Allotrope (adf) file format:

   pip install h5py
   pip install git+https://github.com/HDFGroup/h5ld@master
   

5. If you want to use mocca using JupyterLab notebooks:

   pip install jupyterlab
   ipython kernel install --user --name=mocca
   

Getting started

MOCCA is currently best used via JupyterLab notebooks. The notebooks folder of the GitHub repository contains a tutorial notebook with corresponding HPLC–DAD test data for the first steps.

Additionally, a full test data set from the scientific publication is added (cyanation of aryl halides via well plate screening). The corresponding notebook contains full data analysis details from the raw data level until the presented visualizations in the manuscript (Fig. 7e) and SI (Fig. S17).

How to cite

Peer-reviewed, open access:

Haas, C. P., Lübbesmeyer, M., Jin, E. H., McDonald, M. A., Koscher, B. A., Guimond, N., Di Rocco, L., Kayser, H., Leweke, S., Niedenführ, S., Nicholls, R., Greeves, E., Barber, D. M., Hillenbrand, J., Volpin, G., and Jensen, K. F. Open-Source Chromatographic Data Analysis for Reaction Optimization and Screening. ACS Cent.Sci. 2023. https://doi.org/10.1021/acscentsci.2c01042.

Preprint:

Haas, C. P., Lübbesmeyer, M., Jin, E. H., McDonald, M. A., Koscher, B. A., Guimond, N., Di Rocco, L., Kayser, H., Leweke, S., Niedenführ, S., Nicholls, R., Greeves, E., Barber, D. M., Hillenbrand, J., Volpin, G., and Jensen, K. F. Open-Source Chromatographic Data Analysis for Reaction Optimization and Screening. ChemRxiv 2022. https://doi.org/10.26434/chemrxiv-2022-0pv2d.

Note

This project has been set up using PyScaffold 4.1.1. For details and usage information on PyScaffold see https://pyscaffold.org/.

mocca's Issues

ImportError: cannot import name 'appengine' from 'requests.packages.urllib3.contrib'

set up environment mostly according to the tutorial:

conda create -n mocca python=3.9
conda activate mocca
pip install git+https://github.com/HaasCP/mocca@master
pip3 install -U datapane==0.14
pip install notebook

while going through the cyanation tutorial, I use this code:

# folders handling
import os
from glob import glob

# user interaction
from mocca.user_interaction.campaign import HplcDadCampaign
from mocca.user_interaction.user_objects import Gradient
from mocca.user_interaction.user_objects import Compound
from mocca.user_interaction.user_objects import InternalStandard
from mocca.user_interaction.user_objects import HplcInput
from mocca.user_interaction.settings import Settings

# reporting
from mocca.report.main import report

And I get this error:

---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
File ~\AppData\Local\anaconda3\envs\mocca\lib\site-packages\requests_toolbelt\_compat.py:48
     47 try:
---> 48     from requests.packages.urllib3.contrib import appengine as gaecontrib
     49 except ImportError:

ImportError: cannot import name 'appengine' from 'requests.packages.urllib3.contrib' (C:\Users\CGong\AppData\Local\anaconda3\envs\mocca\lib\site-packages\urllib3\contrib\__init__.py)

During handling of the above exception, another exception occurred:

ImportError                               Traceback (most recent call last)
Cell In[1], line 14
     11 from mocca.user_interaction.settings import Settings
     13 # reporting
---> 14 from mocca.report.main import report

File ~\AppData\Local\anaconda3\envs\mocca\lib\site-packages\mocca\report\main.py:8
      1 # -*- coding: utf-8 -*-
      2 """
      3 Created on Wed May 25 08:39:04 2022
      4 
      5 @author: HaasCP
      6 """
----> 8 from mocca.report.hplc_input import report_hplc_input
      9 from mocca.report.gradient import report_gradients
     10 from mocca.report.chromatograms import report_chroms

File ~\AppData\Local\anaconda3\envs\mocca\lib\site-packages\mocca\report\hplc_input.py:11
      9 import os
     10 import pandas as pd
---> 11 import datapane as dp
     14 def exps_to_df(exps):
     15     """
     16     Transfers relevant information of HplcInput objects in a pandas df.
     17     """

File ~\AppData\Local\anaconda3\envs\mocca\lib\site-packages\datapane\__init__.py:28
     25 from .common.utils import enable_logging, log, _setup_dp_logging
     27 # Public API re-exports
---> 28 from .client.api import (
     29     Attachment,
     30     Media,
     31     BigNumber,
     32     Code,
     33     DataTable,
     34     Divider,
     35     Embed,
     36     Empty,
     37     File,
     38     Group,
     39     FontChoice,
     40     Formula,
     41     HTML,
     42     Media,
     43     Text,
     44     Page,
     45     PageLayout,
     46     Params,
     47     Plot,
     48     Report,
     49     ReportFormatting,
     50     ReportWidth,
     51     Result,
     52     Run,
     53     Schedule,
     54     App,
     55     Select,
     56     SelectType,
     57     Table,
     58     Text,
     59     TextAlignment,
     60     Toggle,
     61     Environment,
     62     by_datapane,
     63     signup,
     64     login,
     65     logout,
     66     ping,
     67     hello_world,
     68     builtins,
     69 )
     70 from .client.config import init
     71 from .common.dp_types import DPMode, set_dp_mode, get_dp_mode

File ~\AppData\Local\anaconda3\envs\mocca\lib\site-packages\datapane\client\api\__init__.py:50
     46 # flake8: noqa F401
     47 # Internal API re-exports
     48 import warnings
---> 50 from .common import HTTPError, Resource
     51 from .dp_object import DPObjectRef
     52 from .report.blocks import (
     53     Attachment,
     54     BigNumber,
   (...)
     71     Toggle,
     72 )

File ~\AppData\Local\anaconda3\envs\mocca\lib\site-packages\datapane\client\api\common.py:27
     25 from packaging.version import Version
     26 from requests import HTTPError, Response  # noqa: F401
---> 27 from requests_toolbelt import MultipartEncoder, MultipartEncoderMonitor
     29 from datapane import _TEST_ENV, __version__
     30 from datapane.client import config as c

File ~\AppData\Local\anaconda3\envs\mocca\lib\site-packages\requests_toolbelt\__init__.py:12
      1 # -*- coding: utf-8 -*-
      2 """
      3 requests-toolbelt
      4 =================
   (...)
      9 :license: Apache v2.0, see LICENSE for more details
     10 """
---> 12 from .adapters import SSLAdapter, SourceAddressAdapter
     13 from .auth.guess import GuessAuth
     14 from .multipart import (
     15     MultipartEncoder, MultipartEncoderMonitor, MultipartDecoder,
     16     ImproperBodyPartContentException, NonMultipartContentTypeException
     17     )

File ~\AppData\Local\anaconda3\envs\mocca\lib\site-packages\requests_toolbelt\adapters\__init__.py:12
      1 # -*- coding: utf-8 -*-
      2 """
      3 requests-toolbelt.adapters
      4 ==========================
   (...)
      9 :license: Apache v2.0, see LICENSE for more details
     10 """
---> 12 from .ssl import SSLAdapter
     13 from .source import SourceAddressAdapter
     15 __all__ = ['SSLAdapter', 'SourceAddressAdapter']

File ~\AppData\Local\anaconda3\envs\mocca\lib\site-packages\requests_toolbelt\adapters\ssl.py:16
     12 import requests
     14 from requests.adapters import HTTPAdapter
---> 16 from .._compat import poolmanager
     19 class SSLAdapter(HTTPAdapter):
     20     """
     21     A HTTPS Adapter for Python Requests that allows the choice of the SSL/TLS
     22     version negotiated by Requests. This can be used either to enforce the
   (...)
     43     properly when used with proxies.
     44     """

File ~\AppData\Local\anaconda3\envs\mocca\lib\site-packages\requests_toolbelt\_compat.py:50
     48         from requests.packages.urllib3.contrib import appengine as gaecontrib
     49     except ImportError:
---> 50         from urllib3.contrib import appengine as gaecontrib
     52 if requests.__build__ < 0x021200:
     53     PyOpenSSLContext = None

ImportError: cannot import name 'appengine' from 'urllib3.contrib' (C:\Users\CGong\AppData\Local\anaconda3\envs\mocca\lib\site-packages\urllib3\contrib\__init__.py)

I'm not sure where the dependencies broke, but I was unable to do pip install mocca because the requirements in the pip package still uses sklearn rather than scikit-learn, so I had to improvise. I am working on Windows.

Edit: To clarify, the HplcDadCampaign and other classes used for analysis works fine, it's just report that is the problem.

Install fails: The 'sklearn' PyPI package is deprecated, use 'scikit-learn'

When attempting to install the mocca package, it fails due to the sklearn being depreciated, and should be replaced with scikit-learn. I was able to install by setting the environmental variable, but this isn't ideal.

  error: subprocess-exited-with-error
  
  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [18 lines of output]
      The 'sklearn' PyPI package is deprecated, use 'scikit-learn'
      rather than 'sklearn' for pip commands.
      
      Here is how to fix this error in the main use cases:
      - use 'pip install scikit-learn' rather than 'pip install sklearn'
      - replace 'sklearn' by 'scikit-learn' in your pip requirements files
        (requirements.txt, setup.py, setup.cfg, Pipfile, etc ...)
      - if the 'sklearn' package is used by one of your dependencies,
        it would be great if you take some time to track which package uses
        'sklearn' instead of 'scikit-learn' and report it to their issue tracker
      - as a last resort, set the environment variable
        SKLEARN_ALLOW_DEPRECATED_SKLEARN_PACKAGE_INSTALL=True to avoid this error
      
      More information is available at
      https://github.com/scikit-learn/sklearn-pypi-package
      
      If the previous advice does not cover your use case, feel free to report it at
      https://github.com/scikit-learn/sklearn-pypi-package/issues/new
      [end of output]

Possible to add more than one compound for an HplcInput() object.

I am hoping to use Mocca in my current research project. I was wondering if would be possible to enable the selection of more than one compound with known concentration and retention time in a HplcInput() object? This would allow me to use calibration samples with more than one analyte at the time, which is often how I prepare them. Currently, it appears to me that HplcInput() objects are limited to one compound object input.
Please let me know if I have misunderstood anything and no worries if the proposed change is unfeasible. Thanks!