joachimwolff / schicexplorer Goto Github PK

I encounter the following error when running the correct matrices command.

scHicCorrectMatrices -m ./tmp/merge/ramani_normalized.scool -o ./tmp/merge/ramani_corrected.scool --threads 20

...
DOne i: 6
DOne i: 7
DOne i: 8
DOne i: 9
Traceback (most recent call last):
  File "/mnt/data3/gianmarco/condaforge/envs/explorer/bin/scHicCorrectMatrices", line 7, in <module>
    main()
  File "/mnt/data3/gianmarco/condaforge/envs/explorer/lib/python3.6/site-packages/schicexplorer/scHicCorrectMatrices.py", line 169, in main
    matrixFileHandler.save(args.outFileName, pSymmetric=True, pApplyCorrection=False)
  File "/mnt/data3/gianmarco/condaforge/envs/explorer/lib/python3.6/site-packages/hicmatrix/lib/matrixFileHandler.py", line 57, in save
    self.matrixFile.save(pName, pSymmetric, pApplyCorrection)
  File "/mnt/data3/gianmarco/condaforge/envs/explorer/lib/python3.6/site-packages/hicmatrix/lib/scool.py", line 46, in save
    bins_dict[coolObject.matrixFile.matrixFileName] = bins_data_frame
TypeError: unhashable type: 'csr_matrix'

So far, the previous commands in the tutorial have run to completion with no apparent errors.

Other downstream tasks applied to the normalized matrix (skipping this step) also fail. I don't know if this is due to the missing correction or if those are separate errors.
For example:

hicPlotMatrix -m ./tmp/merge/ramani_normalized.scool     --log1p -o ./tmp/merge/ramani_bulk.png --dpi 300     --fontsize 5 --rotationX 45

INFO:hicexplorer.hicPlotMatrix:Cooler or no cooler: False
Traceback (most recent call last):
  File "/mnt/data3/gianmarco/condaforge/envs/explorer/bin/hicPlotMatrix", line 7, in <module>
    main()
  File "/mnt/data3/gianmarco/condaforge/envs/explorer/lib/python3.6/site-packages/hicexplorer/hicPlotMatrix.py", line 700, in main
    ma = HiCMatrix.hiCMatrix(args.matrix)
  File "/mnt/data3/gianmarco/condaforge/envs/explorer/lib/python3.6/site-packages/hicmatrix/HiCMatrix.py", line 56, in __init__
    matrixFileHandler_load = self.matrixFileHandler.load()
  File "/mnt/data3/gianmarco/condaforge/envs/explorer/lib/python3.6/site-packages/hicmatrix/lib/matrixFileHandler.py", line 51, in load
    return self.matrixFile.load()
  File "/mnt/data3/gianmarco/condaforge/envs/explorer/lib/python3.6/site-packages/hicmatrix/lib/cool.py", line 72, in load
    count_dtype = matrixDataFrame[0]['count'].dtype
  File "/mnt/data3/gianmarco/condaforge/envs/explorer/lib/python3.6/site-packages/cooler/core/_selectors.py", line 145, in __getitem__
    return self._slice(self.field, i0, i1, j0, j1)
  File "/mnt/data3/gianmarco/condaforge/envs/explorer/lib/python3.6/site-packages/cooler/api.py", line 384, in _slice
    self._is_symm_upper,
  File "/mnt/data3/gianmarco/condaforge/envs/explorer/lib/python3.6/site-packages/cooler/api.py", line 713, in matrix
    reader = CSRReader(h5['pixels'], h5['indexes/bin1_offset'][:])
  File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
  File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
  File "/mnt/data3/gianmarco/condaforge/envs/explorer/lib/python3.6/site-packages/h5py/_hl/group.py", line 288, in __getitem__
    oid = h5o.open(self.id, self._e(name), lapl=self._lapl)
  File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
  File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
  File "h5py/h5o.pyx", line 190, in h5py.h5o.open
KeyError: "Unable to open object (object 'pixels' doesn't exist)"

And

 scHicCluster -m ./tmp/merge/ramani_normalized.scool     --numberOfClusters 7 --clusterMethod kmeans -o ./tmp/merge/clusters_raw_kmeans.txt --threads 20
 
Traceback (most recent call last):
  File "/mnt/data3/gianmarco/condaforge/envs/explorer/bin/scHicCluster", line 7, in <module>
    main()
  File "/mnt/data3/gianmarco/condaforge/envs/explorer/lib/python3.6/site-packages/schicexplorer/scHicCluster.py", line 200, in main
    labels_clustering = kmeans_object.fit_predict(neighborhood_matrix)
  File "/mnt/data3/gianmarco/condaforge/envs/explorer/lib/python3.6/site-packages/sklearn/cluster/_kmeans.py", line 1077, in fit_predict
    return self.fit(X, sample_weight=sample_weight).labels_
  File "/mnt/data3/gianmarco/condaforge/envs/explorer/lib/python3.6/site-packages/sklearn/cluster/_kmeans.py", line 982, in fit
    accept_large_sparse=False)
  File "/mnt/data3/gianmarco/condaforge/envs/explorer/lib/python3.6/site-packages/sklearn/base.py", line 421, in _validate_data
    X = check_array(X, **check_params)
  File "/mnt/data3/gianmarco/condaforge/envs/explorer/lib/python3.6/site-packages/sklearn/utils/validation.py", line 63, in inner_f
    return f(*args, **kwargs)
  File "/mnt/data3/gianmarco/condaforge/envs/explorer/lib/python3.6/site-packages/sklearn/utils/validation.py", line 653, in check_array
    accept_large_sparse=accept_large_sparse)
  File "/mnt/data3/gianmarco/condaforge/envs/explorer/lib/python3.6/site-packages/sklearn/utils/validation.py", line 414, in _ensure_sparse_format
    _check_large_sparse(spmatrix, accept_large_sparse)
  File "/mnt/data3/gianmarco/condaforge/envs/explorer/lib/python3.6/site-packages/sklearn/utils/validation.py", line 761, in _check_large_sparse
    % indices_datatype)
ValueError: Only sparse matrices with 32-bit integer indices are accepted. Got int64 indices.

Could you please advise? I can provide the scool file or any additional information.

It would be helpful if more output files are available for users, especially outputs of different clustering methods.

While running scHiCExplorer with tutorial, I have some questions about scHiCExplorer tools such as Demultiplex and BuildMatrix.
(The tutorial means Analysis of single-cell Hi-C data index)

At first, in case of hicBuildMatrix, three arguments should be used, restrictionSequence, restrictionCutFile and danglingSequence

But in the tutorial, there are no required options in example command.
So I tried that command including these options and it work well.
(When I used this command without three required options, some error message were occurred)
Is that right the example command is false?
also some command is skipped the space between option and argument, such as -n1 -P1-I ...
Creation of Hi-C interaction matrices index is the example.

And what is the forward barcode and reverse barcode?
I think the one is barcode sequence and the other is the i5 sequence.
Is that right?

An error occurs when I run the following code：
scHicClusterMinHash -m raw_normalized.scool --numberOfHashFunctions 1200 --numberOfClusters 7 --clusterMethod kmeans -o clusters_minhash_kmeans.txt --threads 20

2022-12-23 16:06:01.484465: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2022-12-23 16:06:01.651899: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory
2022-12-23 16:06:01.651934: I tensorflow/compiler/xla/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
2022-12-23 16:06:02.484702: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory
2022-12-23 16:06:02.484799: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer_plugin.so.7'; dlerror: libnvinfer_plugin.so.7: cannot open shared object file: No such file or directory
2022-12-23 16:06:02.484811: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.
Traceback (most recent call last):
File "/home/lihaoxing/miniconda3/bin/scHicClusterMinHash", line 7, in
main()
File "/home/lihaoxing/miniconda3/lib/python3.9/site-packages/schicexplorer/scHicClusterMinHash.py", line 299, in main
cluster_object = KMeans(n_clusters=args.numberOfClusters, random_state=0, n_jobs=args.threads, precompute_distances=True)
TypeError: init() got an unexpected keyword argument 'n_jobs'

Another error occurs when I run the following code：
scHicPlotClusterProfiles -m raw_normalized.scool --clusters clusters_svl_spectral.txt -o clusters_svl_spectral.png --dpi 300 --threads 20

Traceback (most recent call last):
File "/home/lihaoxing/miniconda3/bin/scHicPlotClusterProfiles", line 7, in
main()
File "/home/lihaoxing/miniconda3/lib/python3.9/site-packages/schicexplorer/scHicPlotClusterProfiles.py", line 330, in main
cbar = plt.colorbar()
File "/home/lihaoxing/miniconda3/lib/python3.9/site-packages/matplotlib/pyplot.py", line 2065, in colorbar
ret = gcf().colorbar(mappable, cax=cax, ax=ax, **kwargs)
File "/home/lihaoxing/miniconda3/lib/python3.9/site-packages/matplotlib/figure.py", line 1277, in colorbar
cb = cbar.Colorbar(cax, mappable, **cb_kw)
File "/home/lihaoxing/miniconda3/lib/python3.9/site-packages/matplotlib/_api/deprecation.py", line 384, in wrapper
return func(*inner_args, **inner_kwargs)
File "/home/lihaoxing/miniconda3/lib/python3.9/site-packages/matplotlib/colorbar.py", line 380, in init
self._reset_locator_formatter_scale()
File "/home/lihaoxing/miniconda3/lib/python3.9/site-packages/matplotlib/colorbar.py", line 1165, in _reset_locator_formatter_scale
self._process_values()
File "/home/lihaoxing/miniconda3/lib/python3.9/site-packages/matplotlib/colorbar.py", line 1099, in _process_values
self.norm.vmin, self.norm.vmax = mtransforms.nonsingular(
File "/home/lihaoxing/miniconda3/lib/python3.9/site-packages/matplotlib/colors.py", line 1249, in vmin
self._changed()
File "/home/lihaoxing/miniconda3/lib/python3.9/site-packages/matplotlib/colors.py", line 1277, in _changed
self.callbacks.process('changed')
File "/home/lihaoxing/miniconda3/lib/python3.9/site-packages/matplotlib/cbook/init.py", line 312, in process
self.exception_handler(exc)
File "/home/lihaoxing/miniconda3/lib/python3.9/site-packages/matplotlib/cbook/init.py", line 96, in _exception_printer
raise exc
File "/home/lihaoxing/miniconda3/lib/python3.9/site-packages/matplotlib/cbook/init.py", line 307, in process
func(*args, **kwargs)
File "/home/lihaoxing/miniconda3/lib/python3.9/site-packages/matplotlib/image.py", line 326, in changed
cm.ScalarMappable.changed(self)
File "/home/lihaoxing/miniconda3/lib/python3.9/site-packages/matplotlib/cm.py", line 683, in changed
self.callbacks.process('changed', self)
File "/home/lihaoxing/miniconda3/lib/python3.9/site-packages/matplotlib/cbook/init.py", line 312, in process
self.exception_handler(exc)
File "/home/lihaoxing/miniconda3/lib/python3.9/site-packages/matplotlib/cbook/init.py", line 96, in _exception_printer
raise exc
File "/home/lihaoxing/miniconda3/lib/python3.9/site-packages/matplotlib/cbook/init.py", line 307, in process
func(*args, **kwargs)
File "/home/lihaoxing/miniconda3/lib/python3.9/site-packages/matplotlib/colorbar.py", line 495, in update_normal
self._draw_all()
File "/home/lihaoxing/miniconda3/lib/python3.9/site-packages/matplotlib/colorbar.py", line 530, in _draw_all
self._process_values()
File "/home/lihaoxing/miniconda3/lib/python3.9/site-packages/matplotlib/colorbar.py", line 1103, in _process_values
b = self.norm.inverse(b)
File "/home/lihaoxing/miniconda3/lib/python3.9/site-packages/matplotlib/colors.py", line 1708, in inverse
raise ValueError("Invalid vmin or vmax")
ValueError: Invalid vmin or vmax

Dear developers, thank you for this tool. I want to apply it to the data published in this paper (https://pubmed.ncbi.nlm.nih.gov/28135255/), and I was wondering if you have any advice on how to proceed. From my understanding, I should aim to obtain single-cell level bam files, and then I can proceed with the standard workflow. Is this correct? Do you have any experience in processing that type of data? I hope this is a good place to ask.
Best

Hi There,

Firstly, thanks for the great tools to explore the sc HiC data! I tested scHiCExplorer pipe following the instruction. I started from Download of the fastq files. Everything went on very well until Correction
The command line used:
scHicCorrectMatrices -m nagano2017_normalized.scool -o nagano2017_corrected.scool --threads 20
Error messages:(both python3.6 and 3.8 did not work and got the similar error)
_

_

Any suggestion on how to fix hashing errors in scHiCExplorer? Many thanks!