francois-rozet / torchist

NumPy-style histograms in PyTorch

NumPy-style histograms in PyTorch

The torchist package implements NumPy's histogram and histogramdd functions in PyTorch with CUDA support. The package also features implementations of ravel_multi_index, unravel_index and some useful functionals like entropy or kl_divergence.

Installation

The torchist package is available on PyPI, which means it is installable with pip.

pip install torchist

Alternatively, if you need the latest features, you can install it from the repository.

pip install git+https://github.com/francois-rozet/torchist

Getting Started

import torch
import torchist

x = torch.rand(100, 3).cuda()

hist = torchist.histogramdd(x, bins=10, low=0.0, upp=1.0)

print(hist.shape)  # (10, 10, 10)

Benchmark

The implementations of torchist are on par or faster than those of numpy on CPU and benefit greately from CUDA capabilities.

$ python torchist/__init__.py
CPU
---
np.histogram : 1.2559 s
np.histogramdd : 20.7816 s
np.histogram (non-uniform) : 5.4878 s
np.histogramdd (non-uniform) : 17.3757 s
torchist.histogram : 1.3975 s
torchist.histogramdd : 9.6160 s
torchist.histogram (non-uniform) : 5.0883 s
torchist.histogramdd (non-uniform) : 17.2743 s

CUDA
----
torchist.histogram : 0.1363 s
torchist.histogramdd : 0.3754 s
torchist.histogram (non-uniform) : 0.1355 s
torchist.histogramdd (non-uniform) : 0.5137 s

torchist's Issues

histogramdd with sparse= True fails with input tensor = vector (dim 1)

Description

First of all THANK YOU very much for this nice work: it helps me a lot, and I though contacting you because given your code you likely work on closely related topic with information topology problematic i work on. The problem (if it is not a misunderstanding of mine) encountered is:
if a 1 dimensional tensor (vector) is given in input to Histogramdd with sparse= True then one get the error:
"line 194, in histogramdd
hist = torch.sparse_coo_tensor(idx.t(), values, shape)
RuntimeError: number of dimensions must be sparse_dim (10) + dense_dim (0), but got 1"

Reproduce

A minimal working example demonstrating the current behavior.

import torch
import torchist

x = torch.rand(100, 1).cuda()

hist = torchist.histogramdd(x, bins=10, low=0.0, upp=1.0, sparse = True)

Expected behavior

One could expect a sparse vector in output

Shape error in histogramdd

Hi, thank you for the nice package. I am experiencing an error that I cannot figure out while using the histogramdd function. I am making multiple histograms in a for-loop and I always feed the histogramdd function tensors of the same shape.

>>> inputs = torch.stack([y[i], x[i], z[i]], -1)  # shape [4000, 3]
>>> weights = w[i]                                # shape [4000]
>>> h0 = torchist.histogramdd(inputs, edges=[30, 32, 30], weights=weights)

It works fine most of the time, but then it breaks all of a sudden with this kind of error:

>>> h0 = torchist.histogramdd(inputs, edges=[30, 32, 30], weights=w[i])
File ".../torchist/__init__.py", line 225, in histogramdd
    hist = idx.bincount(weights, minlength=shape.numel()).view(shape)
RuntimeError: shape '[30, 32, 30]' is invalid for input of size 29358

Since the shape of what I feed in is always the same I am confused about what might be generating the error. It looks like idx has somehow the wrong shape, i.e. not 30x32x30. Or I am just misunderstanding something?

Edit: I am using version '0.1.8' and tensors are all CUDA tensors.

Assertion error with lower/upper bound lists

Description

bug 1: assert torch.all(upp > low), "The upper bound must be strictly larger than the lower bound"

    print(torch.all(upp > low), upp>low)
TypeError: all(): argument 'input' (position 1) must be Tensor, not bool

bug 2: func quantize:
quantize

    x = torch.clip(x, min=0, max=bins - 1)  # in [0, bins)
TypeError: clip() received an invalid combination of arguments - got (Tensor, max=Tensor, min=int), but expected one of:
 * (Tensor input, Tensor min, Tensor max, *, Tensor out)
 * (Tensor input, Number min, Number max, *, Tensor out)

Reproduce

bug 1: happens when low/upp is a list/sequence of scalars. torch.all requires tensor inputs, otherwise the output is False by default.

    t_hist = torchist.histogramdd(t_tensor, bins =[64, 64, 32], 
                                  low=[-a, -b, -c],  # variable low is a list/sequence of scalars
                                  upp=[a, b, c] # variable low is a list/sequence of scalars
                                  )

bug 2: bug2 appears after bug 1, because 'max=bins-1' is a tensor while 'min=0' is not.

Will add a minimal example later. Perhaps a pytorch version issue?

Expected behavior

Causes and solution

To fix:
bug 1: asset func stays after as_tensors;

    bins = torch.as_tensor(bins).squeeze().long()
    low = torch.as_tensor(low).squeeze().to(x)
    upp = torch.as_tensor(upp).squeeze().to(x)

    assert torch.all(upp > low), "The upper bound must be strictly larger than the lower bound"

bug 2: replace 'min=0' by 'min=x.new_zeros(bins.numel())'

x = torch.clip(x, min=x.new_zeros(bins.numel()), max=bins - 1)  # in [0, bins)

Environment

• torchist version: [0.2.0]
• PyTorch version: [1.12.0]
• Python version: [3.9.16]
• OS: [Ubuntu 22.04]

Error when using min() and max() to find bin edges

Description

Runtime error when calculating bin edges using .min() and .max() functions, instead of specifying fixed numerical values.

RuntimeError: shape '[10]' is invalid for input of size 11

Using np.histogram works correctly.

Reproduce

A minimal working example demonstrating the current behavior.

import torch
import torchist
import numpy as np

x = np.random.rand(10000)
x_min, x_max = x.min(), x.max()

edges = np.linspace(x_min, x_max, 11)
hist_np, _ = np.histogram(x, bins=edges)
print(hist_np.shape)

x_t = torch.from_numpy(x).cuda()
edges_t = torch.from_numpy(edges).cuda()
hist_t = torchist.histogram(x_t, edges=edges_t) # Error is reported here. 
print(hist_t.shape)  

Expected behavior

Both torchist and numpy results should work and produce matching results.

Causes and solution

Slightly changing the values of x_min and x_max solves the issue.
x_min, x_max = x.min()*1.01 , x.max()*1.01

Environment

• torchist version: 0.2.1
• PyTorch version: 1.12.1
• Python version: 3.10
• OS: Ubuntu 20.04 LTS