MultiRoaringBitmap slicing return type about roaringbitmap

[andreasvc]

It sounds like you should map your keys to integers and use https://roaringbitmap.readthedocs.io/en/latest/#roaringbitmap.MultiRoaringBitmap.intersection

[andreasvc]

A MultiRoaringBitmap manages and owns the storage in memory or on disk of the bitmaps it contains, so returning a MultiRoaringBitmap requires copying the data. If you do want a new instance of MultiRoaringBitmap with copies, you can do MultiRoaringBitmap(mrb[0:5]) as in your example.

Do you have a use case where having slices without making copies is crucial? It is probably possible to create a version of MultiRoaringBitmap that references a slice of another one, but I'm not sure implementing that is worth the effort ...

[remiadon]

My use case is the following

I am using Bitmaps to represent a transactional database : for every item in my database, I keep track of transaction ids where this item occurred.

>> from collections import defaultdict
>> from roaringbitmap import RoaringBitmap
>> database = [['milk', 'chocolate', 'banana'], ['banana', 'nuts', 'milk'], ['orange juice', 'milk', 'chocolate']]
>> d = defaultdict(RoaringBitmap)
>> for idx, transaction in enumerate(database):
>>    for item in transaction:
>>       d[item].add(idx)

>> d
defaultdict(roaringbitmap.RoaringBitmap,
            {'milk': RoaringBitmap({0, 1, 2}),
             'chocolate': RoaringBitmap({0, 2}),
             'banana': RoaringBitmap({0, 1}),
             'nuts': RoaringBitmap({1}),
             'orange juice': RoaringBitmap({2})})

Let's imagine I want to know how many times 'milk' and 'chocolate' occurred together :

from functools import reduce
>> keys = ['milk', 'chocolate'] 
>> len(reduce(RoaringBitmap.intersection, (d[k] for k in keys)))
2

As my database definition is static (I ingest data only once and do not modify my data representation), I was hopping to get better performance using MultiRoaringBitmap, computing intersections on subsets of it

[remiadon]

Right on point,
thanks for the support

Closing this