[Bug]: High memory consumption spikes in ht.array method about heat

[mrfh92]

So far, the experiments show the split option that anyway assumes that the entire array fits into memory. Do the same spikes also occure for is_split? (In my opinion this would be a larger problem because in the is_split option one might work with arrays that are that large that memory really can become a problem easily)

[github-actions]

Branch bugs/1412-_Bug_High_memory_consumption_spikes_in_ht_array_method created!

[JuanPedroGHM]

Did some further experimenting, found out that the spikes appear because I was printing the first element of each array after creating/copying the DNDArray.

I will close the issue now, but leave the script for future reference:

import heat as ht
import time
import argparse

import numpy as np
import torch as pt
import perun
import time

matrix_size = 750

factories = {
    "heat": {
         "zeros": ht.zeros,
         "ones": ht.ones,
         "random": ht.random.random
    },
    "torch": {
        "zeros": pt.zeros,
        "ones": pt.ones,
        "random": pt.rand
    },
    "numpy": {
        "zeros": np.zeros,
        "ones": np.ones,
        "random": np.random.random
    }
}

if __name__ == "__main__":
    # Application should accept the following arguments: device, split, copy, factory, source

    # Parse arguments
    parser = argparse.ArgumentParser(description='Memory test')
    parser.add_argument('--device', type=str, default='cpu', help='Device to use [cpu, gpu]')
    parser.add_argument('--split', type=int, default=-1, help='Split [None, 0, 1, 2]')
    parser.add_argument('--copy', type=str, default="false", help='Copy [false, true]')
    parser.add_argument('--factory', type=str, default='zeros', help='Factory [zeros, ones, random]')
    parser.add_argument('--source', type=str, default='numpy', help='Source, [numpy, torch, heat]')


    args = parser.parse_args()
    print(args)
    
    device = args.device
    split = args.split if args.split != -1 else None
    copy = True if args.copy == "true" else False
    factory = args.factory
    source = args.source

    if device == "cpu":
        pt.set_default_device("cpu")
        ht.use_device("cpu")
    elif device == "gpu":
        pt.set_default_device("cuda")
        ht.use_device("gpu")

    print(f"{ht.comm.rank}: Start")
    start = time.time()
    time.sleep(2)
    
    print(f"{ht.comm.rank}: Create: {time.time() - start}")
    if source == "heat":
        original = factories[source][factory]((matrix_size, matrix_size, matrix_size), split=None, dtype=ht.float32)
    elif source == "torch":
        original = factories[source][factory]((matrix_size, matrix_size, matrix_size), dtype=pt.float32)
    elif source == "numpy":
        original = factories[source][factory]((matrix_size, matrix_size, matrix_size)).astype(np.float32)

    time.sleep(2)

    print(f"{ht.comm.rank}: Copy: {time.time() - start}")
    copy = ht.array(original, split=split, copy=copy)
    copy += 1
    print(f"{ht.comm.rank}: Copy done {time.time() - start}")
    print(original[0,0,0])
    time.sleep(2)

    print(f"{ht.comm.rank}: Delete: {time.time() - start}")
    del original
    del copy
    pt.cuda.empty_cache()
    time.sleep(2)
    print(f"{ht.comm.rank}: End: {time.time() - start}")