fccd flatfield function doesn't handle the broken part of the camera very well about csxtools

[ambarb]

From @patryk-w

def get_half_fastccd_flatfield(light, darks, flat=None, limits=(0.6, 1.4),
                               module_width=480, right_side=0):
    """Calculate flat field for the working module of the fCCD
    This function calculates the flat field correction array for the fCCD
    detector with only one of the modules working (right module), setting the
    other half to NaNs.
    Parameters:
    -----------
    light (int): transient scan id for the flat field,
    darks (array of ints): transient scan id for the dark images,
    flat (ndarray): flatfield image (optional)
    Returns
    -------
    flat_corrected (ndarray): corrected flat field array for the whole fccd
        detector including only the working module
    """
    light = db[int(light)]
    dark = []
    if darks is not None:
        for d in darks:
            if d is not None:
                dark.append(db[(int(d))])
            else:
                dark.append(None)
    else:
        dark = None
    flat_images = get_images_to_3D(get_fastccd_images(light, dark))
    flat_image = stackmean(flat_images)
    flat = flat_image[:, flat_image.shape[-1] - module_width : flat_image.shape[-1] - right_side]
    limits = np.array(limits)
    limits = np.nanmedian(flat) * limits
    flat[flat < np.min(limits)] = np.nan
    flat[flat > np.max(limits)] = np.nan
    flat = np.nanmean(flat) / flat
    flat_corrected = np.empty_like(flat_image)
    flat_corrected[:, :] = np.nan
    flat_corrected[:, flat_image.shape[-1] - module_width:flat_image.shape[-1] - right_side] = flat
    flat_corrected = np.rot90(flat_corrected)
    removed = np.sum(np.isnan(flat))
    if removed != 0:
        msg = 'Flat field correction removed {} pixels'.format(removed)
        msg += ' ({:.2f}%) out of the working part of the detector'.format(removed * 100 / flat.size)
        print(msg)
    return flat_corrected

[mpmdean]

Hi @ambarb and @patryk-w

Thanks for posting the modified function.

I think what we actually want, however, is an additional argument to get_half_fastccd_flatfield that specifies a minimum_ADUs, below which we consider the fCCD to have not detected any phonons. This can be a variable number, which is set to a default that we consider sensible. If pixels are below this they are set to np.NaN from the outset.

[ambarb]

@mpmdean I think using a threshold makes for the most generic solution. However, we should either edit the original function or make a new function that isn't specified for half of the ccd.

In this way, the new function can be generically applied no matter the mode the ccd is running (framestore, non-framestore, with or without overscan) and is more robust than the original function created many years ago.

[ambarb]

we've noticed that the levels on the broken side change. This makes it a little hard to use a general threshold in an automated way. Lets just get something that works now.

For future threshold, we can now build some historical data for generalized testing of whatever this new function will be. There is a new function that labels the flatfield data appropriately at csx ct_flatfield()

[ambarb]

sophisticated fix is a global issue (half or not, FCCD or not). Making new issue for enhancement request.