Hi,
I am working right now on atmospheric fit from DES Y1 images. By looking PSF’s parameters averages across 23 exposures, I notice (like @cpadavis ), that you can observed a systematic average, different from zeros and that vary across the FoV. Some of those average fluctuation are correlated to some CCD position. Other looks like rings pattern, which I guess is something known from SV and Y1 data and solved for Y3 with better astrometric solution.
Even if those problem disappear in Y3, I expect to have this same kind of problem (average across exposures !=0), and this is an issue for both Gaussian Process algorithms implemented within Piff. Indeed, the assumption of the GP is that the gaussian fluctuation are around an average function that can be a function of the coordinates (which is what I observed on Y1 data). So, not take into account this average will affect, the shape of the spatial correlation, the hyperparameters found and the quality of the interpolation.
This is not a big deal to treat that correctly, it just needs to subtract this average function determine across different exposure on a single exposure before to start the interpolation and to add it back when you get your GP interpolation.
I would like to implement this new features within both GP algorithm that are implemented within Piff. Basically to do that, I want to feed the GP class with a new argument that are a function of the coordinate and which return the average of the PSF’s parameters. However, I know that Piff should work also from a config files so my idea is to do something like that:
Using the dill package (which is pip installable and equivalent to pickle) you can wrap a function/class within a pickle files and after re-used the function by opening the pickle file. An example:
import dill
class average:
def __init__(self, coeff_x, coeff_y):
self.coeff_x = coeff_x
self.coeff_y = coeff_y
def __call__(self, x, y):
return self.coeff_x * x + self.coeff_y * y
mean = average(2, 3)
print mean(5, 2)
dill.dump(mean, open("test.pickle", mode="wb"))
and now from everywhere this is possible to re-used this function with coefficient 2 and 3:
import dill
mean = dill.load(open("test.pickle", mode="rb"))
print mean(5, 2)
I guess with this implementation, you just need to feed the config file with this pickle file and within the GP class, used directly the function implemented within the pickle.
Do you think this is a good API to do it like that?
I am writing a new branch to implement this spatial average define in a pickle file. If you think, it should be done in an other way, comments are welcome. But I am convince at least, GP implementation should care about average across exposures because this is something needed to make GP interpolation work on real data.
Cheers,
PF