Unit Tests about piff

[rmjarvis]

I guess these questions are for @cpadavis. You seem to be talking about tests on the combined_wavefront_psf branch, which seems to be his work...

[areshernandez]

When I ran test_optatmo.py with the new changes to the "create_synthetic_optatmo()" function I got a new error:

File "test_optatmo.py", line 395, in
test_optical_fit()
File "/afs/slac.stanford.edu/u/ec/aresh/Piff-wavefront/tests/piff_test_helper.py", line 78, in f2
result = f(*args, **kwargs)
File "test_optatmo.py", line 365, in test_optical_fit
optpsf.fit(stars_clean, wcs, pointing, logger=logger)
File "/u/ec/aresh/Piff-wavefront/piff/optatmo.py", line 430, in fit
logger=logger)
File "/u/ec/aresh/Piff-wavefront/piff/optatmo.py", line 455, in _minuit_fit
self._minuit.migrad(ncall=self.kwargs['max_iterations'])
File "_libiminuit.pyx", line 331, in iminuit._libiminuit.Minuit.migrad (iminuit/_libiminuit.cpp:5099)
File "/u/ec/aresh/Piff-wavefront/piff/optatmo.py", line 579, in _fit_func
shapes = self._measure_shapes(self.drawStarList(self._fit_stars),)# logger=logger)
File "/u/ec/aresh/Piff-wavefront/piff/optatmo.py", line 634, in drawStarList
stars = self.decaminfo.pixel_to_focalList(stars)
File "/u/ec/aresh/Piff-wavefront/piff/des/decaminfo.py", line 321, in pixel_to_focalList
return [Star(self.pixel_to_focal_stardata(star.data), star.fit) for star in stars]
File "/u/ec/aresh/Piff-wavefront/piff/des/decaminfo.py", line 290, in pixel_to_focal_stardata
stardata['chipnum'], stardata['x'], stardata['y'])
File "/u/ec/aresh/Piff-wavefront/piff/star.py", line 682, in getitem
return self.properties[key]
KeyError: 'chipnum'

Looking in the decaminfo.py file it looks like there is trouble with the pixel to focal x and focal y conversion and that the "chipnum" property is missing. Is this due to the WCS coordinate conversion issue or is it because we are using Simple PSF rather than SingleChip?

[cpadavis]

Neither, actually. If it were that, then the code wouldn't have run even before your changes. It looks like when the atmospheric piece of the combined PSF draws the stars, it doesn't preserve any of the properties in the star data except for x/y and u/v. I have fixed that and pushed it to the combined_wavefront_psf branch. Give it a pull and see if that fixes your problem :)

…

---chris

On Mon, Jul 10, 2017 at 5:35 PM Ares Hernandez ***@***.***> wrote: When I ran test_optatmo.py with the new changes to the "create_synthetic_optatmo()" function I got a new error: File "test_optatmo.py", line 395, in test_optical_fit() File "/afs/ slac.stanford.edu/u/ec/aresh/Piff-wavefront/tests/piff_test_helper.py", line 78, in f2 result = f(*args, **kwargs) File "test_optatmo.py", line 365, in test_optical_fit optpsf.fit(stars_clean, wcs, pointing, logger=logger) File "/u/ec/aresh/Piff-wavefront/piff/optatmo.py", line 430, in fit logger=logger) File "/u/ec/aresh/Piff-wavefront/piff/optatmo.py", line 455, in _minuit_fit self._minuit.migrad(ncall=self.kwargs['max_iterations']) File "_libiminuit.pyx", line 331, in iminuit._libiminuit.Minuit.migrad (iminuit/_libiminuit.cpp:5099) File "/u/ec/aresh/Piff-wavefront/piff/optatmo.py", line 579, in _fit_func shapes = self._measure_shapes(self.drawStarList(self._fit_stars),)# logger=logger) File "/u/ec/aresh/Piff-wavefront/piff/optatmo.py", line 634, in drawStarList stars = self.decaminfo.pixel_to_focalList(stars) File "/u/ec/aresh/Piff-wavefront/piff/des/decaminfo.py", line 321, in pixel_to_focalList return [Star(self.pixel_to_focal_stardata(star.data), star.fit) for star in stars] File "/u/ec/aresh/Piff-wavefront/piff/des/decaminfo.py", line 290, in pixel_to_focal_stardata stardata['chipnum'], stardata['x'], stardata['y']) File "/u/ec/aresh/Piff-wavefront/piff/star.py", line 682, in getitem return self.properties[key] KeyError: 'chipnum' Looking in the decaminfo.py file it looks like there is trouble with the pixel to focal x and focal y conversion and that the "chipnum" property is missing. Is this due to the WCS coordinate conversion issue or is it because we are using Simple PSF rather than SingleChip? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#54 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABsw5MvJdiY5nxrdDfkyFvOJ45ol2AQrks5sMsNigaJpZM4ORngF> .

[areshernandez]

With the new updates, the error is no longer there. However, I now receive a different error that comes from the test_optical_fit() function:

  File "test_optatmo.py", line 395, in <module>
    test_optical_fit()
  File "/afs/slac.stanford.edu/u/ec/aresh/Piff-wavefront/tests/piff_test_helper.py", line 78, in f2
    result = f(*args, **kwargs)
  File "test_optatmo.py", line 378, in test_optical_fit
    assert abs(misalignment[key] - misalignment_fit[key]) < 2 * misalignment_fit_errors[key], '{0} Failed Optical Fit. Parameter: {1}, Fit: {2} +- {3}'.format(key, misalignment[key], misalignment_fit[key], misalignment_fit_errors[key])
AssertionError: z04d Failed Optical Fit. Parameter: -0.5, Fit: -1.99999999998 +- 2.40979307846e-05

This error seems to appear where the misalignment fitting is done. Is this likely due to the problem with the number of parameters used? If so, should I try to remove the z04d parameter that seems to be causing problems? I'm going to attempt that next.

[aaronroodman]

Ares, it looks like the fit is just returning a bad value - should find -0.5 for z04d (thats focus delta) but finds -2. Does the fit seem to run? I believe you should see it looking for its minimum and should see the value of each parameter cycle through reasonable values. Aaron On Jul 11, 2017, at 2:29 PM, Ares Hernandez <[email protected]<mailto:[email protected]>> wrote: With the new updates, the error is no longer there. However, I now receive a different error that comes from the test_optical_fit() function: File "test_optatmo.py", line 395, in <module> test_optical_fit() File "/afs/slac.stanford.edu/u/ec/aresh/Piff-wavefront/tests/piff_test_helper.py<http://slac.stanford.edu/u/ec/aresh/Piff-wavefront/tests/piff_test_helper.py>", line 78, in f2 result = f(*args, **kwargs) File "test_optatmo.py", line 378, in test_optical_fit assert abs(misalignment[key] - misalignment_fit[key]) < 2 * misalignment_fit_errors[key], '{0} Failed Optical Fit. Parameter: {1}, Fit: {2} +- {3}'.format(key, misalignment[key], misalignment_fit[key], misalignment_fit_errors[key]) AssertionError: z04d Failed Optical Fit. Parameter: -0.5, Fit: -1.99999999998 +- 2.40979307846e-05 This error seems to appear where the misalignment fitting is done. Is this likely due to the problem with the number of parameters used? If so, should I try to remove the z04d parameter that seems to be causing problems? I'm going to attempt that next. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub<#54 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AEEa8z5XyNaiHby6sGFeKNXII67ZHjdbks5sM78EgaJpZM4ORngF>.

…

_________________________________________________________________________ Prof. Aaron Roodman Chair Dept. of Particle Physics & Astrophysics SLAC National Accelerator Laboratory Stanford University SLAC National Accelerator Laboratory E-mail: [email protected]<mailto:[email protected]> 2575 Sand Hill Rd. Phone: [phone] MS 29 Menlo Park, CA 94025 URL: http://www.slac.stanford.edu/~roodman

_________________________________________________________________________

[cpadavis]

I think you were adding in the atmospheric PSF, but that test doesn't expect there to be an additional atmospheric piece... On Tue, Jul 11, 2017 at 11:36 AM aaronroodman <[email protected]> wrote:

…

Ares, it looks like the fit is just returning a bad value - should find -0.5 for z04d (thats focus delta) but finds -2. Does the fit seem to run? I believe you should see it looking for its minimum and should see the value of each parameter cycle through reasonable values. Aaron On Jul 11, 2017, at 2:29 PM, Ares Hernandez ***@***.*** ***@***.***>> wrote: With the new updates, the error is no longer there. However, I now receive a different error that comes from the test_optical_fit() function: File "test_optatmo.py", line 395, in <module> test_optical_fit() File "/afs/ slac.stanford.edu/u/ec/aresh/Piff-wavefront/tests/piff_test_helper.py< http://slac.stanford.edu/u/ec/aresh/Piff-wavefront/tests/piff_test_helper.py>", line 78, in f2 result = f(*args, **kwargs) File "test_optatmo.py", line 378, in test_optical_fit assert abs(misalignment[key] - misalignment_fit[key]) < 2 * misalignment_fit_errors[key], '{0} Failed Optical Fit. Parameter: {1}, Fit: {2} +- {3}'.format(key, misalignment[key], misalignment_fit[key], misalignment_fit_errors[key]) AssertionError: z04d Failed Optical Fit. Parameter: -0.5, Fit: -1.99999999998 +- 2.40979307846e-05 This error seems to appear where the misalignment fitting is done. Is this likely due to the problem with the number of parameters used? If so, should I try to remove the z04d parameter that seems to be causing problems? I'm going to attempt that next. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub< #54 (comment)>, or mute the thread< https://github.com/notifications/unsubscribe-auth/AEEa8z5XyNaiHby6sGFeKNXII67ZHjdbks5sM78EgaJpZM4ORngF>. _________________________________________________________________________ Prof. Aaron Roodman Chair Dept. of Particle Physics & Astrophysics SLAC National Accelerator Laboratory Stanford University SLAC National Accelerator Laboratory E-mail: ***@***.*** ***@***.***> 2575 Sand Hill Rd. Phone: [phone] <(650)%20926-2705> MS 29 Menlo Park, CA 94025 URL: http://www.slac.stanford.edu/~roodman _________________________________________________________________________ — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#54 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABsw5KM9rJ4Eud7Ac_eQfPiHqKNAxH9oks5sM8DIgaJpZM4ORngF> .

[areshernandez]

After test_optatmo.py finished running, it went through test_optical_fit() and did not cause any problems like usual. However, when it got to the optical test in test_fit() it failed in the same way that test_optical_fit() used to fail (which was due to feeding optatmo_stars_clean from the full PSF into the fit() function inside the OpticalWavefrontPSF(PSF) class instead of just opt_stars_clean):

  File "test_optatmo.py", line 483, in <module>
    
  File "/afs/slac.stanford.edu/u/ec/aresh/Piff-wavefront/tests/piff_test_helper.py", line 78, in f2
    result = f(*args, **kwargs)
  File "test_optatmo.py", line 373, in test_fit
    # construct results
AssertionError: z04d Failed Optical Fit inside full test_fit() function. Parameter: -0.5, Fit: -1.99999999996 +- 1.92922281265e-05

In test_fit(), however, I fed optatmo_stars_clean into the fit() function inside the OptAtmoPSF(PSF) class, rather than the fit() function inside the OpticalWavefrontPSF(PSF) class. Is there likely a bug interfering with the fit() function inside the OptAtmoPSF(PSF) class that is causing it to behave like the fit() function from the OpticalWavefrontPSF(PSF) class?

[cpadavis]

So it might be what we talked about earlier today -- there might not be enough stars sampling the focal plane distribution, and so a parameter degeneracy develops. But: another thing might be that we aren't allowing the model the ability to compensate for the atmosphere during the fit. The thinking is this: when you make the opt + atmo stars, you are including the atmospheric piece in making the stars. Here, this means adding a constant atmosphere and ellipticity, or possibly a 1st order polynomial across the focal plane. But, looking at your code, the optical piece has no way of fitting that (since we said it must have a fixed r0 == atmospheric seeing == affects the size of the stars), so it leans heavy on the defocus to get the larger stars. The solution is to allow r0, g1, and g2 to be fitted in the optical part, but then "turn them off" and let the atmosphere fit that piece later. I thought this was what the model was doing, but I might not have done a good job coding that up. I'll take a look at it this weekend, and we can chat on Monday about this idea.

…

---chris

On Fri, Jul 14, 2017 at 5:47 PM Ares Hernandez ***@***.***> wrote: After test_optatmo.py finished running, it went through test_optical_fit() and did not cause any problems like usual. However, when it got to the optical test in test_fit() it failed in the same way that test_optical_fit() used to fail (which was due to feeding optatmo_stars_clean from the full PSF into the fit() function inside the OpticalWavefrontPSF(PSF) class instead of just opt_stars_clean): File "test_optatmo.py", line 483, in <module> File "/afs/slac.stanford.edu/u/ec/aresh/Piff-wavefront/tests/piff_test_helper.py", line 78, in f2 result = f(*args, **kwargs) File "test_optatmo.py", line 373, in test_fit # construct results AssertionError: z04d Failed Optical Fit inside full test_fit() function. Parameter: -0.5, Fit: -1.99999999996 +- 1.92922281265e-05 In test_fit(), however, I fed optatmo_stars_clean into the fit() function inside the OptAtmoPSF(PSF) class, rather than the fit() function inside the OpticalWavefrontPSF(PSF) class. Is there likely a bug interfering with the fit() function inside the OptAtmoPSF(PSF) class that is causing it to behave like the fit() function from the OpticalWavefrontPSF(PSF) class? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#54 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABsw5JKkStxjZThNRz9LzeYGbhBhPwvcks5sOAw_gaJpZM4ORngF> .

[areshernandez]

I have narrowed down that the reason why the graphs for fitted_opt_stars are empty is because something goes wrong in the second line below:

opt_stars_drawn = optpsf.drawStarList(opt_stars_clean)
fitted_opt_shapes = optpsf._measure_shapes(stars=opt_stars_drawn)

I have successfully printed optpsf._measure_shapes(stars=opt_stars_clean), so I know there is nothing wrong with opt_stars_clean. I have printed the focal_x and focal_y coordinates of opt_stars_drawn, so I don't think there is anything wrong with opt_stars_drawn either. However, fitted_opt_shapes is a large list filled with "nan" when I print it.

It thus looks like there may be a bug inside of _measure_shapes() that only happens when I feed it opt_stars_drawn. Do you think this is the case? If so, is there an alternative method of retrieving the shapes besides _measure_shapes() that I can use?

[cpadavis]

Have you looked at the drawn stars? You can get the image for the i-th star via opt_stars_drawn[i].image.array and then you can plot the image with matplotlib imshow.

…

On Wed, Jul 19, 2017 at 16:36 Ares Hernandez ***@***.***> wrote: I have narrowed down that the reason why the graphs for fitted_opt_stars are empty is because something goes wrong in the second line below: opt_stars_drawn = optpsf.drawStarList(opt_stars_clean) fitted_opt_shapes = optpsf._measure_shapes(stars=opt_stars_drawn) I have successfully printed optpsf._measure_shapes(stars=opt_stars_clean), so I know there is nothing wrong with opt_stars_clean. I have printed the focal_x and focal_y coordinates of opt_stars_drawn, so I don't think there is anything wrong with opt_stars_drawn either. However, fitted_opt_shapes is a large list filled with "nan" when I print it. It thus looks like there may be a bug inside of _measure_shapes() that only happens when I feed it opt_stars_drawn. Do you think this is the case? If so, is there an alternative method of retrieving the shapes besides _measure_shapes() that I can use? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#54 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABsw5AeCsCmZMuwjVdaxS38N9myRyZe5ks5sPpLkgaJpZM4ORngF> .

[areshernandez]

I see a small yellow square inside a purple background. Does this mean the images from opt_stars_drawn are corrupted somehow? In other words, are the star image normally large yellow ellipses in a purple background? If so, could this indicate that there is a bug inside of drawStarList() causing this problem?

[cpadavis]

hmmm I would say something is going wrong :) This is in test_fit, right? where optpsf is the optical piece of the combined psf fit, right? Remember how, after the fit, we turn off the optical psf's constant size and ellipticity portions (since that will be covered by the atmospheric psf)? It might be when you are making those stars using only the optical psf that that piece is turned off. So since r0 is turned off, then the psf never gets scattered by the atmosphere, and you get effectively space quality images == basically only in one pixel. This means you should do two things: - see if you can turn that bit back on for the purposes of testing. I'm pretty sure the fitted r0, g1, g2 live in optosf._fitter_kwargs, so you can use that to put them back in. - double check that when you generated the stars from the original combined psf you weren't folding in the atmosphere as well. Here's an easy way to do that: generate 2 combined psfs with the same optical psf parameters but different atmosphere coeffs, and use the same star coordinates for both. Do your optical psf stars look the same or different?

…

On Wed, Jul 19, 2017 at 5:23 PM Ares Hernandez ***@***.***> wrote: I see a small yellow square inside a purple background. Does this mean the images from opt_stars_drawn are corrupted somehow? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#54 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABsw5AxXWKBu3Iot7KFzU2jxfspykjnkks5sPp3xgaJpZM4ORngF> .

[areshernandez]

I tried putting r0, g1, and g2 back in before drawing the stars:

    optpsf.fitter_kwargs['fix_r0'] = False
    optpsf.fitter_kwargs['fix_g1'] = False
    optpsf.fitter_kwargs['fix_g2'] = False
    optpsf.model.kolmogorov_kwargs = {'r0':optpsf.backup_shapes[0]}
    optpsf.model.g1 = optpsf.backup_shapes[1]
    optpsf.model.g2 = optpsf.backup_shapes[2]
    optpsf._update_psf_params(r0=optpsf.backup_shapes[0], g1=optpsf.backup_shapes[1], g2=optpsf.backup_shapes[2])
    opt_stars_drawn = optpsf.drawStarList(opt_stars_clean)
    optpsf._update_psf_params(r0=None, g1=None, g2=None)
    optpsf.fitter_kwargs['fix_r0'] = True
    optpsf.fitter_kwargs['fix_g1'] = True
    optpsf.fitter_kwargs['fix_g2'] = True
    optpsf.model.kolmogorov_kwargs = {}
    optpsf.model.g1 = None
    optpsf.model.g2 = None
    fitted_opt_shapes = optpsf._measure_shapes(stars=opt_stars_drawn)

However, after doing this I got a new error:

  File "test_optatmo.py", line 889, in <module>
    test_fit()
  File "/afs/slac.stanford.edu/u/ec/aresh/Piff-wavefront/tests/piff_test_helper.py", line 78, in f2
    result = f(*args, **kwargs)
  File "test_optatmo.py", line 593, in test_fit
    opt_stars_drawn = optpsf.drawStarList(opt_stars_clean)
  File "/u/ec/aresh/Piff-wavefront/piff/optatmo.py", line 644, in drawStarList
    stars = [self.model.draw(star) for star in stars]
  File "/u/ec/aresh/Piff-wavefront/piff/optical_model.py", line 258, in draw
    prof = self.getProfile(star.fit.params)
  File "/u/ec/aresh/Piff-wavefront/piff/optical_model.py", line 227, in getProfile
    atm = galsim.Kolmogorov(gsparams=self.gsparams, **self.kolmogorov_kwargs)
  File "/nfs/slac/g/ki/ki18/cpd/anaconda2/lib/python2.7/site-packages/galsim/base.py", line 593, in __init__
    "r0_500) must be specified for Kolmogorov")
TypeError: One of lam_over_r0, fwhm, half_light_radius, or lam (with r0 or r0_500) must be specified for Kolmogorov

It seems that I'm missing something called "r0_500," which I need in addition to r0, g1, and g2 in order to use optpsf.drawStarList(). If this is the case, is there a specific location where this "r0_500" value is stored?

I should add that I've checked optpsf.backup_shapes[0], optpsf.backup_shapes[1], and optpsf.backup_shapes[2] and compared them to optpsf.model.kolmogorov_kwargs['r0'], optpsf.model.g1, and optpsf.model.g2 and can confirm that they are the same, as they should be.

[cpadavis]

looking at it a little closer, you'll want to do this: kolmogorov_keys = ('lam', 'r0', 'lam_over_r0', 'scale_unit', 'fwhm', 'half_light_radius', 'r0_500') optpsf.model.kolmogorov_kwargs = { key : optpsf.model.kwargs[key] for key in optpsf.model.kwargs if key in kolmogorov_keys } optpsf.model.kolmogorov_kwargs['r0'] = optpsf.backup_shapes[0] if we constantly have to turn on and off the kolmogorov piece, this could be made into a function.

…

On Thu, Jul 20, 2017 at 12:25 PM Ares Hernandez ***@***.***> wrote: I tried putting r0, g1, and g2 back in before drawing the stars: optpsf.fitter_kwargs['fix_r0'] = False optpsf.fitter_kwargs['fix_g1'] = False optpsf.fitter_kwargs['fix_g2'] = False optpsf.model.kolmogorov_kwargs = {'r0':optpsf.backup_shapes[0]} optpsf.model.g1 = optpsf.backup_shapes[1] optpsf.model.g2 = optpsf.backup_shapes[2] optpsf._update_psf_params(r0=optpsf.backup_shapes[0], g1=optpsf.backup_shapes[1], g2=optpsf.backup_shapes[2]) opt_stars_drawn = optpsf.drawStarList(opt_stars_clean) optpsf._update_psf_params(r0=None, g1=None, g2=None) optpsf.fitter_kwargs['fix_r0'] = True optpsf.fitter_kwargs['fix_g1'] = True optpsf.fitter_kwargs['fix_g2'] = True optpsf.model.kolmogorov_kwargs = {} optpsf.model.g1 = None optpsf.model.g2 = None fitted_opt_shapes = optpsf._measure_shapes(stars=opt_stars_drawn) However, after doing this I got a new error: File "test_optatmo.py", line 889, in <module> test_fit() File "/afs/slac.stanford.edu/u/ec/aresh/Piff-wavefront/tests/piff_test_helper.py", line 78, in f2 result = f(*args, **kwargs) File "test_optatmo.py", line 593, in test_fit opt_stars_drawn = optpsf.drawStarList(opt_stars_clean) File "/u/ec/aresh/Piff-wavefront/piff/optatmo.py", line 644, in drawStarList stars = [self.model.draw(star) for star in stars] File "/u/ec/aresh/Piff-wavefront/piff/optical_model.py", line 258, in draw prof = self.getProfile(star.fit.params) File "/u/ec/aresh/Piff-wavefront/piff/optical_model.py", line 227, in getProfile atm = galsim.Kolmogorov(gsparams=self.gsparams, **self.kolmogorov_kwargs) File "/nfs/slac/g/ki/ki18/cpd/anaconda2/lib/python2.7/site-packages/galsim/base.py", line 593, in __init__ "r0_500) must be specified for Kolmogorov") TypeError: One of lam_over_r0, fwhm, half_light_radius, or lam (with r0 or r0_500) must be specified for Kolmogorov It seems that I'm missing something called "r0_500," which I need in addition to r0, g1, and g2 in order to use optpsf.drawStarList(). If this is the case, is there a specific location where this "r0_500" value is stored? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#54 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABsw5M1Z7dVPU7nLMgjVvalmvXAy5Yenks5sP6mOgaJpZM4ORngF> .

[areshernandez]

I have tried the following input misalignment terms to be used for test_fit():

1. z04d=-0.5, z05d=0.1
2. z04d=-0.5, z05d=0.1, z06d=0.06
3. z04d=-0.5, z07d=0.01, z08d=0.03
4. z04d=-0.5, z08d=0.03
5. z04d=-0.5, z09d=0.1, z10d=0.1

So far, non have produced a good fit. Are there any possible sets of inputs that I haven't tried that you think would be more likely to succeed than others?

[cpadavis]

Hmm. I am honestly not sure what the best inputs are. How bad are the fits? How many stars are you using? Do the fits converge (as in, do they finish before they run out of available steps)? What do the residuals look like? It's not the end of the world if there is a degeneracy between our input optical model and the final atmosphere -- it just means that that is something we shouldn't test. Can you make plots of the following: scans of the parameters vs chi2 in the neighborhood of the fitted value and the input value. Taking (1) as an example, say you fit for z04d and z05d -1 and 0.5. Then there would be one plot where you show the chi2 as a function of dz about the two fits of z04d, say dz from -0.2 to 0.2. And another one around z05d. Does that make sense? They should look like parabolas, probably with the incorrect value at a higher y intercept.

…

---chris

On Thu, Jul 27, 2017 at 12:09 PM Ares Hernandez ***@***.***> wrote: I have tried the following input misalignment terms to be used for test_fit(): 1. z04d=-0.5, z05d=0.1 2. z04d=-0.5, z05d=0.1, z06d=0.06 3. z04d=-0.5, z07d=0.01, z08d=0.03 4. z04d=-0.5, z08d=0.03 5. z04d=-0.5, z09d=0.1, z10d=0.1 So far, non have produced a good fit. Are there any possible sets of inputs that I haven't tried that you think would be more likely to succeed than others? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#54 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABsw5ESGceWKbD2v2IkFrM6hlTWa5_Psks5sSOBNgaJpZM4ORngF> .

[areshernandez]

I'm not sure but I think you want me to make a graph of:

the fitted values I get for z04d vs the misalignment_fit_errors['z04d'] values that I get for the fits

Then I should expect this graph to be a parabola centered around z04d=-0.5 (the input value I always used for z04d). The graph should have a misalignment_fit_errors['z04d'] value of 0 there that increases parabolically as you move away from z04d=-0.5. Finally, I do the exact same thing but for z05d for all of the instances where I tested z05d.

Is this correct?

[areshernandez]

On a separate note. I have also tried the following:

6. z04d=-0.5, z05d=0.1, z08d=0.03
7. z04d=-0.5, z06d=0.06, z07d=0.01
8. z04d=-0.5, z08d=0.03, z09d=0.1
9. z04d=-0.5, z07d=0.01, z10d=0.1
10. z04d=-0.5, z10d=0.1
11. z04d=-0.5, z09d=0.1
12. z04d=-0.5, z07d=0.01
13. z04d=-0.5, z06d=0.06
14. z04d=-0.5, z05d=0.1, z10d=0.1
15. z04d=-0.5, z06d=0.06, z09d=0.1
16. z04d=-0.5, z06d=0.06, z08d=0.03
17. z04d=-0.5, z05d=0.1, z07d=0.01
18. z04d=-0.5, z07d=0.01, z09d=0.1
19. z04d=-0.5, z08d=0.03, z10d=0.1
20. z04d=-0.5, z05d=0.1, z09d=0.1
21. z04d=-0.5, z06d=0.06, z10d=0.1

None of these have produced good fits either. They all seem to be pretty bad fits just like (1). For example, for (21) the fit gives z04d=-0.01826, whereas for (1) the fit gives z04d=-0.1579, even though both are supposed to give -0.5

[cpadavis]

Sorry, let me try to be a bit clearer: In OpticalWavefrontPSF there is a method called _fit_func. It takes in all the misalignment and shape parameters (r0, z04d, etc) and returns a chi squared. I want you to vary the value of your fit parameters near their input values, so if z04d = -0.5, I want to know the value returned by optpsf._fit_func for like 30 values of z04d from -0.7 to -0.2 -- call it -0.5 + dz, with dz = np.linspace(-0.2, 0.2, 30) If you also have a suboptimal fit result, I want a similar +- 0.2 range of values. Then I want you to plot the chi2 vs dz. For parameters you don't want to modify, you can pass in np.nan and they will remain unchanged. Does that make sense?

…

On Thu, Jul 27, 2017 at 3:53 PM Ares Hernandez ***@***.***> wrote: On a separate note. I have also tried the following: 1. z04d=-0.5, z05d=0.1, z08d=0.03 2. z04d=-0.5, z06d=0.06, z07d=0.01 3. z04d=-0.5, z08d=0.03, z09d=0.1 4. z04d=-0.5, z07d=0.01, z10d=0.1 5. z04d=-0.5, z10d=0.1 6. z04d=-0.5, z09d=0.1 7. z04d=-0.5, z07d=0.01 8. z04d=-0.5, z06d=0.06 9. z04d=-0.5, z05d=0.1, z10d=0.1 10. z04d=-0.5, z06d=0.06, z09d=0.1 11. z04d=-0.5, z06d=0.06, z08d=0.03 12. z04d=-0.5, z05d=0.1, z07d=0.01 13. z04d=-0.5, z07d=0.01, z09d=0.1 14. z04d=-0.5, z08d=0.03, z10d=0.1 15. z04d=-0.5, z05d=0.1, z09d=0.1 16. z04d=-0.5, z06d=0.06, z10d=0.1 None of these have produced good fits either. They all seem to be pretty bad fits just like (1). For example, for (21) the fit gives z04d=-0.01826, whereas for (1) the fit gives z04d=-0.1579, even though both are supposed to give -0.5 — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#54 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABsw5HWvPiC85Lin4UrOQOZ6Pi-9IitVks5sSRTXgaJpZM4ORngF> .

[areshernandez]

The chi-squared graph for astigmatism-y also does not have a minimum at either the input value or the fitted value. The minimum is to the left of both, just like what was the case with defocus. Unfortunately, the chi-squared graphs for the temporary atmosphere parameters (size, first ellipticity, second ellipticity) failed because of a runtime error:

  File "test_optatmo.py", line 1289, in <module>
    test_fit()
  File "/afs/slac.stanford.edu/u/ec/aresh/Piff-wavefront/tests/piff_test_helper.py", line 78, in f2
    result = f(*args, **kwargs)
  File "test_optatmo.py", line 1109, in test_fit
    chi_squareds_list.append(optpsf._fit_func(r0=0.1+dz_size_due_to_temporary_atmosphere[i], g1=misalignment['g1'], g2=misalignment['g2'], z04d=misalignment['z04d'], z04x=np.nan, z04y=np.nan, z05d=misalignment['z05d'], z05x=np.nan, z05y=np.nan, z06d=np.nan, z06x=np.nan, z06y=np.nan, z07d=np.nan, z07x=np.nan, z07y=np.nan, z08d=np.nan, z08x=np.nan, z08y=np.nan, z09d=np.nan, z09x=np.nan, z09y=np.nan, z10d=np.nan, z10x=np.nan, z10y=np.nan, z11d=np.nan, z11x=np.nan, z11y=np.nan))
  File "/u/ec/aresh/Piff-wavefront/piff/optatmo.py", line 593, in _fit_func
    shapes = self._measure_shapes(self.drawStarList(self._fit_stars),)# logger=logger)
  File "/u/ec/aresh/Piff-wavefront/piff/optatmo.py", line 654, in drawStarList
    stars = [self.model.draw(star) for star in stars]
  File "/u/ec/aresh/Piff-wavefront/piff/optical_model.py", line 261, in draw
    image = prof.drawImage(star.data.image.copy(), method='no_pixel', offset=offset)
  File "/nfs/slac/g/ki/ki18/cpd/anaconda2/lib/python2.7/site-packages/galsim/gsobject.py", line 1356, in drawImage
    added_photons = prof.drawFFT(imview, wmult)
  File "/nfs/slac/g/ki/ki18/cpd/anaconda2/lib/python2.7/site-packages/galsim/gsobject.py", line 1467, in drawFFT
    "If you can handle the large FFT, you may update gsparams.maximum_fft_size.")
RuntimeError: drawFFT requires an FFT that is too large: 7516.If you can handle the large FFT, you may update gsparams.maximum_fft_size.

Do you know what could be causing this runtime error? More importantly, do you know what the minimum in the chi-squared graphs being in the wrong place could imply?

[cpadavis]

So for the runtime error -- what parameters are you putting in? Be aware that r0 ranges from 0.1 to like 0.3 and if you put in a negative value or a value outside that range then it might run out of memory doing the Fourier transform. The chi square is worrisome, so I want to check something. When you vary a parameter about its fitted value, are you putting in for the other parameters their fitted values as well? And when you vary a parameter about its input value, are you setting the other parameters to their input value? The thing we're trying to figure out is whether the fitted location or the input locations are coming out to be minima in our chi square. Let me know if you have more questions.

…

On Tue, Aug 1, 2017 at 11:32 Ares Hernandez ***@***.***> wrote: The chi-squared graph for astigmatism-y also does not have a minimum at either the input value or the fitted value. The minimum is to the left of both, just like what was the case with defocus. Unfortunately, the chi-squared graphs for the temporary atmosphere parameters (size, first ellipticity, second ellipticity) failed because of a runtime error: File "test_optatmo.py", line 1289, in test_fit() File "/afs/ slac.stanford.edu/u/ec/aresh/Piff-wavefront/tests/piff_test_helper.py", line 78, in f2 result = f(*args, **kwargs) File "test_optatmo.py", line 1109, in test_fit chi_squareds_list.append(optpsf._fit_func(r0=0.1+dz_size_due_to_temporary_atmosphere[i], g1=misalignment['g1'], g2=misalignment['g2'], z04d=misalignment['z04d'], z04x=np.nan, z04y=np.nan, z05d=misalignment['z05d'], z05x=np.nan, z05y=np.nan, z06d=np.nan, z06x=np.nan, z06y=np.nan, z07d=np.nan, z07x=np.nan, z07y=np.nan, z08d=np.nan, z08x=np.nan, z08y=np.nan, z09d=np.nan, z09x=np.nan, z09y=np.nan, z10d=np.nan, z10x=np.nan, z10y=np.nan, z11d=np.nan, z11x=np.nan, z11y=np.nan)) File "/u/ec/aresh/Piff-wavefront/piff/optatmo.py", line 593, in _fit_func shapes = self._measure_shapes(self.drawStarList(self._fit_stars),)# logger=logger) File "/u/ec/aresh/Piff-wavefront/piff/optatmo.py", line 654, in drawStarList stars = [self.model.draw(star) for star in stars] File "/u/ec/aresh/Piff-wavefront/piff/optical_model.py", line 261, in draw image = prof.drawImage(star.data.image.copy(), method='no_pixel', offset=offset) File "/nfs/slac/g/ki/ki18/cpd/anaconda2/lib/python2.7/site-packages/galsim/gsobject.py", line 1356, in drawImage added_photons = prof.drawFFT(imview, wmult) File "/nfs/slac/g/ki/ki18/cpd/anaconda2/lib/python2.7/site-packages/galsim/gsobject.py", line 1467, in drawFFT "If you can handle the large FFT, you may update gsparams.maximum_fft_size.") RuntimeError: drawFFT requires an FFT that is too large: 7516.If you can handle the large FFT, you may update gsparams.maximum_fft_size. Do you know what could be causing this runtime error? More importantly, do you know what the minimum in the chi-squared graphs being in the wrong place could imply? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#54 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABsw5Ju-dHcettKwaRxewbt3j6q6Z64Iks5sT280gaJpZM4ORngF> .

[areshernandez]

I changed the chi-squared graphs to make sure the input graphs used input parameters and the fitted graphs used fitted parameters. Most of the chi-squared graphs look parabolic now.

However, some appear off-center (the minimum looks like it's not at correct point). Also, the astigmatism-y graphs look like they might be quartic and the defocus graphs look like they might be a high-order polynomial.

Do you think this is likely due to the range of dz's tested being too large for the defocus and astigmatism-y graphs (I expanded the range to much more than -0.2 to 0.2 for those graphs)? Also, is there a reason why some of the chi-squared graphs might be off-center?

[areshernandez]

I tried Aaron's technique of keeping the temporary atmosphere without disabling it while keeping it at a fixed value; however, it did not fit. I am wondering if the problem is that the stars used to do all the fitting are the optatmo stars, so when the optical fit is attempting it fits to stars that were created with stuff not generated by the input optical terms (i.e. the atmospheric inputs), it doesn't fit correctly. Do you think this is likely the problem? If so, do you think I should change the OptAtmoPSF.fit() function, so that it does the optical fit with the opt stars first before fitting everything else with the optatmo stars?

[areshernandez]

I re-ran the fit using create_data() instead of create_synthetic_optatmo. It still hasn't finished fitting but it looks like it is converging on the wrong values for defocus and astigmatism-y. I'm not sure, but looking at create_data(), it looks like both psf and psf_clean have the temporary atmosphere disabled, even though it is supposed to be enabled for psf_clean (which is what I am using in the fit). Is there something inside optpsf.disable_atmosphere() that allows it to recognize that psf_clean should not have its temporary atmosphere wiped or should I comment out the line that seems to wipe it?:

psf_clean.optpsf.disable_atmosphere(logger=logger)

[cpadavis]

You should remove that line for psf_clean :)

…

On Fri, Aug 11, 2017 at 18:06 Ares Hernandez ***@***.***> wrote: I re-ran the fit using create_data() instead of create_synthetic_optatmo. It still hasn't finished fitting but it looks like it is converging on the wrong values for defocus and astigmatism-y. I'm not sure, but looking at create_data(), it looks like both psf and psf_clean have the temporary atmosphere disabled, even though it is supposed to be enabled for psf_clean (which is what I am using in the fit). Is there something inside optpsf.disable_atmosphere() that allows it to recognize that psf_clean should not have its temporary atmosphere wiped or should I comment out the line that seems to wipe it?: psf_clean.optpsf.disable_atmosphere(logger=logger) — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#54 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABsw5OoiLLYWCvVDKZXnSTSHr1VX6SPsks5sXPqwgaJpZM4ORngF> .