Update to current DESI-doc versions about desimodel

[dkirkby]

Following the instructions in How to update inputs from DocDB, the relevant inputs are:

DESI-nnn	DESIMODEL	Latest in DocDB	Updater
DESI-530	v13	v14	desimodel.inputs.gfa.build_gfa_table()
	v11	v14	desimodel.inputs.fiberpos.update()
DESI-2721	v2	v2	desimodel.inputs.fiberpos.update()
DESI-329	v15	v15	desimodel.inputs.fiberpos.update()
DESI-347	v12	v13	desimodel.inputs.throughput.update()
DESI-334	v3	v3	desimodel.inputs.throughput.update()

Since DESI-530 is also out of date (using 2 different old versions!) I will update that also.

[dkirkby]

The build_gfa_table docstring says:

    Builds the GFA table given the data from DESI-0530-v13 Excel spreadsheet
    and writes a .ecsv file using the astropy table library. The data is pulled from
    the "GFALocation" tab on the spreadsheet and from rows 16-23 and columns A-I.

However, these cell references look like they describe v14 not v13. Also, the values in these cells:

    # Data obtained from DESI-0530-v13 Excel spreadsheet
    x = [-125.10482863, -129.83038525, -159.04283509, -154.31646944]
    y = [-370.01790486, -384.56223777, -375.05643893, -360.51151824]
    z = [-17.053, -18.487, -18.631, -17.198]

do not match those in the spreadsheet:

active area corner	X	Y	Z
N0 (ctr)	333.738	217.766	-20.055
N1	318.529	225.702	-19.266
N2	330.901	234.691	-20.699
N3	348.947	209.830	-20.844
N4	336.574	200.841	-19.410

Since these numbers have not changed between v13 and v14, it looks like gfa.ecsv does not need updating, but the docs should be updated. Can you confirm @sbailey?

[dkirkby]

The fiberpos.update() function needs to updated for a change in structure to the PositionerAndFiducialLocations tab of DESI-530-v14 but it looks like the numbers have not changed.

[dkirkby]

Re-running throughput.update() picks up changes from v12 to v13, as expected:

Component	Change
Telescope to fiber input	total corrector coating throughput (row 10)
Fiber FRD	none
Fiber other	none
Spectrographs	none

Note that throughput.update() divides out the DESI-347 spectrograph throughput and replaces it with the DESI-334 curves, presumably because of their higher resolution (1nm grid).

Since DESI-334 has not been updated since July 2014, I have pinged Pat and @julienguy to see if there are "as-built" curves we should be using instead.

The corrector throughput changes are from updates to C2 (both surfaces) and inclusion of the 4 ADC surfaces (which were ignored before).

[dkirkby]

The throughput changes from v12 to v13 are mostly at the negative few percent level and largest in the NIR band:

The effective bandwidth (integrated throughput) has decreased by ~20, 25, 40A in the b,r,z cameras:

version	b total	r total	z total
v13	852.6A	821.5A	1022.2A
v12	872.0A	846.2A	1062.9A

[dkirkby]

Other changes that affect our simulations are:

• Blur (row 27 of Throughput tab) is a bit smaller, e.g. 13.77um to 12.86um at 500nm:
  • "Manufacturing and alignment error (1-d)" (row 14) has decreased from 7.629um to 4.728um.
  • "M1/Barrel GFA error static defocus (1-d)" (row 18) has increased from 0.667um to 3.990um.
  • "Blur due to total fiber defocus" (row 26) has decreased from 4.035um to 3.544um.
• CCD noise and dark current on Noise tab:
  • blue read noise increased from 3.0e/pix to 3.41e/pix
  • red & NIR read noise increased from 2.0e/pix to 2.6e/pix
  • blue dark current decreased from 10 to 2 e/pix/hr
  • read & NIR dark current decreased from 7.3 to 2.4 e/pix/hr

The blur changes need to be propagated through to DESI-0347_blur.ecsv as described in this notebook.

The noise changes need to be propagated through to desi.yaml, which currently does not agree with either v12 or v13 (!)

[dkirkby]

The file DESI-0347_blur.ecsv uses the achromatic blur contributions from DESI-347, which has the three changes listed above. The net result is that the achromatic blur has decreased from 9.206um to 7.789um. This validation plot was produced by an updated version of the DESI-0347_Throughput notebook:

The main points are that:

• The blur has decreased from v10 to v13, due to a net decrease in the achromatic terms.
• The offset has not changed from v10 to v13.
• The v13 field-averaged (wlen, angle) quantities (curves) are in reasonable agreement with the DESI-347 wlen-only quantities (dots).

[dkirkby]

The desi.yaml noise values are confusing since I cannot explain the values currently in the file, which originated in DESI-347-v4 (which had no Noise tab). In any case, I will update to the latest (v13) values.

Noise Source	Band	yaml v4	yaml v12	noise tab v12	noise tab v13
Readout	b	2.5	3.0	3.0	3.41
	rz	2.9	2.9	2.0	2.6
Dark current	b	3.0	3.0	10	2.0
	rz	2.0	2.0	7.3	2.4

UPDATE: A comment in the yaml file says that the readnoise is taken from the CDR spec (although I can't find those numbers there).

[dkirkby]

Here is a summary of the files I updated in svn under $DESIMODEL/data:

File	Comment	Revision
throughput/thru-[brz].fits	new corrector coatings	118094
throughput/DESI-0347_blur.ecsv	new achromatic blurs	118095
desi.yaml	new read noise and dark currents	118096
focalplane/gfa.ecsv	replace RADIUS_MM with S	118108
throughput/DESI-0347_static_[123].fits	Replace random offset files (RMS=10.886um) with static offset files (RMS=8.0um)	118136

Note that I left the original date (29 Jan 2016) in the desi.yaml header since I have not gone through and systematically checked that everything there is current.

UPDATE: Committed gfa.ecsv after @sbailey's updates.

UPDATE: Replace random with static offset files.

[sbailey]

@dkirkby thanks for this systematic review and update.

Regarding the GFA locations, I confirm that the update_inputs.rst and desimodel.inputs.gfa.build_gfa_table docstring description are out of date, referencing the wrong cells. This is further confused by DESI-0530 describing the GFA location of petal 3, while the numbers in desimodel.inputs.gfa.build_gfa_table are transformed to petal 0, and then re-rotated to all of the petals to produce data/focalplane/gfa.ecsv. i.e. the PETAL=3 corners in gfa.ecsv do match the spreadsheet for the corners P1-P4 "Reference projection of active area" rows 26-29 columns C-E of DESI-0530-v14.

In addition to updating the docs, I think it would be better for desimodel.inputs.gfa.build_gfa_table to have direct cut-and-paste input from DESI-0530, and then take care of the rotation starting at PETAL 3 as part of the code. This might just be pedantic because the end result will be the same numbers in gfa.ecsv, and any future updates would likely come from per-petal metrology directly updating gfa.ecsv rather than one petal rotated to all the others.

@dkirkby would you like me to make these changes in this branch?

[dkirkby]

@sbailey That sounds like a good plan re GFA locations. Please go ahead with the changes in this branch.

Is there anything else you think needs updating in desi.yaml? How do we back-propagate the new desi.yaml to DESI-347? (or do we not bother since its only purpose AFAIK is to forward propagate into DESIMODEL)

[dkirkby]

Note that if we decide to abandon the DESI-347 version of desi.yaml, we also need to update desimodel/inputs/throughput.update() so that it no longer copies this file to $DESIMODEL/data/.

[sbailey]

I will make the GFA updates later today.

In principle systems engineering owns desi.yaml via DESI-347. We can make suggestions that they update it, but we should really try to not have desimodel get ahead of the reference version in DESI-347. Please organize exactly what changes need to be made but then we should get Stu / Bobby to make the actual update.

[dkirkby]

One last change to get in synch with how specsim models centroid offsets (since Oct 2017):

• Change how the RMS of static centroid offsets over the focal plane is determined. It used to be the quadrature sum of cells B43:B72 from the Throughput sheet and is now just cell B52 ("FVC corrector optics figure error").
• The remaining contributions are now modelled with the specsim config parameter instrument.offset.sigma1d calculated as the quadrature sum of B43:B72 without B52.

Since the centroid errors in DESI-347 did not change between v11-v13, there is no change in the instrument model, just how we implement it in our simulations.

The new static RMS value is 8.0 um. The old value was 10.886 um.

Since this requires updating the throughput/DESI-0347_random_offset_[123].fits files, I will also use this opportunity to change random to static in the name.

Note that the value of sigma1d used in specsim was 5.1um, but this should have been 5.22um. It looks like @julienguy did not include the contributions from "Lateral shifts due to barrel misalignments relative to M1" in cells B65:74. The notebook in this package now calculates and documents this value.