Comments (14)
Looks like r8-D had a hot row indeed
from desisurveyops.
@julienguy , @sbailey , do we have a policy for what we want to do with these? Options:
- Mark the row as bad and reprocess
- Mark the amp as bad and reprocess
- Mark the exposure as bad and reobserve
This is rare enough that (3), while wasteful, isn't completely crazy. But we should make a policy.
from desisurveyops.
- is what the current pipeline is set up to do: either we mark 'manually' this amplifier as bad in the exposures table, or the code auto-detects a bad amplifier with an overscan step measurement greater than 5 electron (this apparently did not work well for that exposure).
Looking at this exposure, the header keyword OSTEPD
(overscan step for amplifier D) has a value of 4.67 electrons, and the threshold to declare the amplifier bad is set at 5 electrons. However, because OSTEPD>2, overscan subtraction per row was triggered, see in header:
OMETHD = 'PER_ROW ' / use average overscan per row
Quite unexpectedly, it is the overscan subtraction per row that generates this horizontal band (!) ...
- this is caused by a negative trail from a large charge deposit from a cosmic ray or radioactive decay bleeding in the overscan region (see image below)
- we have in place this 'dark trail correction' (see PR desihub/desispec#798 and issue desihub/desispec#796), but it comes with calibration parameters that have to be tuned for each CCD.
- I am opening a ticket on this in desispec.
from desisurveyops.
The underlying issue is CTE from cosmics (desihub/desispec#1619). I've added this to that ticket as another example. I think it's common enough on Z8-D, Z3-D, Z5-8 that we don't want to throw out these data, but rather figure out how to mitigate in the spectro pipeline.
from desisurveyops.
@schlafly it appears that tile 10184 was flagged as GOOD and thus got slurped up in the archiving on January 25 (last Tuesday) before the fix in PR desihub/desispec#1636 and before reprocessing could have fixed this.
Please confirm whether that is ok (i.e. "good enough"), or whether we need to reprocesses this petal and re-MTL update this tile (to be interleaved with Fuji). I think the danger is the possibility of LyA QSOs getting overridden by bogus low-z galaxy fits due to this overscan dip feature.
from desisurveyops.
Note there have been no overlapping DARK tile observations of this tile between when it was observed (20220106) through last night (20220130), so it would even be possible to just back out these MTL updates and load new ones.
from desisurveyops.
For what is worth, the newlya identified Ly-a diagnosis for this tile looks ok:
It is the point at (x,y) ~ (2.6, 80) in this plot: the y-value is not significantly below the expectation.
Though we are talking here about one petal only, i.e. one tenth of stats, so not sure how much the effect would be visible.
from desisurveyops.
from desisurveyops.
No, it looks like I marked it as good with the commit "Moving more tiles with marginal problems documented at desisurveyops from unsure to good." I don't think it was my intent to get this one---I didn't mark this problem as "doesnotblockmtl"---sorry. For my two cents it doesn't mess with LyA decisions and we can safely ignore. If we want to reincorporate, I'd rather do that through the "normal" MTL reincorporation mechanism than backing out---it just seems safer and more like standard procedure.
from desisurveyops.
My two cents: I agree with Eddie that in general, now we have the reprocessing code for MTL, we should re-archive these sorts of tiles and then apply the MTL reprocessing code, instead of backing tiles out.
Backing out is tempting, in this case, as it's so simple. But, I'd rather have a standard procedure where once a month (say) we re-archive and re-process any problematic tiles. Otherwise we'll be tempted to end up back in a situation where we're doing something "special" on a semi-regular basis.
from desisurveyops.
from desisurveyops.
No, the software fix just fixes this and the data is fine, so nothing is lost if the LyA decisions are okay.
And we can discuss again at the survey-ops telecon, but I understand the new standard to be that if we're not messing up LyA decisions, we can proceed with MTL updates. So marking half a petal later as bad is something that I would say need not hold up MTL updates.
from desisurveyops.
from desisurveyops.
Fixed in software, added to closed issues, and closing.
from desisurveyops.
Related Issues (20)
- Collimator feature (b-camera) on 20240223 sframesky images HOT 1
- tile 20711 from 20240203 needs QA plots HOT 7
- Tiles marked as unsure after r3 FEE crash on 20240321 HOT 5
- Should tertiary executable scripts be moved elsewhere from bin/? HOT 3
- All tiles observed on the night of 20240327 marked as unsure due to a problem with z7. HOT 7
- Marked 2 tiles as unsure due to r5 and z5 CCD restart at the end of night 20240402 HOT 7
- All tiles observed on the night of 20240403 marked as unsure due to multiple problems/doubts HOT 9
- Tile 6157 observed on 20240404 marked as unsure due to feature in redshift distribution HOT 1
- Multiple features on sframesky from 20240406 HOT 1
- redrock files missing from archive folder HOT 2
- archive status of 22432 with missing petals for different reasons HOT 4
- tile 25529 archive 20221008 vs 20221011 HOT 3
- Mark z6 as bad from 20231130 - 20231210?
- Bright tiles 25915 and 21779 affected by CTE in r7 HOT 2
- Bright tile 23163 marked as unsure HOT 2
- r7 has fairly bad CTE starting on 20240603
- Reprocess tiles with "old" QN weights instead of new QN weights HOT 15
- Tiles from 20240627 marked unsure due to possible r1 reset. HOT 11
- Make sure updated sky monitor gets installed at KPNO
- Tile 3395 from 20240811 has low THRUFRAC values on P1/P2/P3 HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from desisurveyops.