User story
GPL-659 | As development team we want to remove samples with high CT values and results = 'limit of detection' from positives on site report to avoid waste sequencing

Who are the primary contacts for this story
Sonia G
Cristina A
Jeff B

Acceptance criteria
To be considered successful the solution must allow:

• Remove samples with result = 'limit of detection' from +ves on site report (assumption: samples with the prefix are treated like negatives so we wont pick them
• Remove sample if all non-missing channels are > 30. (CH1-Cq > 30 && CH2-Cq > 30 && CH3-Cq > 30)
• Although they are like negatives so we don't pick them, don't store ;'negative' state against them to avoid breaking reporting

Dependencies
GPL-642 Parsing and persisting CT values from AP #108
GPL-654 Parse & persist new result value 'limit of detection' #115

Additional context

Discussed post stand up on the 24th with reference to this diagram: https://app.lucidchart.com/invitations/accept/6499b3a0-7a14-4192-b5db-1cd35b84646f

Control plate tracking split out into: #202

User story
As Sample management and R&D we would like to automatically create Labwhere records for source and control plates that will be destroyed following Beckman Cherrypicking on a robot.
Specially this is for the following scenarios:
Control plate is used in a run (these plates are single use and have unique barcodes) - these go to the bin after the run
Source plates that are not readable / missing their barcodes - these go into the output stacks and to the bin
Source plates that contain only negatives - these go into the output stacks and to the bin
Source plates that have been fully picked - these go into the output stacks and to the bin

We do not want Labwhere events written for these scenarios:
Source plate has no plate map data - these go back to the input stack and are manually recorded in labwhere
Source plate is only partially picked - these go back to the input stack and are either used in the next run or manually recorded in labwhere if they are stored overnight
Destination plates - these will be manually recorded in Labwhere

Who are the primary contacts for this story
Scott T
Danni W

Acceptance criteria
To be considered successful the solution must allow:

• Labwhere records are created for plates as for the scenarios above.

Dependencies
https://github.com/sanger/General-Backlog-Items/issues/76
#176

Additional context
Consider adding a Labwhere endpoint to lighthouse that the Beckman method can call with a and either a
or a more human readable abstract location type that is translated in the lighthouse configs.
Or
Can we call Labwhere directly from the Beckman method? does labwhere have a direct API endpoint we can use? what information is required?

See sanger/unified_warehouse#118 for details

This is for the Lighthouse part.

User story
As a Sample Management user, I would like the plates created by the lighthouse service to come into Sequencescape as purpose 'LHR Stock', not 'Stock Plate', to keep them consistent with extracts coming in from CGaP (see GPL-502 - sanger/wrangler#56)

Who are the primary contacts for this story
Danni

Acceptance criteria

• Lighthouse service creates plates in Sequencescape with purpose 'LHR Stock'

User story
GPL-472 | As Stakeholders in sampling strategy (Keith A, Rob A & John S) we would like a report detailing all the positive lighthouse samples on site to send to PHE to select a sub set to sequence

Who the primary contacts are for this story
@rl15
Rich C
Danni W
Keith A
@pjvv

Acceptance criteria
Please see:

• Sequence flow diagram, messages from Sample Management Actor to crawler
• A prototype of the report was generated in RT 690358 on 19th May.

To be considered successful the solution must allow:

• From simple web page option to download list of positive Lighthouse samples on site - in GitLab: psd/lighthouse_reports
• Report format text based
• Crawler produces report listing:
  • Light house id {MK or AP} *1, *4
  • RNA plate barcode *1
  • Location (in reefers) *2 eg: West Pavillion, Reefer 1, R1 Racking 1, Shelf 1, lw-uk-biocentre-box-mk--10-14624
  • root sample id *1
  • result (all should be positive) *1
  • date tested *1
  • Have metadata required for upload of sequence to CLIMB? {unknown, yes, no} *3
  • Update date of 'Have metadata required for upload of sequence to CLIMB' populated *3

*1 From Crawler mongodb
*2 From LabWhere (see backlog item GPL-474 (sanger/labwhere#124))
*3 From upload of file returned from PHE (see backlog item GPL-473 (sanger/crawler#19)) - default to unknown for this feature
*4 Glasgow & Cambridge will come later

Additional context
...

User story
GPL-473 | As Stakeholders in sampling strategy (Keith A, Rob A & John S) we would like an endpoint that for a given root sample id updates value in sequencing with yes or no for SSR to use this in +ves lighthouse samples on site (see GPL-472)

Who the primary contacts are for this story
Rich L
Rich C
Christoph P (PAM)
Danni W (SM)
Acceptance criteria
Please see:
https://ssg-confluence.internal.sanger.ac.uk/x/iKMKBg
Sequence flow diagram, messages from SSR Actor to Sample crawler

To be considered successful the solution updates fields:

1. Lighthouse samples all start initially with field 'value in sequencing' set to 'unknown', with updated date set
2. Via API for a given root sample ID allow value to be set to yes or no + updated date
3. New fields in report available at https://psd.pages.internal.sanger.ac.uk/lighthouse_reports/ of

4. value in sequencing {yes, No, Unknown}
5. Date value in sequencing set

Additional context
Is it wise to have a preshared API key for this?
API will be given 1-? values to update (probably few 10k)

This is a bug coming out of the feature #84 (adding 'Cherrypicked' column to report).

We are setting the 'Cherrypicked' column in the 'Lighthouse positive samples report' by querying the Events warehouse to check if there has been a cherrypick event for that sample. Because we are matching based on the 'Root Sample ID', if we are sent a second plate that contains the same sample (as determined by Root Sample ID), the report will display 'Cherrypicked' : 'Yes' for the new sample as well as the old sample.

Acceptance Criteria
We should match based on plate barcode and well as well as Root Sample ID, so we can differentiate between instances of the same sample on different plates.

Extra context
Discussed with Rich C today. We agreed that:

• Repeat samples would be likely to sneak into cherrypick batches anyway, because the Lighthouse Sentinel feature selects all positive samples in a certain Labwhere location, whether or not they have been cherrypicked.
• Natalie uses the report as part of her wider reports and checks, so it would be good if the data was accurate despite the above.
• Rich is going to ask whether the institute wants to sequence repeat samples (if we get any), in a meeting later today

More insight on how the SSRs use the report:

• They run the report to identify positives on site.
• If they are creating a batch the normal / old way, they will copy-paste the 'plate and well' column into a cherrypick submission in Sequencescape
• If they are creating a batch the new way, they will grab the location barcodes for the positives they want to pick, and paste them into the Sentinel app
• With the new feature, they will filter the above report to only include samples that have not yet been cherrypicked
• Starting from next week, they will be receiving boxes of samples daily rather than the less frequent, larger deliveries, so will switch strategy to creating a cherrypick for boxes on the day they arrive.

User story
As developers, we'd like to change the Broker class to use a context manager workflow. This will simplify calling the class/publishing RabbitMQ messages/error recovery

Who are the primary contacts for this story
Phil
Chris F

Acceptance criteria
To be considered successful the solution must allow:

• Update the Broker class/RabbitMQ message publish workflow to use a context manager workflow

Dependencies
This story originates from the review of changes required for #203

Additional context
See pika documentation pika/pika#627

Dependabot couldn't authenticate with https://pypi.org/simple/.

You can provide authentication details in your Dependabot dashboard by clicking into the account menu (in the top right) and selecting 'Config variables'.

View the update logs.

User story
GPL-582 | As POS (Rich C) I would like to add well location to the heron positives crawler report to save further time in picking

Who are the primary contacts for this story
Rich C

Acceptance criteria
To be considered successful the solution must allow:

• add a list of acceptance criteria here
• ...

Dependencies
This story is blocked by the following dependencies:

• #<issue_no.>
• sanger/#<issue_no.>

Additional context
Rich C wrote(Monday, 20 July 2020 at 15:55)

We would like to harness the dynamic ability of SS to CP using sanger ID or “plate and well location”. We currently use sanger ID to cherrypick however, we realise the use of plate and location could save valuable time for lighthouse samples.

Currently we use the QC report to find sanger IDs for the samples on specific plates to be cherry-picked. If we change tack and add well location to the report, plate ID and well position can be concatenated, then directly copied into SS to for CP. This change would save a 10-20 mins run of the QC report to a 2 mins run of the positives report thus saving 80-90% time, for heron picking list generation.

Please could well position be added to the positives crawler report?

and maybe a column for =concatenate platename,”:”,well location as this is what we will have to add to the report each time?

User story
As developers we would like to add a validation check for use of the destination plate view, in order to cope with the potential synchronization delay between the local databases on the individual robots and the central DART database that the view operates from. Proposal is for Beckman to add a 'number_of_samples' property at the end of the run as they complete the destination then call the API endpoint. If on our side we can see that property on the plate, and the number of samples matches what we get back from the view, then we can proceed with the destination creation. If the property is not there yet, or the number of samples does not align, delay and retry a set number of times before erroring. The Beckman method should wait for a response.

Who are the primary contacts for this story
Eduardo, Chris, Alex, Pjil, Andrew

Acceptance criteria
To be considered successful the solution must allow:

• Beckman need to add a new plate property to the destination plate 'number_of_samples' that reflects the correct number of controls plus picked positives in the plate
• The API endpoint code looks for this new property, and only creates the destination plate in the LIMS if present and the number of samples matches the view data.
• Beckman need to wait on the response that may be delayed.

Dependencies
This story is blocked by the following dependencies:
Beckman need to agree and make the necessary changes to their method.
#145
https://github.com/sanger/General-Backlog-Items/issues/76

Additional context
Consider also story https://github.com/sanger/General-Backlog-Items/issues/79
This is the backup screen for creating a destination. Likely if this is being used that the robot method ended prematurely and the destination has not got the 'number_of_samples' property.

Describe the bug
This week we have been experiencing a very small number of duplicated lines (around 12) in the positve reports generated, that we think could be related with bugfix update performed on Monday.

RT Ticket Number
If applicable

To Reproduce
Steps to reproduce the behaviour:

1. Download a Excel positive report from lightouse -ui
2. Save as .csv (<csv_report>)
3. Run following shell commands to display the duplicated lines:

  cat <csv_report> | sort > duplicated
  uniq -d duplicated

Examples: 201106_1440_positives_with_locations.csv

Expected behaviour

The report should not have any duplication

Screenshots

Desktop (please complete the following information):

• Windows or Mac?
• Browser Chrome, Firefox, Safari or other?
• Browser version (use 'About' to look up)?

Additional context
Add any other context about the problem here.

User story
When a source plate is scanned on the robot that has no barcode, no DART data or has no positives and will not be picked, we would like to write events for these barcodes so that reporting can use them. There should also be an event for when a source plate gets 'completed'. This API endpoint would need to be called from the Beckman method with a barcode (if available), event type and robot serial number at the relevant 4 points in the process.

Who are the primary contacts for this story
Beckman
PSD

Acceptance criteria
To be considered successful the solution must allow:

• Get the event warehouse working locally and improve documentation on how to do so/document why it cannot be used locally as a standalone application and what other dependencies need to be added to get a local, working instance
• Add RabbitMQ to lighthouse so we can write directly to queue (need equivalent of Warren in Ruby, write as module so easily reusable in other Python apps)
• Update deployment project with new configuration variables/files
• See confluence page below for the 4 types of event required. Handle all of these, sending messages with relevant events, subjects etc.

Dependencies
sanger/crawler#156
sanger/unified_warehouse#145
sanger/event_warehouse#102

Additional context
See https://ssg-confluence.internal.sanger.ac.uk/display/PSDPUB/Cherrypicking+Events

• update README
• pin python packages to specific versions
• move config to a file?
• get plate purpose UUID and study using GET request to SS
• create tests for request to SS testing host and headers

Dependabot couldn't authenticate with https://pypi.org/simple/.

You can provide authentication details in your Dependabot dashboard by clicking into the account menu (in the top right) and selecting 'Config variables'.

View the update logs.

User story
GPL-726 | As Process Improvement specialist (Scott T) SM would like to scan a Lighthouse Lab box and see how many plates are in the box, as well as how many plates have metadata, in order to prevent lab staff starting to load plates onto the robot that are all negative results or have no plate map data (no value to be cherrypicked)

Who are the primary contacts for this story
Scott T (R&D)
Lesley S (R&D)
Danni W (SM)
Ian J

Acceptance criteria
To be considered successful the solution must allow:

• Do a quick mock up for Scott T (Proposer) & Danni W (User)
• read 2D barcodes of Lighthouse boxes
• list the plates in the box, indicating plates with plate map data or not with tick/cross (maybe color signpost also green/red)
• plates with plate map data to top of panel, those with no plate map data to bottom
• for each plate show the number of positives on the plate (sort plates by occupancy?) *1
• display summary below panel eg 'Total of 80 plates in box; 75 plates with plates maps, 5 without'
• Handle following error gracefully; a) Box barcode unknown b) Box empty
• Be able to scan the contents of a plate individually (sometimes the user would want to check plates scanning them one by one to take the decision by plate)

*1 It maybe more truthful to display the number +ves that can be picked (eg not high Ct or Cambridge controls) however if this introduces significantly more work just do the positives

Dependencies
This story is blocked by the following dependencies:

• #<issue_no.>
• sanger/#<issue_no.>

Additional context
Assumption this woudl be new UI in http://lighthouse-ui.psd.sanger.ac.uk/, called the 'Box buster'?

This story deprecates sanger-archive/lighthouse-ui#31

User story
As an SSR we want to remove samples with high CT values and results = 'limit of detection' from the samples imported into Sequencescape by the 'Lighthouse Sentinel' process, to avoid waste sequencing.

Acceptance criteria
Lighthouse Sentinel 'Sample Creation' page imports samples into Sequencescape based on following criteria

• result is 'Positive'
• CH1-Cq, CH2-Cq, CH3-Cq fields are either blank or >30

Dependencies
GPL-642 Parsing and persisting CT values from AP #108
GPL-654 Parse & persist new result value 'limit of detection' #115
GPL-659 is similar, but for the 'positives on site' report #120 - might be worth waiting for this to be done first as the code will be similar

Additional context
Lighthouse Sentinel 'Cherrypick' shouldn't have to be changed, because I think it just picks all samples on the plate.

User story
Any labware released from Lighthouse into Sequencescape will have set its values for all SS process; there is the possibility that the lighthouse files are updated and changed it values, which it could trigger updating Lighthouse database, but not SS after this sample has been released.
At the moment the only way to know if a sample has been released to Sequencescape is to check that the labware exists in SS database; it could be useful to know from Lighthouse when a sample has already been released in order to avoid updating its values from crawler, and also trigger an error in the situation when a file from crawler tries to change its value.

Who are the primary contacts for this story
Eduardo Martin
Andrew Sparkes

Acceptance criteria
To be considered successful the solution must allow:

• Store released_at as a timestamp of the moment when a sample has been released into SS
• Trigger a notification error when there is an attempt of updating the values of a sample that has already been released, so it needs to be look into.

Dependencies
This story is blocked by the following dependencies:

• #<issue_no.>
• sanger/#<issue_no.>

Additional context
Add any other context or screenshots about the feature request here.

User story
As a dev team we need to add tests to ensure changes made late on 3rd November are a sustainable change and producing right results confirmed to date only by manual testing.

Who are the primary contacts for this story
Andrew S
Steve I
Philip VV

Acceptance criteria
To be considered successful the solution must allow:

• Agree what we need to test
• Write tests
• Deploy to UAT
• Deploy to PROD

Dependencies

Additional context
Identified in stand up on 3rd Nov
Likert votes in slack in support of doing this

User story
Changes on message for plate creation will affect lighthouse message as well, so it needs to be updated. See GPL-453-2

Who the primary contacts are for this story
Eduardo Martin

Acceptance criteria
To be considered successful the solution must allow:

1. Support new message changes for plate creation

Additional context
New fields needed to be sent:

• Study uuid
• Purpose uuid

User story
GPL-842 | As core LIMS we would like a new collection to persist if a sample is of importance or value to sequence

Who are the primary contacts for this story
Callum S (PAM)
Ryes (PAM)
Harry C
Philip VV
Eduardo MR

Acceptance criteria
To be considered successful the solution must allow:

• New collection with schema
• For each sample PAM are aware of add to new collection
  root_sample_id
must_sequence (Sequence regardless – bypasses existing business rules)
  prferentially_sequence (Sequence if passes existing rules)
• update report generation using rules for new priority samples
• set processed = False on each insert
• use lookup with mergeObjects to get priority samples working
• set processed = False on each update - there seems to be a bug with the notmalize_on_patch setting

Dependencies

Additional context

Root sample id on it's own is not unique enough for that new collection key, it's the four fields of Root sample id, RNA ID, Result and Lab ID.

Parent feature: sanger/General-Backlog-Items#103

These requirements were first specified in #194 GPL-753.

User story
As Sample management and R&D we would like to automatically create Labwhere records for source and control plates that will be destroyed following Beckman Cherrypicking on a robot.
Specially this is for the following scenarios:
Control plate is used in a run (these plates are single use and have unique barcodes) - these go to the bin after the run

Who are the primary contacts for this story
Scott T
Danni W

Acceptance criteria
To be considered successful the solution must allow:

• Labwhere records are created for plates as for the scenarios above.

Dependencies
https://github.com/sanger/General-Backlog-Items/issues/76
#176
sanger/General-Backlog-Items#89 (Unique control plate barcodes generated by Mixito and understood by beckman/dart)

Additional context
Consider adding a Labwhere endpoint to lighthouse that the Beckman method can call with a and either a
or a more human readable abstract location type that is translated in the lighthouse configs.
Or
Can we call Labwhere directly from the Beckman method? does labwhere have a direct API endpoint we can use? what information is required?

User story
If we makes changes to filter positives earlier in the process in order to populate the Dart database, it should be possible to flag the filtered positives at that point in the MLWH and then just use that data to generate the positives report.

Who are the primary contacts for this story
PAM? Ops?

Acceptance criteria
To be considered successful the solution must allow:

• Change the STAGE_MATCH_POSITIVE and POSITIVE_SAMPLES_MONGODB_FILTER to use the new filtered_positive field: a sample is fit to pick if this is True
• Dig around for any more duplications of the fit to pick rules, or RESULT=Positive assumptions, and remove these

Dependencies
This story is blocked by the following dependencies:

• sanger/crawler#167

Additional context
Expecting to add some kind of flag for 'filtered positive' to the MLWH lighthouse_sample table. Should be able to select on those rows.
Assuming the positives report is even needed any more. In which case decommission the functionality.

Dependabot couldn't authenticate with https://pypi.org/simple/.

You can provide authentication details in your Dependabot dashboard by clicking into the account menu (in the top right) and selecting 'Config variables'.

View the update logs.

User story
GPL-741 | As a dev manage LIMS (Rich L) we may need to change the format of the +ves report to avoid truncating actionable data

Who are the primary contacts for this story
Rich L
Rich C
PAM
Ian J

Options

1. Remove samples form the report that are over 6 weeks old (Preferred by operations discussed stand up of 13th Nov)
2. Change the file to a text file format
3. Produce more than one file

Acceptance criteria
To be considered successful the solution must allow:

• change file to sliding 6 week date window

Dependencies
Limit of MS files 1,048,576 rows (1048k) : https://support.microsoft.com/en-us/office/excel-specifications-and-limits-1672b34d-7043-467e-8e27-269d656771c3

Additional context
Need for output may be superseded by New Beckman robotics package WIP Nov
+ves report at 941k rows on 20th Nov (21k/day)
+ves report at 920k rows on 19th Nov (10k/day)
+ves report at 910k rows on 18th Nov (18k/day)
+ves report at 892k rows on 17th Nov (36k/day)
+ves report at 856k rows on 16th Nov (37k/day)
+ves report at 819k rows on 13th Nov (18k/day)
+ves report at 602k rows on 1st Nov (14.7k/day)
+ves report at 336k rows on 13th Oct

User story
As a developer we would like the 422 response of a samples_declarations POST, which contains multiple errors, to inlcude more information about the root_sample_id

Who are the primary contacts for this story
Eduardo
Harry

Acceptance criteria

[ { "declared_at": "2020-06-26T11:37:45", "root_sample_id": "YOR10020466", "value_in_sequencing": "No" }, { "declared_at": "2020-06-26T11:37:45", "value_in_sequencing": "No" }, { "declared_at": "2020-14-500", "root_sample_id": "YOR10020379", "value_in_sequencing": "No" }, { "declared_at": "2020-06-26T11:37:45", "root_sample_id": "YOR10020240", "value_in_sequencing": "Yes" }, { "declared_at": "2020-06-26T11:37:45", "root_sample_id": "YOR10020379", "value_in_sequencing": "Yes" }, { "declared_at": "2020-06-26T11:37:45", "root_sample_id": "YOR10020224", "value_in_sequencing": "Yes" }, { "declared_at": "2020-06-26T11:37:45", "root_sample_id": "YOR10020218", "value_in_sequencing": "Yes" }, { "declared_at": "2020-06-26T11:37:45", "root_sample_id": "YOR10020217", "othertag": "othervalue" }, { "declared_at": "2020-06-26T11:37:45", "root_sample_id": "YOR10020195", "value_in_sequencing": "maybelater" } ]

The above json, with errors, will include the root_sample_id in the response for each respective error message

Describe the bug
A clear and concise description of what the bug is.

RT Ticket Number
If applicable

To Reproduce
Steps to reproduce the behaviour:

1. Go to page '...'
2. Click on button '...'
3. See error

Expected behaviour
What was expected to happen.

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

• Windows or Mac?
• Browser Chrome, Firefox, Safari or other?
• Browser version (use 'About' to look up)?

Additional context
Add any other context about the problem here.

User story
When the destination plate has been created send the destination plate barcode to sequencescape and create a plate with information for each sample pulled from either dart database, mongo database or the warehouse and also generate a COG UK barcode.

Acceptance criteria

• LIMS endpoint accepts a destination plate barcode
• Destination plate is generated in Sequencescape, with following additional data:
  • Wells
  • Samples
  • Sample metadata - Root Sample ID, Result, COG UK ID, RNA ID (this one is new, as would normally be able to track this back via the source plate)
• Plate purpose TBC (see comment below)
• Any error information associated with the run(s) that we can get from the DART database is recorded in Sequencescape (this will actually happen in the Event warehouse as events)

Additional context
Could generate COG UK barcodes earlier, e..g in step 1, for filtered positives, and store in MLWH

See diagram with options here - https://app.lucidchart.com/lucidchart/c029ead4-12c9-4baf-acb2-a2a3fbead2fc/edit?page=0_0#?folder_id=home&browser=icon

Describe the bug
Discovered an SQL injection vulnerability when working on #155.
Fixed it alongside. Ticket to track. More details once live.

RT Ticket Number
If applicable

To Reproduce
Steps to reproduce the behaviour:

1. Go to page '...'
2. Click on button '...'
3. See error

Expected behaviour
What was expected to happen.

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

• Windows or Mac?
• Browser Chrome, Firefox, Safari or other?
• Browser version (use 'About' to look up)?

Additional context
Add any other context about the problem here.

User story
As a SSR (Rich C) I would like the positive crawler report to include whether samples have been cherry picked (included in a released Batch in Sequencescape), so that we can check if requested samples have already been started.

Who are the primary contacts for this story
Richard C

Acceptance criteria
To be considered successful the solution must allow:

• Report should add a True/False or Date/NULL field for cherry picking (needs refining)

User story
If a run is mid-way and fails, or ends a run partially complete (end of day run, no sources remaining) we need an option for the user to scan the destination plate barcode directly into a LIMS screen (as robot method trigger step not reached) to either a) generate plate from Dart data so it can continue in pipeline partially filled or b) fail plate with a reason from a list.

Who are the primary contacts for this story
Scott T
Ops (Rich C?)

Acceptance criteria
To be considered successful the solution must allow:

• For creation: Scan entry of a destination plate barcode, selection of a robot, entry of a user if cannot be got automatically, 'create plate in LIMS' button triggers standard destination creation process that generates the plate in the LIMS and created the same set of events as a normal completion on the robot would have.
• For failure: Scan entry of a destination plate barcode, selection of a robot, entry of a user if cannot be got automatically, choice of failure reason from a list, 'fail plate in LIMS' button. Creates a destination failed event of some type here that will record what we know about any samples lost.

Dependencies
This story is blocked by the following dependencies:
#145

Additional context
Aim is to 'save' destination plates as much as possible. Samples when transferred leave nothing behind in the source plate, so if we cannot recover the destination then those samples are 'lost'.
If we know what samples are being 'lost' in the failed plate, these should be recorded in an event, such that PAM or others running reporting can find this failure event and understand what happened to those samples.
It will be important that Ops have a procedure in place for failed plates, to make them use this screen, as well.

Implementation Details
The screen failure/creation screen should exist in lighthouse-ui. The UI should call endpoints in the lighthouse backend. The creation operation should call the existing cherrypicked-plates/create?... endpoint (#145); the failure operation should call a new cherrypicked-plates/fail?... endpoint which will send an event to the warehouse as per https://ssg-confluence.internal.sanger.ac.uk/display/PSDPUB/Cherrypicking+Events

User story
This is a story for psd-deployment project.
When restarting the stack the services take a while to connect with each other. We have to check if this is because they are not started in order. If that is the case, we may need to add dependencies in the config for the stack so they can start in a better order

Who are the primary contacts for this story
Eduardo Martin

Acceptance criteria
To be considered successful the solution must allow:

• Services start in order without sending errors about not connecting to dependent services.

Dependencies
This story is blocked by the following dependencies:

• #<issue_no.>
• sanger/#<issue_no.>

Additional context
Add any other context or screenshots about the feature request here.

User story
In adopting the Beckman workflow, event reporting is changing. This will break anything dependent on these events. One such thing is the filtering for cherrypicked samples in both lighthouse and crawler. These are SQL queries which currently query for the Sentinel cherrypicking events only. They need to be updated to query for Beckman picking events as well.

The alternative is to fire the same cherrypick_layout_set event for the Beckman process, but since no real Beckman action corresponds to this event, this solution has been rejected.

Who are the primary contacts for this story
Katy T
Phil
Chris F

Acceptance criteria
To be considered successful the solution must allow:

• Update lighthouse method to get cherrypicked samples to query for both Sentinel or Beckman cherrypicking events. This is used to generate the positives report.
• Update crawler methods to get cherrypicked samples to query for both Sentinel or Beckman cherrypicking events (introduced as part of sanger/crawler#144) See sanger/crawler#192
• (Optional/Do as a separate task) - have only a single implementation of these "is a sample cherrypicked" rules. See sanger/unified_warehouse#152

Additional context
Add any other context or screenshots about the feature request here.

Describe the bug
User tried sending a request of 70.000 declarations to uat but service failed with error code 413.
This is probably a change to do in psd-deployment for lighthouse.

RT Ticket Number

To Reproduce
Steps to reproduce the behaviour:

1. Create a big file:

truncate -s 10M bigfile.txt

2. Send it to the service:
   curl -X POST http://....samples_declarations -H "x-lighthouse-client: ..." -F '[email protected]'

3. See error

413 Request Entity Too Large

Expected behaviour
Not having 413.

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

• Windows or Mac?
• Browser Chrome, Firefox, Safari or other?
• Browser version (use 'About' to look up)?

Additional context
Add any other context about the problem here.

User story
As developers we'd like to record all Result=Positive samples for source plate complete and source plate all negatives cherrypicking events

Who are the primary contacts for this story
Chris F

Acceptance criteria
To be considered successful the solution must allow:

• Send all Result=Positive samples as event subjects for the lh_beckman_cp_source_completed and lh_beckman_cp_source_all_negatives events
• Record the behaviour change in https://ssg-confluence.internal.sanger.ac.uk/display/PSDPUB/Cherrypicking+Events

Dependencies
This story is blocked by the following dependencies:

• #176

Additional context
See Cherrypicking events documentation: https://ssg-confluence.internal.sanger.ac.uk/display/PSDPUB/Cherrypicking+Events

User story
GPL-648 | As heron stakeholder (Sonia G) I would like LH Cambridge internal control samples labelled as positives in the plate map to be filtered out of Sentinel Cherry pick and the 'positives on site report' because we don't want to sequence controls (no value)

Who are the primary contacts for this story
Sonia G
Rich L

Acceptance criteria
To be considered successful the solution must allow:

• Remove samples with prefix 'CBIQA_' from +ves on site report and Sentinel Cherry pick (assumption: samples with the prefix are treated like negatives so we wont pick them)
• Although they are like negatives so we don't pick them, don't store 'negative' state against them to avoid breaking reporting

Assumptions
If sample with prefix 'CBIQA_' id null, void or negative will also be filtered out

Questions
Q. Should we log that these undeclared controls are in a given plate map?

Additional context
This is the lowest priority backlog item of the LH changes package of work as operations are currently manually working around this (assume deleting samples from the positives report)

Its possible that the other labs are running undeclared +ves controls in their process. Consider a means that we can by configuration discard samples with a given prefix. (aware possible premature optimisation)

User story
As a dev team we would like the lighthouse service to create a plate with samples in SS when a request is made from asset audits and the plate is found in the external sites' data in mongo.

Who the primary contacts are for this story
Philip

Acceptance criteria
To be considered successful the solution must allow:

1. an endpoint is required which AA can call, e.g. /plate/new/<barcode>
2. once validation passes, SS should be called with the sample and sample metadata

Additional context
Blocked by sanger/crawler#1

Describe the bug
From mail:

Our monitoring system is again warning us that there are more than 300 concurrent connections to MLWH, and as before, this is due to the sudden appearance of a large number (241) connections They appeared at around 11:30, and they have been idle for the past 50 minutes.

RT Ticket Number
If applicable

To Reproduce
Steps to reproduce the behaviour:

1. Go to mlwh
2. Run: show status;
3. See number for 'threads_connected'.

Expected behaviour
The number of max threads is 500 and is shared with everybody using the MLWH database. We should not create 200 threads just for creating a report.

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

• Windows or Mac?
• Browser Chrome, Firefox, Safari or other?
• Browser version (use 'About' to look up)?

Additional context
Add any other context about the problem here.

Describe the bug
When viewing the results from the lighthouse service, the timestamps look wrong but those timestamps don't appear in mongo so I suspect it's a Eve thing.

RT Ticket Number
If applicable

To Reproduce
Steps to reproduce the behaviour:

1. Go to page '...'
2. Click on button '...'
3. See error

Expected behaviour
What was expected to happen.

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

• Windows or Mac?
• Browser Chrome, Firefox, Safari or other?
• Browser version (use 'About' to look up)?

Additional context
Add any other context about the problem here.

Describe the bug
Lighthouse +ves on site report failing. Root issue is query that builds the ‘Lims submission;’ which I used as ‘cherry picked yes/no’ flag is failing - SQL packet size limits

RT 704417
reported vai slack at 1:36pm

To Reproduce
Steps to reproduce the behaviour:

1. Go to page '...'
2. Click on button '...'
3. See error

Expected behaviour
What was expected to happen.

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

• Windows or Mac?
• Browser Chrome, Firefox, Safari or other?
• Browser version (use 'About' to look up)?

Additional context
Add any other context about the problem here.

User story
As a dev team we would like to update the call to baracoda to include the number of barcodes to create.

Who the primary contacts are for this story
EMR

references sanger/baracoda#5

Describe the bug
https://psd.pages.internal.sanger.ac.uk/lighthouse_reports/

RT Ticket Number
If applicable

To Reproduce
Steps to reproduce the behavior:

1. Go to page https://psd.pages.internal.sanger.ac.uk/lighthouse_reports/
2. Click on button 'Create report'
3. See error (attached)

Expected behaviour
Get report - last reported was on 3rd - automatically generated.

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

• Browser Firefox

Additional context
Add any other context about the problem here.

Dependabot couldn't authenticate with https://pypi.org/simple/.

You can provide authentication details in your Dependabot dashboard by clicking into the account menu (in the top right) and selecting 'Config variables'.

View the update logs.

Describe the bug

POST request to sample_declarations:
When providing a JSON with a missing sample ID, I do not get a JSON with error message, but an html with "internal server error"

User story
Update the existing report generation/example queries to reflect the fit-to-pick rules changed in sanger/crawler#150. We aren't fully committing to the new filtered_positive fields (we're only adding to new samples for now), and require that the old and new "fit-to-pick" workflows are run in parallel until we do.

Who are the primary contacts for this story
Chris F
Harriet C

Acceptance criteria
To be considered successful the solution must allow:

• update queries in lighthouse

Additional context
Add any other context or screenshots about the feature request here.

User story
All reports are stored in the root filesystem, which is decreasing available storage for other purposes. Same as we've done in crawler, it will be desirable to move the reports into its own openstack volume so it doesn't affect to other systems if the number of reports increases.

Who are the primary contacts for this story
Eduardo Martin

Acceptance criteria
To be considered successful the solution must allow:

• All reports stored in a new Openstack volume (not under / as it is happening now)

Dependencies
This story is blocked by the following dependencies:

• #<issue_no.>
• sanger/#<issue_no.>

Additional context
Add any other context or screenshots about the feature request here.

User story
When the destination plate has been created, create an event in MLWH events to say which samples and robot it was created on.

Who are the primary contacts for this story
Beckman
PSD

Acceptance criteria
To be considered successful the solution must allow:

• On creating the plate, create a broadcast event in SS database to trigger generate a new event in MLWH events
• Check that the events have been generated with the right subjects associated as described in document (Cherrypicked Events)

Dependencies
This story is blocked by the following dependencies:
sanger/crawler#156
sanger/unified_warehouse#145

Additional context
See https://ssg-confluence.internal.sanger.ac.uk/display/PSDPUB/Cherrypicking+Events
We need to find out whether the plate still needs to be registered in Labwhere for reporting purposes.
SS issue: sanger/sequencescape#2959

User story

Who are the primary contacts for this story
Callum S (PAM)
Ryes (PAM)
Harry C
Philip VV
Eduardo MR

Acceptance criteria
To be considered successful the solution must allow:

• Undo GPL-473

Dependencies
This story is blocked by the following dependencies:

• #<issue_no.>
• sanger/#<issue_no.>

Additional context
Parent feature: sanger/General-Backlog-Items#103

User story
GPL-619 | As an SSR for Heron (Rich C) I would like the column in the +ves on site report renamed from cherry picked to LIMS submission to keep naming consistent and avoid confusion

Who are the primary contacts for this story
Rich C

Acceptance criteria
To be considered successful the solution must allow:

• change last column label cherry picked to LIMS submission

Additional context
Raised in Monday 10am project meeting by Rich C, seconded by PM Tanya B.
The report is available here: http://lighthouse-ui.psd.sanger.ac.uk/ (reports)

User story
Every day a new report with estimated size 5mb is created in filesystem. As it is not very useful to keep very old reports and we can run out of disk spaces eventually, I suggest to modify reports to remove old copies and keep a reasonable number (last 20?).

Who are the primary contacts for this story
Eduardo Martin

Acceptance criteria
To be considered successful the solution must allow:

• Keep the number of reports under 20.

Dependencies
This story is blocked by the following dependencies:

• #<issue_no.>
• sanger/#<issue_no.>

Additional context
Add any other context or screenshots about the feature request here.