Fields of job information sent over tube about container-pipeline-service

[navidshaikh]

For example: We do not add user email address at first place when we are populating the data structure to be sent over the tube for build_phase, but we add it later when we want to trigger notify_user job. Now notify user worker is depending on the code which adds such information, however this information could have been part of build_details at the first place when we parsed the YAML file.

[dharmit]

I see an upside to this, sure. This would help in syncing various workers in the terms of data being consumed and generated and might be a good to have. However, I'm not sure how much value it adds at the moment since we're moving fast and, of course, breaking certain things while trying to add features or fix something. If the data being passed across different workers is causing an issue, this needs to be prioritized and worked upon sooner rather than later.

WDYT @bamachrn @kbsingh @rtnpro @mohammedzee1000 ?

[mohammedzee1000]

Yea definitely makes sense. It will require some refactor so im not sure we do it now or do it in the near future.

[kbsingh]

+1, sure - lets put the entire metadata we have for the specific job on the queue at one go. if nothing else it simplifies the work process, and does not need to reference the original data source file anymore. i.e once the job is started, it should keep going even when the orignating data stock changes ( and will be reflected in a subsequent job ).