deletion of workflow logs about knix

[iakkus]

I can have a look at this one too. It seems like

1. Need to assign a workflow index during creation of the workflow. (e.g., mfnwf-) (see below)
2. ManagementService log retrieval needs to use the workflow's index.
3. ManagementService log clearing now can really delete the index and re-create it. (see below)
4. The SandboxAgent needs to update the fluent-bit configuration to use the workflow index.

Regarding clearing the logs, does elasticsearch treat these two indices differently via some internal versioning (the one deleted and re-created with the same name)? If not, this may not do the intended job of actually deleting the logs because the old log entries would still be retrieval with the 'newly' created index that still has the same name. Then we may need to keep the version of the index in the workflow metadata and pass that to the sandbox agent when starting.

Regarding index creation, an alternative method could be to create the index the first time a workflow gets deployed, or a sandbox starts up (i.e., from within the sandbox agent). However, I think it is better to do it at the workflow creation (i.e., when the user adds a new workflow) for the following three reasons:

• The deletion of the workflow index can only happen when the workflow is deleted. One can also choose to do it during the undeployment of the workflow, but that would make the logs inaccessible for an undeployed (but not deleted) workflow. Users may still want to access the logs even if the workflow has been undeployed. Furthermore, if the deployment itself fails, having the logs would be helpful to debug the problem.
• If the index creation is done during startup of the sandbox, this may add a small overhead for the sandbox startup. If additional instance of the sandbox are starting (e.g., for scaling), the creation of the index, even if it is idempotent, would still do redundant a job.
• The symmetry of the creation/deletion of the index following the creation/deletion of the workflow.

Comments, thoughts? Did I miss any other modification that needs to be made?

[iakkus]

Branch feature/individual_indices_for_workflow_logs.

Tasks 1, 2, 4 are finished.
Task 3 is TODO.