Should framework be more tollerant of cleanup() exceptions? about autotest-docker

[cevich]

My opinion: #36 (comment)

What do others thing?

[cevich]

I'm somewhat against it for the simple reason that it can make debugging dockertest harder. In other words, allowing cleanup() exceptions through makes it easier to tell the difference between a docker problem and a dockertest problem. OTOH, I'm not against catching/hiding specific exceptions in cleanup() (or elsewhere) as long as there's a clear reason for it.

---

@ldoktor @jzupka @Lorquas what do you guys think on this one?

[ldoktor]

Well, my idea is clearly stated there. I'd like the cleanup to be very tolerant. You won't miss any exceptions as you have all of them listed in the log and the cleanup raises Exception.

About the dependent parts, it's harder to write these kind of cleanups, but you can use blocks of cleanup code (eg. call cleanup_container() for each container, which removes container and it's image. When remove container fails, you won't execute the remove image part, just log the failure and proceed with another container).

So my idea of cleanup is put short cleanups directly into cleanup() put longer modules into separate cleanup_xxx() functions.

Anyway after the test everything should be clean, with no hanging containers, nor images or other resources... (eg. failed os.unlink() shouldn't interrupt container cleanup).

Btw I'd also like to create framework function for container cleanup, which should include workarounds like umount /dev/mapper/docker-* in case docker rm failure. But it's just for the future...

[cevich]

oh sure, yeah I understand your reasoning. I guess my position is more inclusive, where cleanup() is also a part of what we're testing (instead of purely a mechanical/plumbing thing).

See also: #55

Another option is to do both, consider cleanup() as I describe above, then also have between-test guaranteed cleanup that always runs (i.e. after envcheck scripts). If we keep the later part somewhat generalized, it can be re-used for the "best effort" cleanup() part too.

A third option is we simply forget about cleanup() in tests entirely, then only rely on automatic, guaranteed cleanup enforced lower-down the stack (maybe similar to envcheck). This would have the benefit of making nearly every test one-method simpler. The down-side is if/when the enforced cleanup breaks, potentially everything breaks.

[ldoktor]

You won't miss the cleanup failures in the 1st scenario. All failures will still be logged, you'll just proceed the additional cleanup steps.