Concurrent deploy failures? about payara

[dmatej]

Isn't it somehow connected to this?
https://java.net/jira/browse/GLASSFISH-21180
We have to restart domain before/after each redeploy since SGES2, there is also some memory leak in deployment/undeployment - but that is another problem.

And another problem is that from Glassfish4.0 the undeploymens are not completed and redeploy or deploy after undeploy of the same application is unsuccessful. Workaround is to manually delete applications/... directory before deployment.
https://java.net/jira/browse/GLASSFISH-21147

It seems to me that the deploy/undeploy/redeploy actions are very buggy now (in Glassfish4.0, 4.1, trunk, and Payara too). Only first deploy works.

[smillidge]

There's a grizzly patch available for https://java.net/jira/browse/GLASSFISH-21180 which is referenced in #53.

We are currently working on some other deployment issues for a specific customer. However I have to admit it took us a while to reproduce the issue as in our test systems we have been continually redeploying applications without problems.

BTW feel free to raise issues on here. If we can reproduce the issue in Payara and then also in upstream GlassFish we will raise a corresponding GlassFish JIRA and crosslink. Similarly if you have a particular GlassFish issue you would like to see addressed feel free to raise an issue here containing a reference to the GlassFish JIRA issue.

[smillidge]

@dmatej please feel free to raise memory leaks in deployment as a separate issue as well as GLASSFISH-21147

[realityforge]

I have not had time to invest in getting this reproducible but will try to dedicate some time to do it in about 10 days or so. It turns out we are actually experiencing lots of issues with our deploys.

• We are still hangs as described in GLASSFISH-21175 despite #24 being merge in before the 114 release.
• We see this crash intermittently, often when restarting a server with multiple applications deployed or when launching from an IDE (Intellij IDEA) with multiple applications deployed.
• GLASSFISH-21180 we work around in our automation so it is less of an issue for deployment but is an issue with some of our applications.
• Surprisingly we have never experienced GLASSFISH-21147 despite having ~50 deploys a day but we don't use ear's.

[smillidge]

Can you split each issue out into a separate case. Just to note, as we haven't documented our issue labels in any detail yet "try to reproduce" is for us rather then yourself although we will ask for assistance on the ticket. We are currently working a high priority deployment issue for one of our customers where the hidden file .glassfishStaleFiles remains in the deployment directory after undeploy.

We believe this maybe due to timing due to NFS as for this customer the deploy directory is hosted on an NFS mount.

[smillidge]

We have committed a specific deployment fix 51f03dc I'd be interested to see if that helps with any of the problems listed in this issue

[smillidge]

Also we have a further commit related to GLASSFISH-21175 just gone in so I'd be interested to hear if that has helped as well.

[smillidge]

https://java.net/jira/browse/GLASSFISH-21180 should also be fixed in the latest builds

[realityforge]

I have used the version for the last day and it seems like GLASSFISH-21175 is fixed. I will continue working with it and see if I can reproduce any of the other concurrency problems. Thanks!

[realityforge]

Great work - I can not reproduce most of the concurrency problems after moving to that version of payara. I have however been able to reproduce one exception that only seems to occur during concurrent deploys. It is relatively rare only occurring 2-3 times a day with active development. Obviously it is only when deploying wars with web services inside.

Great work on all the rest - thanks!

https://gist.github.com/realityforge/838b015cc409f62a5e8c

[smillidge]

I'm closing this issue and opening a separate issue for the Webservices deployment. Issue #124 raised.