Failed to generate the reports due to Too many open files about bootstraped-multi-test-results-report-plugin

[BogdanLivadariu]

Had similar issues with cucumber reports, will fix it on Monday and release a new version with the fix.

Thanks for the report

[feiniao0308]

Thanks a lot for the quick response :)

[BogdanLivadariu]

@feiniao0308
can you update the jenkins plugin to 1.4.3 and see if the issue persists ?

I think this https://github.com/BogdanLivadariu/testng-reporting-handlebars/commit/08a27aad4fc67df6cc0159235d96d53168d97f98
should fix it, but want to double check with you, before closing this issue

[feiniao0308]

Great, the fixing works for so large number of xml files.

But I hit another issue when the xml file has large size, about 5.5 MB. And it throws another error. Do you have any idea?

[TestNGReportPublisher] Compiling TestNG Html Reports ...
[TestNG test report builder] Copying all xml files from slave: /alps/logs to master reports directory: /var/lib/jenkins/jobs/PGI_vSphereGuestOps_RHEL66_64_CRS-ME_Layer1/builds/2015-09-28_08-00-28/testng-reports-with-handlebars
[TestNGReportPublisher] Found 1 xml files.
[TestNG test report builder] 0. Found a xml file: 09_26_2015_08_07_56/Suite1testNGLogs/testng-results.xml
[TestNG test report builder] Generating HTML reports
processing: /var/lib/jenkins/jobs/PGI_vSphereGuestOps_RHEL66_64_CRS-ME_Layer1/builds/2015-09-28_08-00-28/testng-reports-with-handlebars/xmlData/09_26_2015_08_07_56/Suite1testNGLogs/testng-results.xml
FATAL: com/github/jknack/handlebars/HandlebarsError
java.lang.NoClassDefFoundError: com/github/jknack/handlebars/HandlebarsError
    at com.github.jknack.handlebars.internal.Partial.merge(Partial.java:134)
    at com.github.jknack.handlebars.internal.BaseTemplate.apply(BaseTemplate.java:130)
    at com.github.jknack.handlebars.internal.TemplateList.merge(TemplateList.java:71)
    at com.github.jknack.handlebars.internal.BaseTemplate.apply(BaseTemplate.java:130)
    at com.github.jknack.handlebars.internal.BaseTemplate.apply(BaseTemplate.java:117)
    at com.github.jknack.handlebars.internal.ForwardingTemplate.apply(ForwardingTemplate.java:82)
    at com.github.jknack.handlebars.internal.ForwardingTemplate.apply(ForwardingTemplate.java:60)
    at com.github.bogdanlivadariu.reporting.testng.builder.TestNgReportBuilder.writeTestCaseSummaryReport(TestNgReportBuilder.java:98)
    at com.github.bogdanlivadariu.reporting.testng.builder.TestNgReportBuilder.writeReportsOnDisk(TestNgReportBuilder.java:110)
    at com.github.bogdanlivadariu.jenkins.reporting.testng.TestNGTestReportPublisher.perform(TestNGTestReportPublisher.java:128)
    at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19)
    at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:717)
    at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:692)
    at hudson.model.Build$BuildExecution.post2(Build.java:183)
    at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:639)
    at hudson.model.Run.execute(Run.java:1527)
    at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
    at hudson.model.ResourceController.execute(ResourceController.java:88)
    at hudson.model.Executor.run(Executor.java:236)
Caused by: java.lang.ClassNotFoundException: com.github.jknack.handlebars.HandlebarsError
    at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
    ... 19 more

[feiniao0308]

With the large xml file, Sometimes, throw following error and jenkins will be restarted.

[TestNGReportPublisher] Compiling TestNG Html Reports ...
[TestNG test report builder] Copying all xml files from slave: /alps/logs to master reports directory: /var/lib/jenkins/jobs/PGI_vSphereGuestOps_RHEL66_64_CRS-ME_Layer1/builds/2015-09-28_08-21-51/testng-reports-with-handlebars
[TestNGReportPublisher] Found 1 xml files.
[TestNG test report builder] 0. Found a xml file: 09_25_2015_03_01_53/Suite1testNGLogs/testng-results.xml
[TestNG test report builder] Generating HTML reports
processing: /var/lib/jenkins/jobs/PGI_vSphereGuestOps_RHEL66_64_CRS-ME_Layer1/builds/2015-09-28_08-21-51/testng-reports-with-handlebars/xmlData/09_25_2015_03_01_53/Suite1testNGLogs/testng-results.xml

Status Code: 500

Exception: /var/lib/jenkins/jobs/PGI_vSphereGuestOps_RHEL66_64_CRS-ME_Layer1/builds/2015-09-28_08-21-51/log (Too many open files)
Stacktrace: 
java.io.FileNotFoundException: /var/lib/jenkins/jobs/PGI_vSphereGuestOps_RHEL66_64_CRS-ME_Layer1/builds/2015-09-28_08-21-51/log (Too many open files)
    at java.io.RandomAccessFile.open(Native Method)
    at java.io.RandomAccessFile.<init>(RandomAccessFile.java:233)
    at org.kohsuke.stapler.framework.io.LargeText$FileSession.<init>(LargeText.java:351)
    at org.kohsuke.stapler.framework.io.LargeText$1.open(LargeText.java:75)
    at org.kohsuke.stapler.framework.io.LargeText.writeLogTo(LargeText.java:164)
    at hudson.console.AnnotatedLargeText.writeHtmlTo(AnnotatedLargeText.java:158)
    at hudson.console.AnnotatedLargeText.writeLogTo(AnnotatedLargeText.java:145)
    at org.kohsuke.stapler.framework.io.LargeText.doProgressText(LargeText.java:216)
    at hudson.console.AnnotatedLargeText.doProgressiveHtml(AnnotatedLargeText.java:91)
    at sun.reflect.GeneratedMethodAccessor191.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:601)
    at org.kohsuke.stapler.Function$InstanceFunction.invoke(Function.java:288)
    at org.kohsuke.stapler.Function.bindAndInvoke(Function.java:151)
    at org.kohsuke.stapler.Function.bindAndInvokeAndServeResponse(Function.java:90)
    at org.kohsuke.stapler.MetaClass$1.doDispatch(MetaClass.java:111)
    at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:53)
    at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:574)
    at org.kohsuke.stapler.Stapler.invoke(Stapler.java:659)
    at org.kohsuke.stapler.MetaClass$4.doDispatch(MetaClass.java:203)
    at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:53)
    at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:574)
    at org.kohsuke.stapler.Stapler.invoke(Stapler.java:659)
    at org.kohsuke.stapler.MetaClass$12.dispatch(MetaClass.java:384)
    at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:574)
    at org.kohsuke.stapler.Stapler.invoke(Stapler.java:659)
    at org.kohsuke.stapler.MetaClass$6.doDispatch(MetaClass.java:241)
    at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:53)
    at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:574)
    at org.kohsuke.stapler.Stapler.invoke(Stapler.java:659)
    at org.kohsuke.stapler.Stapler.invoke(Stapler.java:488)
    at org.kohsuke.stapler.Stapler.service(Stapler.java:162)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
    at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:800)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1669)
    at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:95)
    at hudson.plugins.audit_trail.AuditTrailFilter.doFilter(AuditTrailFilter.java:66)
    at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:98)
    at hudson.plugins.greenballs.GreenBallFilter.doFilter(GreenBallFilter.java:58)
    at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:98)
    at net.bull.javamelody.MonitoringFilter.doFilter(MonitoringFilter.java:206)
    at net.bull.javamelody.MonitoringFilter.doFilter(MonitoringFilter.java:179)
    at net.bull.javamelody.PluginMonitoringFilter.doFilter(PluginMonitoringFilter.java:86)
    at org.jvnet.hudson.plugins.monitoring.HudsonMonitoringFilter.doFilter(HudsonMonitoringFilter.java:84)
    at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:98)
    at hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:87)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
    at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:47)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
    at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84)
    at hudson.security.UnwrapSecurityExceptionFilter.doFilter(UnwrapSecurityExceptionFilter.java:51)
    at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
    at org.acegisecurity.ui.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:166)
    at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
    at org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(AnonymousProcessingFilter.java:125)
    at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
    at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:142)
    at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
    at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:271)
    at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
    at org.acegisecurity.ui.basicauth.BasicProcessingFilter.doFilter(BasicProcessingFilter.java:173)
    at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
    at jenkins.security.ApiTokenFilter.doFilter(ApiTokenFilter.java:63)
    at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
    at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:249)
    at hudson.security.HttpSessionContextIntegrationFilter2.doFilter(HttpSessionContextIntegrationFilter2.java:66)
    at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
    at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:76)
    at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:164)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
    at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:50)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
    at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:81)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
    at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
    at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:553)
    at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
    at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
    at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
    at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
    at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
    at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
    at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
    at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
    at org.eclipse.jetty.server.Server.handle(Server.java:497)
    at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
    at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:245)
    at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
    at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
    at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
    at java.lang.Thread.run(Thread.java:722)

[BogdanLivadariu]

have you restarted jenkins after you updated the jenkins plugin ?

[feiniao0308]

Yes, I did.

[feiniao0308]

I may miss another info. My jenkins is older, the verison is really older 1.480. I rebuilt the bootstraped-multi-test-results-report. Does it cause the compatible issue?

[BogdanLivadariu]

can't you update it to 1.602

[feiniao0308]

We have some customized plugins developed based on 1.48. So we can not update directly without testing. But it's strange to me, the only one small size xml could be processing successfully, why the larger one can not. Anyway, thanks a lot for the fixing of multiple xml file issue. I'm going to close this ticket. And try to setup a latest jenkins and parse the large xml file. Will file another ticket if it still has the problem. Thanks a lot!!!

[BogdanLivadariu]

can you send me a zip with the xml files to : [email protected] ?
I have tested with a larger result file, about 10 MB, and did not encountered any issues

[feiniao0308]

Send out, pls check it. Thanks!

[JuanRodriguezAbarca]

Hi I am having problems using this reporting tool, as the xml generated is wrong created. It is supposed to grab the testNG xml results and convert them. But here are the logs of the issue:

[TestNG test report builder] there was an error generating the reports: javax.xml.bind.UnmarshalException
 - with linked exception:
[com.sun.istack.internal.SAXParseException2; lineNumber: 2; columnNumber: 1; unexpected element (uri:"", local:"testsuite"). Expected elements are <{}exception>,<{}full-stacktrace>,<{}line>,<{}message>,<{}method>,<{}reporter-output>,<{}suite>,<{}test>,<{}test-method>,<{}testng-results>]
com.sun.xml.internal.bind.v2.runtime.unmarshaller.UnmarshallerImpl.handleStreamException(Unknown Source)
com.sun.xml.internal.bind.v2.runtime.unmarshaller.UnmarshallerImpl.unmarshal0(Unknown Source)
com.sun.xml.internal.bind.v2.runtime.unmarshaller.UnmarshallerImpl.unmarshal(Unknown Source)
com.github.bogdanlivadariu.reporting.testng.builder.TestNgReportBuilder.<init>(TestNgReportBuilder.java:57)
com.github.bogdanlivadariu.jenkins.reporting.testng.TestNGTestReportPublisher.perform(TestNGTestReportPublisher.java:126)
hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:779)
hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:726)
hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.post2(MavenModuleSetBuild.java:1053)
hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:671)
hudson.model.Run.execute(Run.java:1766)
hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:537)
hudson.model.ResourceController.execute(ResourceController.java:98)
hudson.model.Executor.run(Executor.java:381)
Build step 'Publish TestNG reports generated with handlebars' changed build result to FAILURE
Finished: FAILURE

I don´t know how to fix it, I´ll really appreciate some help.

Thank you very much, Juan

[BogdanLivadariu]

Can you open a new issue on this, and attach the xml file that is causing the plugin to throw errors ?

Thanks.