BIRT 4.17, designer & Spudsoft excel: Spudsoft XLS & XLSX not longer created from designer-preview & designer-viewer about birt

[merks]

Wow, it's really kind of a broken framework design. If the org.apache.logging.log4j.util.Activator is started, it lock ProviderUtil.STARTUP_LOCK and then later it unlocks it if and only if there is a provider.

Then, if a different thread comes to org.apache.logging.log4j.util.ProviderUtil.lazyInit() it will find that ProviderUtil.STARTUP_LOCK is locked and will block indefinitely:

How broken is that?

---

I'm still concerned whether I'm seeing the same problem as @speckyspooky or a different problem.

The early activation appears to be a side-effect of the start levels of the product and although we might change those, if BIRT is installed in an Eclipse IDE, I would expect the same problem to arise.

[merks]

I can reproduce the problem in the IDE when the start levels of the debug launch are like those specified in the product:

The key to the problem is the early start thread holding the lock. I have no idea what's (if anything) has changed. But it's a fundamental problem in the logging framework and I really want a fix that will allow BIRT to run in any IDE.

In any case, it's easy to reproduce now and I have prototyped horrible, sickening hack in uk.co.spudsoft.birt.emitters.excel.framework.ExcelEmitterPlugin.start hat does actually fix the problem at least when debugging:

I will build the all-in-one product locally and verify that the launched unzipped product works before proceeding with a PR.

[merks]

Yes, with this approach the following information is logged on startup about all the services that are registered:

Aug. 17, 2024 4:58:07 PM org.apache.aries.spifly.BaseActivator log
INFO: Registered provider org.slf4j.simple.SimpleServiceProvider of service org.slf4j.spi.SLF4JServiceProvider in bundle slf4j.simple
Aug. 17, 2024 4:58:08 PM org.apache.aries.spifly.BaseActivator log
INFO: Registered provider org.apache.logging.log4j.util.EnvironmentPropertySource of service org.apache.logging.log4j.util.PropertySource in bundle org.apache.logging.log4j.api
Aug. 17, 2024 4:58:08 PM org.apache.aries.spifly.BaseActivator log
INFO: Registered provider org.apache.logging.log4j.util.SystemPropertiesPropertySource of service org.apache.logging.log4j.util.PropertySource in bundle org.apache.logging.log4j.api
Aug. 17, 2024 4:58:08 PM org.apache.aries.spifly.BaseActivator log
INFO: Registered provider org.apache.logging.slf4j.SLF4JProvider of service org.apache.logging.log4j.spi.Provider in bundle org.apache.logging.log4j.to.slf4j

And then we see POI doing actual logging, which might be useful in the future for tracking down problems:

!ENTRY uk.co.spudsoft.birt.emitters.excel 1 0 2024-08-17 16:59:33.259
!MESSAGE start:org.eclipse.birt.report.engine.content.impl.ReportContent@7d24dcb3
[qtp805898248-117] WARN org.apache.poi.POIDocument - DocumentSummaryInformation property set came back as null
[qtp805898248-117] WARN org.apache.poi.POIDocument - SummaryInformation property set came back as null

Because there is now Provider, the deadlock is avoided.

[speckyspooky]

@merks Merci beaucoup!

I have done the retest with the latest build version BIRT 4.17, N202408171542.
The retest is successfully and the excel output works now perfect. The excel documents (XLSX & XLS) are created normally.

[speckyspooky]

The other output formats like PDF, DOCX, PPTX, ODT, ODS and ODP can be created normally.

[speckyspooky]

Retest done with BIRT 4.17 N202408170550 with the same result:

1. all-in-one-designer
   1.1 all-in-one-designer, create the excel directly: excel = error
   1.2 all-in-one-designer, start preview and afterwards create the excel: preview = ok, excel = error

2. OSGi runtime

• create the demo at viewer level and afterwards create the excel: preview = ok, excel = ok

3. Local environment

• 2 test cases (see above): preview = ok, excel = ok

(I have tested with FireFox & Edge.)

[merks]

For me it just hangs. When I exit the application, which I started with -consoleLog, the log contains the following, which suggests that POI is waiting for a logging service

2024-08-17T06:53:11.114430900Z qtp1861377027-100 FATAL Interrupted before Log4j Providers could be loaded. java.lang.InterruptedException
        at java.base/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer.java:1011)
        at java.base/java.util.concurrent.locks.ReentrantLock$Sync.lockInterruptibly(ReentrantLock.java:161)
        at java.base/java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:372)
        at org.apache.logging.log4j.util.ProviderUtil.lazyInit(ProviderUtil.java:151)
        at org.apache.logging.log4j.util.ProviderUtil.hasProviders(ProviderUtil.java:138)
        at org.apache.logging.log4j.LogManager.<clinit>(LogManager.java:89)
        at org.apache.poi.POIDocument.<clinit>(POIDocument.java:65)
        at uk.co.spudsoft.birt.emitters.excel.XlsEmitter.createWorkbook(XlsEmitter.java:49)
        at uk.co.spudsoft.birt.emitters.excel.ExcelEmitter.start(ExcelEmitter.java:343)
        at org.eclipse.birt.report.engine.api.impl.RenderTask$PageRangeRender.render(RenderTask.java:531)
        at org.eclipse.birt.report.engine.api.impl.RenderTask.render(RenderTask.java:278)
        at org.eclipse.birt.report.service.ReportEngineService.renderReport(ReportEngineService.java:1263)
        at org.eclipse.birt.report.service.BirtViewerReportService.renderReport(BirtViewerReportService.java:303)
        at org.eclipse.birt.report.service.actionhandler.BirtRenderReportActionHandler.__execute(BirtRenderReportActionHandler.java:63)
        at org.eclipse.birt.report.service.actionhandler.AbstractBaseActionHandler.execute(AbstractBaseActionHandler.java:88)
        at org.eclipse.birt.report.presentation.aggregation.layout.FramesetFragment.doService(FramesetFragment.java:190)
        at org.eclipse.birt.report.presentation.aggregation.layout.FramesetFragment.service(FramesetFragment.java:80)
        at org.eclipse.birt.report.servlet.ViewerServlet.__doGet(ViewerServlet.java:167)
        at org.eclipse.birt.report.servlet.BirtSoapMessageDispatcherServlet.doPost(BirtSoapMessageDispatcherServlet.java:233)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:523)
        at org.apache.axis.transport.http.AxisServletBase.service(AxisServletBase.java:332)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:590)

When I try to move the initialized then it hangs there:

2024-08-17T09:25:30.232295400Z qtp466409782-84 FATAL Interrupted before Log4j Providers could be loaded. java.lang.InterruptedException
	at java.base/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer.java:1011)
	at java.base/java.util.concurrent.locks.ReentrantLock$Sync.lockInterruptibly(ReentrantLock.java:161)
	at java.base/java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:372)
	at org.apache.logging.log4j.util.ProviderUtil.lazyInit(ProviderUtil.java:151)
	at org.apache.logging.log4j.util.ProviderUtil.hasProviders(ProviderUtil.java:138)
	at uk.co.spudsoft.birt.emitters.excel.ExcelEmitter.start(ExcelEmitter.java:323)

I have developed a strong dislike for logging frameworks the past weeks So much complexity and then strange incompatibilities when different frameworks use different libraries/versions.

I think I would need to attach a remove debugger to see why there is apparently a deadlock and I have to hope this is the same problem as @speckyspooky is seeing.

[merks]

Even moving it to the bundle start then hangs there:

2024-08-17T10:40:29.124930200Z qtp324477266-116 FATAL Interrupted before Log4j Providers could be loaded. java.lang.InterruptedException
	at java.base/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer.java:1011)
	at java.base/java.util.concurrent.locks.ReentrantLock$Sync.lockInterruptibly(ReentrantLock.java:161)
	at java.base/java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:372)
	at org.apache.logging.log4j.util.ProviderUtil.lazyInit(ProviderUtil.java:151)
	at org.apache.logging.log4j.util.ProviderUtil.hasProviders(ProviderUtil.java:138)
	at uk.co.spudsoft.birt.emitters.excel.framework.ExcelEmitterPlugin.start(ExcelEmitterPlugin.java:60)
	at org.eclipse.osgi.internal.framework.BundleContextImpl$2.run(BundleContextImpl.java:833)
	at org.eclipse.osgi.internal.framework.BundleContextImpl$2.run(BundleContextImpl.java:1)

[speckyspooky]

The situation is strange and I don't know what is the reason of this changed behavior may be it is a side-effect with 2024-09
and only in combination with the IDE.
So I doubled checked my IDE-dev-console and see the following line which could be the confirmation of your finding:

But I don't know why it is working from eclipse-IDE. Could it be that the process handling is another one or there is later a logger given and will be used...

[merks]

I think I have a much better/nicer way, i.e., ensure there is a dependency on org.apache.logging.slf4j. Given that the product is using slf4j.simple, that should actually be better and allow POI to do actual logging. I will try that approach now. It works in debug mode, so it looks promising!

[speckyspooky]

It looks very nice. I will do me retest with the new version.

[wimjongman]

Good news coming back from vacation. Thanks, guys.