mrray / vvisf-gl Goto Github PK

Hi!

We have a repository for an AfterEffects plugin that was originally developed for Mac, working seamlessly with VVISF-GL. However, when we ported it to Windows, we encountered issues with the Multi-Frame-Rendering mode, which implies multithreading. When rendering occurs in the background, the application crashes due to the plugin since cpuBackingPtr becomes nullptr after GLTexToCPUCopier::downloadTexToCPU. Checking the GLERRLOG, I noticed that even before this, within the same cycle, there's an issue with glBindBuffer(inPBOBuffer->desc.target, inPBOBuffer->name); here inside the GLCPUToTexCopier::_beginProcessing function. I'm currently uncertain about the root cause. Can you take a look at our code?

We're using the following approach:

In the GlobalSetup function (initialized once during the effect's initial setup), we create a context, BufferPool, and scenes:

#ifdef _WIN32
GLContext::bootstrapGLEnvironmentIfNecessary();
// VVGL on Windows Doesn't have pixel format functions
globalData->context = VVGL::CreateNewGLContextRef(nullptr, nullptr);
#else
globalData->context = VVGL::CreateNewGLContextRef(NULL, CreateCompatibilityGLPixelFormat());
#endif
VVGL::CreateGlobalBufferPool(globalData->context->newContextSharingMe());

globalData->downloader = VVGL::CreateGLTexToCPUCopierRefUsing(globalData->context->newContextSharingMe());
globalData->uploader = VVGL::CreateGLCPUToTexCopierRefUsing(globalData->context->newContextSharingMe());

globalData->defaultScene = VVISF::CreateISF4AESceneRefUsing(globalData->context->newContextSharingMe());
globalData->defaultScene->useCode(SystemUtil::readResourceShader(IDR_DEFAULT_FS), "");

The main work then occurs in SmartRender, which is executed on every render frame, regardless of whether the threads are background or main.

We take the primary layer (essentially an RGBA image) and convert it into a buffer based on color bit depth (VVGL::CreateRGBACPUBufferUsing, VVGL::CreateRGBAShortCPUBufferUsing or VVGL::CreateRGBAFloatCPUBufferUsing) through the createRGBACPUBufferWithBitdepthUsing function using GlobalBufferPool: VVGL::GLBufferPoolRef bp = VVGL::GetGlobalBufferPool();. Then, we upload the image via the uploader to the texture:

VVGL::GLBufferRef imageAECPU = createRGBACPUBufferWithBitdepthUsing(bufferSizeInPixel, layerDef->data, imageSize, bitdepth);

auto imageAE = globalData->uploader->uploadCPUToTex(imageAECPU);

// Note that AE's inputImage is cropped by mask's region and smaller than ISF resolution.
glBindTexture(GL_TEXTURE_2D, imageAE->name);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_BORDER);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_BORDER);
glBindTexture(GL_TEXTURE_2D, 0);

auto origin = VVISF::ISFVal(VVISF::ISFValType_Point2D, layerDef->origin_x, layerDef->origin_y);

globalData->ae2glScene->setBufferForInputNamed(imageAE, "inputImage");
globalData->ae2glScene->setValueForInputNamed(origin, "origin");

outImage = createRGBATexWithBitdepth(outImageSize, globalData->context, bitdepth);

globalData->ae2glScene->renderToBuffer(outImage);

// Though ISF specs does not specify the wrap mode of texture, set it to CLAMP_TO_EDGE to match with online ISF
// editor's behavior.
glBindTexture(GL_TEXTURE_2D, outImage->name);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);
glBindTexture(GL_TEXTURE_2D, 0);

// ...

input->setCurrentImageBuffer(outImage);

At this point, if the render is on a background thread, there will be an error with glBindBuffer in the void GLCPUToTexCopier::_beginProcessing function. If we set an assert, it will trigger only in these scenarios:

glBindBuffer(inPBOBuffer->desc.target, inPBOBuffer->name);
if (glGetError() != 0) {
    assert("Cannot Bind to Buffer" && false);
}

Following the renderISFToCPUBuffer code which executed from SmartRender, we then attempt to retrieve the OpenGL execution result from the texture, download the result, and free the resources:

// Render ISF
auto isfImage = createRGBATexWithBitdepth(outSize, globalData->context, bitdepth);

// ...

scene.renderToBuffer(isfImage, outSize, time);

// ...

// Download the result of ISF
auto& gl2aeScene = *globalData->gl2aeScene;

gl2aeScene.setBufferForInputNamed(isfImage, "inputImage");

auto outputImage = createRGBATexWithBitdepth(outSize, globalData->context, bitdepth);
gl2aeScene.renderToBuffer(outputImage);

(*outBuffer) = globalData->downloader->downloadTexToCPU(outputImage);

// Release resources
VVGL::GetGlobalBufferPool()->housekeeping();

Are we handling everything correctly? Have we overlooked anything? How should the VVGL lifecycle proceed in a multi-thread rendering mode?

Thank you!

Hello,
I use VVISF inside a cinder project. https://github.com/oogre/ORAGE/tree/ISF
I only use ISFDoc::generateShaderSource and ISFAttr.
My integration is working without problem on OSx,
I'm struggling to make it work on Windows.
Currently I receive this error message

cinder.lib(ImageSourceFileStbImage.obj) : error LNK2005: "void __cdecl stbi__tga_read_rgb16(struct stbi__context *,unsigned char *)" (?stbi__tga_read_rgb16@@YAXPEAUstbi__context@@PEAE@Z) already defined in GLBufferPool.obj

It seems there is a conflict with stb_image.h which is used by cinder and by VVGL.

From your point of view what's the best solution :

• modify VVISF-GL to split VVISF from VVGL (cause I don't need VVGL) ?
• use VVISF-GL in a small app for the interpretation and implement my own ISFAttr inside my app
• compile VVISF-GL inside cinder to handle the conflict
• make a VVISF-GL branch to turn it into cinder block
• something else with preprocessor magic to handle this conflict

What can I do to debug this conflict

Have a nice day

• make a couple changes in the text editor
• add or delete an INPUT using the GUI
• note that the changes made in the text editor no longer exist

as far as i can tell, this is a bug with Qt specific to the mac platform (the windows version doesn't have this issue). this bug has been documented and reported to Qt- if this bothers you, please consider giving more visibility to this issue:

https://bugreports.qt.io/browse/QTBUG-71776?jql=text%20~%20qopenglwidget%20ORDER%20BY%20created%20DESC

...the only workaround i'm aware of now is to simply restart the application- this bug only occurs occasionally on launch!

Currently using ISF Editor Windows beta 2.9.11.0. I have a "Network Error 99" consistently. What does 99 mean, assuming its not a catch-all network error. Get it when importing from ShaderToy but not from GL Sandbox.

Hello

I can't put the folder examples (ISFSandbox.xcodeproj) with GLFW to run. Already included the library (had to download). Any tips? The ISF editor compiles perfectly

Thanks!

When using the Windows ISF beta on a 4K monitor the text size of the source code, is very very small. I can't seem to be able to increase the text size anywhere. Otherwise thanks for porting to Windows!

Hello, I've been trying to compile the Raspbian example using GLES 3,2 by copying GLBuffer_Enums_IOS.h to GLBuffer_Enums_RPI.h

Do you think it would be possible to update the example and add a target VVGL_SDK_RPI4?

Hello there,

I tried using the ISFEditor to convert some of my shaders to ISF format but apparently there are a few issues
Here's the shader I tried
https://www.shadertoy.com/view/7s2Sz3

The errors I spotted :

• It does not import the common tab (it's probably easy to fix as it simply requires code concatenation)
• When doing multipass shaders, it does not handle multiple functions having the same name (would require a kind of mangling / namespace to the names)
  Here's a detailed list of the errors I got from the editor

Fragment shader errros:

0(41) : error C1038: declaration of "iChannel2" conflicts with previous declaration at 0(15)
0(42) : error C1038: declaration of "_iChannel2_imgRect" conflicts with previous declaration at 0(16)
0(43) : error C1038: declaration of "_iChannel2_imgSize" conflicts with previous declaration at 0(17)
0(44) : error C1038: declaration of "_iChannel2_flip" conflicts with previous declaration at 0(18)
0(91) : error C1503: undefined variable "r2d"
0(93) : error C1503: undefined variable "sat"
0(100) : error C1503: undefined variable "r2d"
0(106) : error C1503: undefined variable "r2d"
0(107) : error C1503: undefined variable "sat"
0(173) : error C1503: undefined variable "sat"
0(215) : error C1503: undefined variable "sat"
0(215) : error C1503: undefined variable "sat"
0(234) : error C1503: undefined variable "sat"
0(235) : error C1503: undefined variable "sat"
0(244) : error C1503: undefined variable "sat"
0(249) : error C1503: undefined variable "sat"
0(249) : error C1503: undefined variable "sat"
0(264) : error C1503: undefined variable "sat"
0(265) : error C1503: undefined variable "sat"
0(300) : error C1503: undefined variable "sat"
0(300) : error C1503: undefined variable "sat"
0(303) : error C1503: undefined variable "r2d"
0(306) : error C1503: undefined variable "r2d"
0(325) : error C1503: undefined variable "iChannel0"
0(364) : error C1013: function "rdr" is already defined at 0(262)