See: openwebwork/webwork2#1120 by @Alex-Jordan which proposes adding a "raw" output format which provides a JSON object with essentially everything the renderer produced, to allow a client system access to the structured data for additional processing.

Such an output format may be quite useful in the contexts where the standalone renderer can be of use, assuming the system making the request can do significant post-processing.

According to the documentation, specifically the README.md file, the name of the API parameter for specifying the output format is outputFormat, with uppercase F. That does not seem to work, and when grepping the source for outputFormat, the line in README.md is the only one that comes up, The all lowercase version, outputformat, seems to work, though.

Description of Bug

After starting the container, the application in the web browser is not accessible but the error message raptor not found. shows up, see the screenshot below

Steps to Reproduce

Use the current repository on Linux Mint 20.2 and carry out the docker installation instructions as provided in README.md.

Further Infos

I used git bisect to find the commit that may have broken the app:

6307b7123cd61bdbdccd43716cff5d65ab9e34e8 is the first bad commit
commit 6307b7123cd61bdbdccd43716cff5d65ab9e34e8
Author: K. Andrew Parker <[email protected]>
Date:   Thu Jul 15 16:03:51 2021 -0400

    minimal access in production mode

 lib/RenderApp.pm                       | 48 ++++++++++++--------
 lib/RenderApp/Controller/Render.pm     | 80 ++++++++++++++++++----------------
 templates/exception.production.html.ep | 27 ++++++++++++
 3 files changed, 100 insertions(+), 55 deletions(-)
 create mode 100644 templates/exception.production.html.ep

Here ist the complete trajectory of my git bisect

container$ git bisect log
git bisect start
# bad: [d2d844efa2f7a2a651b1ba91cfb49e5ff48a84a1] Merge pull request #62 from drdrew42/feature/strict-production-mode
git bisect bad d2d844efa2f7a2a651b1ba91cfb49e5ff48a84a1
# good: [cb433f7607965c4f0a9c6c9b97be0a4e17ddaf9c] add jwt/jwe-generator endpoints
git bisect good cb433f7607965c4f0a9c6c9b97be0a4e17ddaf9c
# good: [084fec0860edc4f83dd6d5c3b58d246fda7c4dc0] Merge pull request #60 from pstaabp/update-install-instructions
git bisect good 084fec0860edc4f83dd6d5c3b58d246fda7c4dc0
# bad: [6307b7123cd61bdbdccd43716cff5d65ab9e34e8] minimal access in production mode
git bisect bad 6307b7123cd61bdbdccd43716cff5d65ab9e34e8
# good: [7fc2e5cf40d5cf8d51df7a8b4796e82968fd915b] Merge pull request #61 from drdrew42/feature/jwt-jwe-api
git bisect good 7fc2e5cf40d5cf8d51df7a8b4796e82968fd915b
# first bad commit: [6307b7123cd61bdbdccd43716cff5d65ab9e34e8] minimal access in production mode

drdrew42, thank you for creating this repo! It is super helpful to look at a smaller cut of the Webwork functionality.

We are running an Ubuntu server. We need to convert problem text string to an HTML text string as a utility function for this server. The simple thing would be to just call a perl function with some PGML (+ other parameters) and get back the response.

It seemed like it would be a simple task to write a module to call several methods in Problem.pm serially, eventually getting back the result. That is not working, simply because I know so freakin' little Perl script. (Maybe other reasons too.) I am not setting up the parameters right.

If you have any suggestions, I will eagerly take them. Even a few lines of untested code that are in the right ballpark would advance the cause.

There is always the fallback of running the Webwork renderer against a port, as this repo instructs, and connecting our existing web server to it. Pipe the conversion dialog through http.

Running the Dockerfile now fails with the following error:

Can't load application from file "/usr/app/script/render_app": Route pattern "/webwork2_files/*path" contains a reserved stash value at /usr/local/share/perl/5.30.0/Mojolicious/Routes/Route.pm line 222.
        Mojolicious::Routes::Route::_route(Mojolicious::Routes=HASH(0x55c47cde1098), "/webwork2_files/*path") called at /usr/local/share/perl/5.30.0/Mojolicious/Routes/Route.pm line 197
        Mojolicious::Routes::Route::_generate_route(Mojolicious::Routes=HASH(0x55c47cde1098), ARRAY(0x55c480a8c4b0), "/webwork2_files/*path", CODE(0x55c480a8bf10)) called at /usr/local/share/perl/5.30.0/Mojolicious/Routes/Route.pm line 39
        Mojolicious::Routes::Route::any(Mojolicious::Routes=HASH(0x55c47cde1098), "/webwork2_files/*path", CODE(0x55c480a8bf10)) called at /usr/app/lib/RenderApp.pm line 90
        RenderApp::startup(RenderApp=HASH(0x55c47e47a868)) called at /usr/local/share/perl/5.30.0/Mojolicious.pm line 164
        Mojolicious::new("RenderApp") called at /usr/local/share/perl/5.30.0/Mojo/Server.pm line 18
        Mojo::Server::build_app(Mojo::Server=HASH(0x55c47e47a760), "RenderApp") called at /usr/local/share/perl/5.30.0/Mojolicious/Commands.pm line 82
        Mojolicious::Commands::start_app("Mojolicious::Commands", "RenderApp") called at /usr/app/script/render_app line 11
        require /usr/app/script/render_app called at (eval 64) line 1
        eval 'package Mojo::Server::Sandbox::5e71eca6a9880a56092744e063afc268; require $path' called at /usr/local/share/perl/5.30.0/Mojo/Server.pm line 56
        Mojo::Server::load_app(Mojo::Server::Prefork=HASH(0x55c47ca69f38), "/usr/app/script/render_app") called at /usr/local/share/perl/5.30.0/Mojo/Server/Hypnotoad.pm line 51
        Mojo::Server::Hypnotoad::run(Mojo::Server::Hypnotoad=HASH(0x55c47ca4a650), "/usr/app/script/render_app") called at /usr/local/bin/hypnotoad line 14
Compilation failed in require at (eval 64) line 1.

The only references I've found were this issue which seems accurate to the cause of the problem. I can start the Dockerfile with the following patch as a workaround:

index 468810cd..c1527fa4 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -35,7 +35,7 @@ RUN apt-get update \
     && apt-get clean \
     && rm -fr /var/lib/apt/lists/* /tmp/*
 
-RUN cpanm install Mojo::Base Statistics::R::IO::Rserve Date::Format Future::AsyncAwait \
+RUN cpanm install https://cpan.metacpan.org/authors/id/S/SR/SRI/Mojolicious-8.73.tar.gz Statistics::R::IO::Rserve Date::Format Future::AsyncAwait \
     && rm -fr ./cpanm /root/.cpanm /tmp/*
 
 ENV MOJO_MODE=production```

I think the render should probably be able to handle the new control settings be added in: openwebwork/webwork2#1174 :

• $pg{specialPGEnvironmentVars}{parseAlternatives}
• $pg{specialPGEnvironmentVars}{convertFullWidthCharacters}
  The settings and the changes to PG are intended to support the use of some Unicode characters are equivalents for "standard" ones, which will be optionally handled by PG in PG 2.16 (PR to develop at present).

See:

• openwebwork/pg#517
• openwebwork/pg#518

I have tried to look at whether the current version can handle PG files with UTF-8 encoded characters and ran into quite a bit of trouble. I just put a branch on my fork with some modifications which seem to help.

1. When a problem with UTF-8 characters is stored in a location where it can be loaded with the Load button, what appears in the Editor window is mangled. The UTF-8 text fails to appear properly - mojibake shows up instead. However when the problem is rendered - the text appears properly.
   • I suspect that something like $self->{problem_contents} = Encode::decode( "UTF-8", $self->{read_path}->slurp ); should be used in the load method in lib/RenderApp/Model/Problem.pm.
   • With that change - I do see the problem text loaded properly in the editor window, but then the problem fails to render, and get the same error as mentioned in item 2.
2. If UTF-8 text is typed into the problem text, the characters appear properly in the Editor but the problem fails to render. In such cases, the JavaScript console reported: navbar.js:87 Uncaught DOMException: Failed to execute 'btoa' on 'Window': The string to be encoded contains characters outside of the Latin1 range. at HTMLButtonElement.<anonymous> (http://localhost:3000/navbar.js:87:40)
   • The problem is that btoa is restricted to 8-bit characters. See: https://developer.mozilla.org/en-US/docs/Glossary/Base64
   • One possible replacement is https://github.com/dankogai/js-base64 which provides a Unicode compatible replacement for btoa, but first the JavaScript UTF-16 string needs to be converted to UTF-8.
   • For the conversion to UTF-8, the approach my changes propose using makes user of TextEncoder - see: https://developer.mozilla.org/en-US/docs/Web/API/TextEncoder and https://encoding.spec.whatwg.org/#dom-textencoder
   • See also https://stackoverflow.com/questions/23223718/failed-to-execute-btoa-on-window-the-string-to-be-encoded-contains-characte
3. Saving UTF-8 text typed in from the editor also fails.
   • Probably there is need to encode into UTF-8 at write time, as spurt seems to expect a stream of bytes:
     • https://metacpan.org/pod/Mojo::File#spurt
   • I suspect that something like $write_path->spurt( Encode::encode('UTF-8', $self->{problem_contents} ) ); should be used in lib/RenderApp/Model/Problem.pm, but that did not suffice.
   • There was also a need to decode from UTF-8 after the base64 decode.

Have a look at the branch at https://github.com/taniwallach/renderer/tree/utf8-support-v1 which seems to fix all of these issues.

Small sample UTF-8 problem text:

DOCUMENT();
loadMacros(
  "PGstandard.pl",
);
BEGIN_TEXT
שלום
END_TEXT
ENDDOCUMENT();

I would recommend pruning all branches that have either been merged in or are otherwise stale. As far as I can tell in the Git tree, only the master branch needs to remain since everything else appears to be already merged.

In addition, in repository settings I would recommend you turn on Automatically delete head branches as after a pull request is merged it will automatically prune the associated branch.

Add a flag showComments that when set to true displays the output of the COMMENT() function.

For Docker, I would recommend for each stable release to have a prebuilt image on Dockerhub or GitHub Container Registry etc. for easy initial deployment. This means that users wouldn't even have to download the source to get started as they can just docker pull.

I just build the container using the current master (is at commit 30d3526).

I tried to load:

• Library/SDSU/Discrete/IntegersAndRationals/basemulttableB5.pg
  and got the error message:

Could not reach the API: Precondition Failed

It took me some time to realize that the issue was that there was a training space in the filename box.

I recommend stripping leading/trailing whitespace from the box (on the client side) before running the Load routine.

The line

git clone --recursive https://github.com/rederly/renderer container/

in README.md probably needs to be replaced by

git clone --recursive https://github.com/drdrew42/renderer container/

However, maybe the main repo should be transferred to openwebwork. I just made a standaloneRendered team there, but as team creator - I'm on it. https://github.com/orgs/openwebwork/teams/standalonerenderer/members

A PG file with a syntax error causes minimal debugging data to be provided.(Tested with master from 2021-Jan-18. at 30d3526)

Below is a small example (with an intentional bug).

• Uncommenting the "MathObjects.pl", line fixes the issue in the PG code.
• But the issue with the standalone renderer is that it fails to provide the debugging output.

DOCUMENT();
loadMacros(
  "PGstandard.pl",
#  "MathObjects.pl",
);

BEGIN_TEXT
Calculate \( 2 \cdot 3 \)
and enter the result: \{ ans_rule(4) \}.
END_TEXT

ANS( Real(6)->cmp() );

ENDDOCUMENT();

The rendered problem section shows:

        Problem 1
2. ERROR caught by Translator while processing this problem 

The developer mode shows the response as containing the following in the JSON structure:

"debug" : {
  "debug":null,
  "internal":["Error in obtaining debug messages from PGcore"],
  "perl_warn":"",
  "pg_warn":null,
  "render_warn":[]
},

When a regular WW 2.15 server tries to render this problem, the error output is:

        Problem1
1. ERROR caught by Translator while processing problem file:5.pg
****************
ERRORS from evaluating PG file: 
 Undefined subroutine &main::Real called at line 12 of (eval 3840)



****************

------Input Read
1		DOCUMENT();
2		loadMacros(
3		  "PGstandard.pl",
4		#  "MathObjects.pl",
5		);
6		
7		BEGIN_TEXT
8		Calculate \( 2 \cdot 3 \)
9		and enter the result: \{ ans_rule(4) \}.
10		END_TEXT
11		
12		ANS( Real(6)->cmp() );
13		
14		ENDDOCUMENT();

https://github.com/rederly/renderer/blob/59f8cc2bb6abe74b36692d5c3d33ca41dce1eddb/lib/WeBWorK/lib/WeBWorK/PG.pm#L97

Incoming

• render requests via problemJWT
• Endpoint for creating problemJWT

Future

• swap out problemJWT from inside sessionJWT

Initialization

• create sessionJWT from incoming problemJWT request
• retain past answer logs (drop MaThQuIlL_ & others) as array
• support caching of KEPT_EXTRA_ANSWERS
• stop updating sessionJWT once status == max_score
• derive attempts data from sessionJWT if present

Usage

• support forward/backward navigation through past answers
• override problemJWT setting so that past answer view uses static template

Testing

• confirm that problemJWT settings are enforced 'through' the sessionJWT
• reload a problem's state from scratch with a sessionJWT
• confirm that sessionJWT stops updating once status == max_score
• test showHints vs attempts with sessionJWT

I recommend adding a "Clear" button to the top control bar.
It should clear the editor data, and the filename.
Also the header with the name of the file being edited (which appears above the editor window).

The master branch was properly handling problemJWT being sent by the XBlock under development as of commit d2d844e (Merge pull request #62 from drdrew42/feature/strict-production-mode), but now (after commits from Aug 22) it is not.

In the Docker logs for the container I see:

[2021-09-01 06:35:06.27719] [21] [trace] [tPiMnSYHXP7n] POST "/render-api"
[2021-09-01 06:35:06.27744] [21] [trace] [tPiMnSYHXP7n] Routing to controller "RenderApp::Controller::Render" and action "problem"
[2021-09-01 06:35:06.27784] [21] [info] [tPiMnSYHXP7n] Received JWT: using problemJWT
problemSeed, sourceFilePath, numIncorrect, showComments, showHints, psvn, outputFormat, problemSourceURL, format, answersSubmitted, language, displayMode, showSolutions, permissionLevel, numCorrect, problemJWT, showSummary, includeTags, processAnswers, problemSource, aud at /usr/app/lib/RenderApp/Controller/Render.pm line 64.
[2021-09-01 06:35:06.27866] [21] [info] [tPiMnSYHXP7n] CREATED: Problem created from webwork-open-problem-library/OpenProblemLibrary/Dartmouth/setMTWCh2S4/problem_5.pg with random seed #1795344405
[2021-09-01 06:35:06.35775] [21] [debug] [tPiMnSYHXP7n] 500 Render failed: Can't locate object method "decode" via package "RenderApp::Model::JWT" (perhaps you forgot to load "RenderApp::Model::JWT"?) at /usr/app/lib/RenderApp/Controller/RenderProblem.pm line 480.
[2021-09-01 06:35:06.35794] [21] [warn] [tPiMnSYHXP7n] Render failed: Can't locate object method "decode" via package "RenderApp::Model::JWT" (perhaps you forgot to load "RenderApp::Model::JWT"?) at /usr/app/lib/RenderApp/Controller/RenderProblem.pm line 480.
[2021-09-01 06:35:06.35826] [21] [debug] [tPiMnSYHXP7n] 500 Internal Server Error (0.081055s, 12.337/s)
[2021-09-01 06:35:06.35861] [21] [error] [tPiMnSYHXP7n] TRASH: [80.1ms] render from webwork-open-problem-library/OpenProblemLibrary/Dartmouth/setMTWCh2S4/problem_5.pg failed with error: Internal Server Error

The problem may be related to the fact that 6b0e059#diff-d7ce13f075eeb062a67c5ad1b95701a86ad1d0e80679b0f12d14e39e7c31ec97 has

    my $session = RenderApp::Model::JWT->decode($inputs_ref->{sessionJWT});

but it seems that #71 which is still a pending PR contains lib/RenderApp/Model/JWT.pm which is a new file.

Bottom line the current status of the master branch is broken.

Hello!
According to Git, it appears that the PG submodule reference is still pointing to rederlyhq/pg@fd85c5e rather than a commit on https://github.com/openwebwork/pg. This can be confirmed by running git submodule status. While it looks like the submodule has been preliminarily updated in the .gitmodules file at 0148cd6, Git requires you to run some git submodule commands to complete the changeover.

Also, I am unsure if there are features on the Rederly fork that need to be merged into openwebwork/pg before you finish retargeting the submodule.

Renderer needs to accept some level of permission in the render request to make scaffold problems behave as expected.

   # only supply key-values that are not already provided
    # e.g. numCorrect/numIncorrect or restarting an interrupted session
    foreach my $key (keys %$claims) {
      $params{$key} ||= $claims->{$key};
    }

Possible bug?:
if the current value of the params is supposed to be zero (or blank) and it will still be overwritten by a non- zero value from claims even if this is not desired. Using // will replace the params value only if it is actually undefined (as opposed to being zero)

Add modes which generate LaTeX and PreTeXt output.

The main project is making some changes to applet support, and this includes adding a new ADD_JS_FILE() method to PG, and support to handle the new change on the webwork2 site.

Similar support will need to be added to this project. The PG changes are likely to come in when a newer version of PG is included via the sub-module mechanism. However, the webwork2 side changes need to be made in the renderer codebase.

• openwebwork/pg#489
• openwebwork/webwork2#1121

In particular see the changes to

• lib/WeBWorK/ContentGenerator/Problem.pm
• lib/WeBWorK/ContentGenerator/GatewayQuiz.pm

Clicking the Load button when no template is selected works, and will load the PG file. However, if the Render contents of editor button is clicked when no value of template is set, the "problem display" section displays "Loading..." and nothing loads.

It would be nice if this behaved more nicely.

I suggest either reporting that a Template must be selected, or defaulting to one of the templates and maybe reporting that a default template was chosen.

The value of psvn is used to allow several problems to share randomization with a common seed. See: https://webwork.maa.org/wiki/LinkingProblems and is used in close to 200 OPL (including Contrib) problems.

It would be nice to have the standalone system accept a provided value of psvn in addition to problemSeed but fall back to a default value when none is provided.

Apparently one modification would be near:
https://github.com/rederly/renderer/blob/6ebd63e07134107856f59304b2731b774ee7273b/lib/RenderApp/Controller/Render.pm#L8
but the hard-wired value in
https://github.com/rederly/renderer/blob/6ebd63e07134107856f59304b2731b774ee7273b/lib/RenderApp/Controller/RenderProblem.pm#L326
may need to be overridden.

An explanation of how the "trick" the Standalone renderer into displaying an incorrect image file is at the bottom of this post.

The problem has to do with how the directory name under which the images (and other auxiliary files) are stored is generated.

The issue is quite similar to what was discussed in https://webwork.maa.org/moodle/mod/forum/discuss.php?d=4615 (which still occurs on the page http://spot.pcc.edu/math/APEXCalculus/sec_imp_deriv.html as of 07-JUN-2021).

A partial solution to this issue was for webwork2/pg was attempted in:

• openwebwork/pg#415
• openwebwork/webwork2#976
  but still requires extra effort from the user of the html2xml subsystem of webwork.

The root cause is that create_unique_id from PGresource.pm which generates the second portion of the UID used in the directory name where images are generated depends on the filename (and path), and the first portion generated as unique_id_stub in PGalias.pm depends the items:

• studentLogin
• psvn
• courseID
• setNumber
• probNum
• problemSeed
• problemUUID
  most of which are irrelevant (constant) in the context of the standalone rendered (and html2xml). When problem code is provided (at present) from "outside" to the Standalone renderer (or to html2xml) without some suitable value changing (ex. the use of a different problemSeed or psvn) the data available does not suffice to set different directory names.

problemUUID was provided to allow the "end user" system to set a value to differentiate between different code which could otherwise be confused. However, simple use of the Standalone renderer for problem coding/testing is not likely to get end users to manually provide different values of problemUUID when needed, and not all users of the Standalone renderer for embedding problems will realize the importance of problemUUID when sending the question to render via problemSource.

It seems that a reasonable partial fix which can be implemented relatively easily when the problem source code is being provided via problemSource would be to use that data to calculate a hash-value which can be automatically set as problemUUID (if a value is not provided). (It seems that the same approach would be useful also for webwork2's html2xml code.) This is likely to work reasonable well, so long as the provided PG code is not loading other files of problem code which could change without the system (create_unique_id) being able to detect such a change. Although not 100% reliable, it is very likely "good enough" for almost all use cases.

It is quite likely that similar issues will effect cases when the PG code is provided via a problemSourceURL.

Persisting the value of problemUUID for return calls as is done in html2xml could also probably be of value, both for when used by external systems which can generate suitable UUID values, and for when a value was calculated based on the problemSource data.

It seems somewhat reasonable to expect remote systems which provide a non-trivial value for problemUUID to then be responsible to avoid conflicts. However, if multiple remote systems can all call the same renderer, guaranteeing uniqueness requires some sort of coordination (ex. a source system based prefix).

Note: I do not think that such issues should occur when the PG code is obtained via sourceFilePath so long as the source code of the problem (including files loaded by the main file) is not modified. Since a PG file can load other PG files, automatically generating a value for problemUUID may be more challenging when PG files are loaded from the server.

Triggering the issue:

1. Load the contents of https://raw.githubusercontent.com/openwebwork/webwork2/master/courses.dist/modelCourse/templates/setDemo/limits.pg
   into the source code window of the standalone renderer.
2. Click the "Render contents of editor" button.
3. Pay attention to how the graph looks, and use the browser to open the image in a new tab.
4. Edit the line $a=random(-3,3,1); by adding *(-1) before the closing semicolon.
5. Click the "Render contents of editor" button.
6. Pay attention that the graph remains unchanged, and that the URL for the image is the same.
   - The wrong image is being displayed.
7. Type something into the "file path" box, but do NOT click Load or Save.
8. Click the "Render contents of editor" button.
9. Pay attention that the graph changed, and use the browser to open the image in a new tab, and note that the URL for the image is different than it was in steps 3 and 6.
10. Change the new (-1) to (1) (drop the minus).
11. Click the "Render contents of editor" button.
12. Pay attention that the graph remains unchanged, and that the URL for the image is the same (as in step 9).
    - The wrong image is being displayed.

I recommend adding a format with both the "Preview My Answers" and "Submit Answers" button but without the "Show correct answers" button.