Support programs that import `racket/gui/base` about racket-mode

[greghendershott]

Thanks for the report. I have seen the cannot instantiate racket/gui/base' a second time in the same processmessage, for example when usingracket/plot`. However I haven't seen the segfault.

I wonder if the root issue is that the GUI module just doesn't cooperate with racket/sandbox and custodians -- that it won't "un-initialize" as will ports and other things that custodians do manage.

I haven't encountered this often, because I don't use racket/gui much, and if I do, I usually use DrRacket. Running a GUI program from command-line racket has always seemed like a slightly weird experience, for me, at least on Windows and OSX. So, this is a use mode I honestly just haven't given much thought to, so far.

If that sounds like me justifying that this isn't a high-priority fix...I guess that's accurate. But having said that, I would be happy to fix it if I knew how or at least where to start. If you have any ideas or clues, please let me know. Otherwise I will look at this when I can set aside enough time to research and debug. Thanks again for the report.

[stchang]

I don't think this problem is specific to racket-mode.

Do you still get the error with (require redex/reduction-semantics) instead of (require redex)?

[florence]

(require redex/reduction-semantics) causes no error.

Its possible this is a bug in the sandbox, where racket/gui/base is instantiating some state that escapes the sandbox. I'll try to investigate later tonight.

[florence]

I asked about this on the mailing list:
http://www.mail-archive.com/[email protected]/msg21853.html

Looks like this is something inherent to racket/gui.

[greghendershott]

Robby's comment suggests it would be possible for racket-mode's do-run (in sandbox.rkt) to instantiate racket/gui exactly once, and share it with the sandbox. I'm just not sure how to go about doing that.

Because:

• Don't want to instantiate racket/gui at all, if the program doesn't require it (because overhead for non-GUI programs, and because weird headless process on OS X and Windows). Want to wait until the sandbox loads the first program with (require racket/gui), if any.
• How to "share" it with the sandbox. namespace-attach-module and friends?
• How to nullify the program's own (require racket/gui) -- to avoid the second or more instantiation. Munge the syntax? But what if it's a compiled zo?
• Maybe create a dummy racket/gui and try to get it in the collection load path ahead of the real one??

[florence]

It looks like simply requiring racket/gui/base in the parent allows child evaluators to use it, without changing the child:

#lang racket/base
(require racket/sandbox racket/gui/base)

(call-with-trusted-sandbox-configuration
 (lambda ()
   (define (make) 
     (make-evaluator 'racket/base
                     #:requires '(racket/gui/base)))

   (make)
   (make)))

That solves bullets 2 and 3. 1 and 4 might be solvable with some dynamic-require or lazy-require, plus some magic, but I'm not sure how to inspect the user program to see when its needed. Perhaps just adding a racket-run-gui or racket-mode-initialize-gui might be enough. It's not elegant, but it's simple. racket-mode could still get into states where a child loads the gui before racket-mode-initialize-gui is run, but a quick restart would fix that. Maybe racket-mode-initialize-gui could restart the repl if it sees (dynamic-require 'racket/gui/base 0) error.

The only other option I could think of would be to have 'racket-run' restart the whole sandbox, which isn't the best idea.

[greghendershott]

Good ideas. I'll mull it over and try to tackle it later (unless you wanted to work on it and submit a PR, sooner).

[florence]

I will do that! :]

[viperscape]

I'm receiving this issue as well, when using #lang frtime, running a second or consecutive time causes the error: ; cannot instantiate racket/gui/base' a second time in the same process`

[greghendershott]

I had to focus on another project for most of the last week. Taking a look at this again, now.

Perhaps just adding a racket-run-gui or racket-mode-initialize-gui might be enough. It's not elegant, but it's simple.

That is probably the pragmatic thing to do. I'll do that if this drags on unresolved too much longer.

---

Meanwhile, I did spend some time yesterday thinking about this. My latest idea is: Don't require racket/gui/base in sandbox.rkt. After each ,run foo.rkt, check whether racket/gui/base is in the namespace (because the program that just ran, foo.rkt, required it). If so, use namespace-attach-module to attach that existing instance of racket/gui/base to the fresh, new, otherwise-empty namespace for the next ,run. This should mean that the next program's (require racket/gui/base) will be a "no-op", i.e. no "second instantiation" error. (It will also mean that the next program has racket/base/gui required, like it or not, but I think that's unavoidable due to how racket/gui/base works.)

But I wasn't able to get that to work. And I didn't understand why. One concern is that racket/sandbox makes a namespace for you; maybe to do the above successfully I need to directly control creating the namespace?

Then I got to thinking, why am I using racket/sandbox at all? Originally I thought it would make it simple to "blow away things and start fresh each time"; also I'd heard (maybe incorrectly) that DrRacket uses a sandbox. But as I thought about it more, and looked at racket/sandbox source as well as xrepl source, I think it's probably just a layer of complication. It's not like I'm racket/sandbox's actual sandboxing capabilities (restricting access or memory or compute time). Why use it at all? Why not do something more directly?

So I spent a fair amount of time yesterday going all the way down to the source for Racket's enter! and dynamic-rerequire, and trying to understand how to do something similar, rather than use racket/sandbox. This would be closer to how xrepl works, but although xrepl uses enter!, I'd use my own variation of it.

I'm still on that detour, now. Will report back with success or if I decide to abandon that.

[florence]

dynamic-require doesn't work because racket-mode creates a new namespace, and attaching racket/gui/base to that namespace doesn't seem to set sandbox-gui-enabled to true. This new namespace NOT needed, as racket-mode uses make-module-evaluator which already creates its own namespace, and has some fancy tricks for allowing gui usage---It uses either make-base-namespace or make-gui-namespace depending of if gui is available. I'm working on a fix where the current-namespace is not parameterized and racket-mode just dynamic-requires racket/gui/base on request.

[greghendershott]

If you can figure out how to make it work with racket/sandbox, that would great to know.

I think more basically I was wondering why I was even using racket/sandbox in the first place. I think it was from a misguided notion on my part. I really don't use any of what it offers.

---

I just pushed an experimental branch. Its sandbox.rkt turns out to be much simpler. It solves the racket/gui/base instantiation problem (at least for me). Of course there may be other bugs I haven't discovered yet. Also, if we were to go this way, before merging to master I would probably rename sandbox.rkt to something like eval.rkt instead. :)

[florence]

If one can accept that sandboxes are not needed your solution is definitely better. (its more of a rewrite but the resulting code is simpler.) I had a working solution working with racket/sandbox by not creating a new namespace, but I rejected it assuming the namespace was necessary. I'll go recreate if you think thats a better way to go than your experimental branch.

[greghendershott]

I ought to dogfood my experimental branch for a day or two of real use, and make sure. But I think it's probably the way to go. I really appreciate you taking time to work on this, and I'm really sorry.

If one can accept that sandboxes are not needed

Yeah I'm saying I think it was my own damn fault for making this more complicated than it needed to be, by using racket/sandbox at all. All I really needed was a fresh namespace and a custodian (in order to do the DrRacket-style "clean slate").

Although in fairness, racket/sandbox seemed to "just work". And did work for me for many months, until this issue with racket/gui and its unique once-per-process requirement.

Whereas my new code in experimental, although it's simpler, is the result of me spending quite a few hours reading through Racket's source for enter!, dynamic-rerequire, xrepl, and re-reading the somewhat opaque docs for namespaces, modules, and so on. Much of which I'd already spent hours puzzling over in the past. In other words, I'm not sure I could have written it like this in the first place, anyway. Live and learn.

[greghendershott]

@florence if you have time could you please try the new gui branch and see how it works for you?

1. This should handle on-demand creation of a single instance of racket/gui/base, correctly, provided the (require racket/gui/base) is in a .rkt file you are Run-ing.
2. Whereas if you type (require racket/gui/base) at the REPL prompt, it will tell you it's not possible to do so. From long emails with Robby, it seems that it's not possible to instantiate racket/gui/base correctly from inside the sandbox evaluator. So I catch and prevent that, with an error message.

BTW in the above (require racket/gui/base) means that directly, and also any require that transitively does that. I'm using a custom module name resolver to watch the requires.

I think this is as good as I can get it. Certainly it was waaaaaay more involved than I expected. O_o. If the gui branch works fine for you then I would like to merge it to master.

[florence]

Uhhh.... So all of my redex code is giving me this really weird error:

; link: bad variable linkage;
;  reference to a variable that is not a procedure or structure-type constant across all instantiations
;   reference phase level: 1
;   variable module: "/Applications/Racket/pkgs/math-pkgs/math-lib/math/private/flonum/flonum-exp.rkt"
;   variable phase: 0
;   reference in module: "/Applications/Racket/pkgs/math-pkgs/math-lib/math/private/flonum/flonum-log.rkt"
;   in: flexpm1

Which isn't happening when I use racket or raco make. My racket version is a bit old (6.0.0.4-somethingorother), So ill try recompiling on HEAD when I have a power source again.

[greghendershott]

Hmm, OK.

If even after a rebuild, you have the problem, I'd love a short-ish sample that elicits this.

Because for instance the following from the tutorial does not error for me:

#lang racket
(require redex)

(define-language L
  (e (e e)
     (λ (x t) e)
     x
     (amb e ...)
     number
     (+ e ...)
     (if0 e e e)
     (fix e))
  (t (→ t t) num)
  (x variable-not-otherwise-mentioned))

(redex-match
   L
   e
   (term (λ (x) x)))

But maybe just because it's too simple. I haven't used redex enough to know.

[florence]

OK fixed. Now I seem to be unable to operate the GUI of any window I launch at all. They all hang:

#lang racket/gui
(define f (new frame% [label "test"]))
(define h (new button% [label "test"] [parent f]))
(send f show #t)

#lang racket
(require redex)
(define-language L (e := x y))
(define ->
  (reduction-relation 
   L
   (--> x y)))
(traces -> `x)

I bet this is because the sandbox isn't letting threads, well, thread:

#lang racket
(require racket/sandbox)
(define m
  `(module m racket
    (thread (thunk (let l () (displayln 5) (l))))))
(define E (call-with-trusted-sandbox-configuration
           (thunk (make-module-evaluator m))))
(E (void))

(let loop () (loop))

[greghendershott]

An example I had been using, which does work for me is using plot. I can interact with the windows using the mouse -- zoom, unzoom:

#lang racket
(require plot math)
(plot-new-window? #t)
(plot (function sin (- pi) pi #:label "y = sin(x)"))
(plot (function sin (- pi) pi #:label "y = sin(x)"))

However, I get the same bad result as you do, trying your examples. Crap.

I bet this is because the sandbox isn't letting threads, well, thread:

That... is not what I expected....

[greghendershott]

@florence By the way

I bet this is because the sandbox isn't letting threads, well, thread:

Actually the problem there is different -- sandboxes default to disconnected I/O. It works if you set sandbox-output to current-output-port:

#lang racket
(require racket/sandbox)
(define m
  `(module m racket
    (thread (thunk (let l () (displayln 5) (l))))))
(define E (call-with-trusted-sandbox-configuration
           (thunk
            (parameterize ([sandbox-input (current-input-port)] ; <--
                           [sandbox-output (current-output-port)] ; <--
                           [sandbox-error-output (current-error-port)]) ; <--
              (make-module-evaluator m)))))
(E (void))

(let loop () (loop))

[greghendershott]

@florence Now I'm even more confused. Re your example:

#lang racket/gui
(define f (new frame% [label "test"]))
(define h (new button% [label "test"] [parent f]))
(send f show #t)

Did this ever work for you under racket-mode -- in the sense that the GUI wasn't "frozen". Because when I revert to master, it's frozen.

There is something designed to be used by REPLs, current-get-interaction-input-port. It returns a wrapped input port which calls yield, if the thread is associated with a GUI eventspace. Its source.

And this works with the plain Racket command-line REPL, including ehanced with xrepl.

However I'm using racket/sandbox. Although my REPL is using current-get-interaction-input-port, the REPL is the main thread, and the sandbox has another, special evaluator thread. And apparently that special thread is the one for which (current-get-interaction-input-port) returns the message-pumping input port; that's the thread with the eventspace associated. Not the main REPL thread, which is calling (current-get-interaction-input-port).

ATM I don't see how to address that while still using racket/sandbox. I seem to keep swinging between a rock and hard place, with this.

Again, I think this would have been an issue, all along, more basic even than racket/gui/base instantiation problems. I don't see how it would have worked, even the first time. But am I wrong? Did you ever see the example program above work, instead of being "frozen"?

[florence]

When I first saw this issue I wasn't including racket/gui/base unintentionally(I was using the non-gui components of a library that had gui components), so I never actually launched a gui: It's entirely likely this behavior already existed. Which is confusing, as the sandbox docs seem to imply gui's should work fine.

[greghendershott]

As an interim step, I pushed a commit to master for MELPA:

Graceful error on attempt to require racket/gui/base.

Although this is not yet a resolution for issue #26, it at least makes
it clear that racket-mode doesn't yet support GUI programs. It avoids
people getting into the state where a GUI app is "frozen", and/or where
running again gives the "cannot instantiate `racket/gui/base' a second
time in the same process" error or even segfaults.

As the commit message says, I don't think it closes this issue. I'm still going to work on it. However it's turned out to be exceptionally difficult (for me, anyway) and I'm not confident when I'll be able to complete it.

[greghendershott]

I guess I did some sort of mind-hack on myself. Because soon after pushing that work-around and saying I'm not confident when I can complete it... I got a second wind. I worked on it more today, step by step, and think I might have it now.

@florence When you have a chance could you please try the new no-sandbox branch?

For me this works with all of your examples: The GUI message pump works. There's no error message or segfault wrt instantiating r/g/b twice in the same process.

When you have a chance, please let me know if same for you? Thank you.

[florence]

Ok I'm getting some strange behavior. This program:

#lang racket
(require redex)
(define-language L (e := x y))
(define ->
  (reduction-relation 
   L
   (--> x y)))
(traces -> `x)

will not launch the racket repl. M-x racket-mode drops the message "Showing all blocks... done" in the mini buffer then... nothing. No repl pane, no messages in the log, nada. I'm not sure how to turn on racket-mode's logger either.

racket-mode works normally otherwise. If I run the sandboxed version of that program from before:

#lang racket
(require racket/sandbox)
(define m
  `(module m racket
           (require redex)
           (define-language L (e := x y))
           (define ->
             (reduction-relation 
              L
              (--> x y)))
           (traces -> `x)))
(define E (call-with-trusted-sandbox-configuration
           (thunk (make-module-evaluator m))))
(E (void))

I get the 'one time instantiation' message, and the program behaves as expected (ie the GUI launches and hangs).

[greghendershott]

M-x racket-mode drops the message "Showing all blocks... done" in the mini buffer then... nothing. No repl pane, no messages in the log, nada.

Well M-x racket-mode tells Emacs to put the edit buffer for the .rkt file into the major mode racket-mode. Which you shouldn't need to do -- it should be in racket-mode already due to the .rkt extension. (The "Showing all blocks... done" message is just some output from Hide/Show mode, which apparently only shows up when you do M-x racket-mode when the buffer is already in racket-mode.)

Remember you need to hit F5 (or M-x racket-run) to Run -- which opens the racket-mode REPL window?

[florence]

Dear @greghendershott

I am an Idiot.

Sincerely spencer

--PS LGTM. my redex traces are all running fine.

[greghendershott]

Whew, great!

I am an Idiot.

Nah. When something has been buggy, and you get a weird message like "Showing all blocks...done", and nothing happens -- it's natural to assume it's still buggy, not that you made a mistake. Certainly I've done similar many, many times. Including while working on the code for this. :)

[UberLambda]

Hi, it seems like the issue wasn't fixed on my setup.

I'm currently trying to run this snippet (via C-CC-K):

#lang planet neil/sicp

(* 3 4 5)

But when I do run it a second time, this appears on the Racket-REPL buffer:

; cannot instantiate `racket/gui/base' a second time in the same process
; Context:
;  /home/paolo/.racket/planet/300/6.1.1/cache/soegaard/sicp.plt/2/1/primitives.rkt:1:1 [traversing imports]
;  /home/paolo/.racket/planet/300/6.1.1/cache/soegaard/sicp.plt/2/1/painters.rkt:1:1 [traversing imports]
;  /home/paolo/.racket/planet/300/6.1.1/cache/soegaard/sicp.plt/2/1/sicp.rkt:1:1 [traversing imports]
;  /home/paolo/.racket/planet/300/6.1.1/cache/neil/sicp.plt/1/17/main.rkt:1:1 [traversing imports]
;  /home/paolo/sicp/sicp.rkt:1:1 [traversing imports]

Was this error intentional? (The previous comments talk about a "graceful error", but it doesn't seem to be this one; sorry if it indeed was)

[greghendershott]

I'm handling racket/gui/base, but not scheme/gui/base (these are very old Planet packages, which use the latter).

I thought scheme/gui/base was simply an alias for racket/gui/base these days, so it would be covered. Apparently I'm wrong and it's not that simple.

I think I'll open a fresh issue for that, as the comment thread on this issue is already very long,

[UberLambda]

Thanks!

[greghendershott]

@UberLambda I pushed a commit. (Mentioning here in case the @ mention on issue #133 didn't notify you.)