Registry Variables not available during streaming return about tg2

[pedersen]

Original Author: kgk, Original Timestamp: 2012-04-18 23:08:34.716000

Original Body: Thread-local variable such tg.request and tg.response are not
available to streaming return values in turbogears applications.

i.e. you cannot do

 @expose()
 def mystream():

     yield "boo"
     if tg.request.environ['a'] == 0:
       yield "bah"

     for x in range (1000):
        yield "blah"

During application initialization the RegistrtyManager is created
to handle thread local variable i.e. tg.request and tg.response.

TurboGears2-2.1.5: configuration.py:954

app = RegistryManager(app)

Paste-1.7.5.1bisque1-py2.6.egg/paste/registry:367

The RegistryManager includes code to pop registry values after the stream
has completed. However you must ask for this behavior with

app = RegistryManager(app, streaming = True)

[pedersen]

Original Author: amolsf, Original Timestamp: 2012-05-02 10:59:01.596000

Original Body: - milestone: 2.2.0 --> Undetermined

[pedersen]

Original Author: amolsf, Original Timestamp: 2012-05-02 10:59:02.099000

Original Body: As it was not one of the targets for 2.2 I'm moving it back to undetermined milestone for evaluation.

We should probably deeply check the side effects of lazy Stacked objects management from Paste RegistryManager as the registry manager expectes the response to always be an iterable he can consume when enabling streaming mode. This might have the side effect of consuming one character at time when returning the html response slowing down the application depending on what the TGApp returns.

[pedersen]

Original Author: krispk, Original Timestamp: 2012-05-02 18:37:17.594000

Original Body: Would this be as simple as checking the return type i.e. string or has
an iter attribute and processing in the correct way?

[kgk]

After some debugging, I found that even though you are returning a string in turbogears it
is placed into a list or tuple before it gets to paste.. The following patch of paste basically
passes strings (templates) unchanged, but allowed streaming apps to access registry variable
(after patching turbogears as mentioned above)

BTW it's impossible to post this bug/enhancement to paste.. their bug reporting site has been down
for months now. http://trac.pythonpaste.org/pythonpaste/report

Revision: 1504
Branch: default
Author: Kristian Kvilekval <[email protected]>  2012-05-08 14:07:36
Committer: Kristian Kvilekval <[email protected]>  2012-05-08 14:07:36
Tags: tip
Parent: 1503:8b76085af20c (allow token parameter to be a list or a preformed string)

    Don't penalize streaming applications.. test response if streaming needed.


------------------------------ paste/registry.py ------------------------------
@@ -372,11 +372,18 @@
         app_iter = None
         reg = environ.setdefault('paste.registry', Registry())
         reg.prepare()
-        if self.streaming:
-            return self.streaming_iter(reg, environ, start_response)
+        #if self.streaming:
+        #    return self.streaming_iter(reg, environ, start_response)

         try:
             app_iter = self.application(environ, start_response)
+            #print "REG ", type(app_iter)
+            if isinstance(app_iter, (list, tuple)):
+                #print "DIRECT"
+                return app_iter
+            #print "STREAMING"
+            return self.streaming_iter(app_iter, reg, environ)
+
         except Exception, e:
             # Regardless of if the content is an iterable, generator, list
             # or tuple, we clean-up right now. If its an iterable/generator
@@ -404,9 +411,9 @@

         return app_iter

-    def streaming_iter(self, reg, environ, start_response):
+    def streaming_iter(self, app_iter, reg, environ):
         try:
-            for item in self.application(environ, start_response):
+            for item in app_iter:
                 yield item
         except Exception, e:
             # Regardless of if the content is an iterable, generator, list

[pedersen]

The first question that comes to mind is this: Can we do this without patching Paste? Having our own, locally modified, copy is something that I do not like, and would very much like to avoid doing. Add in that we're not the maintainers for Paste, and this can be difficult to get published.

Is there a way to modify TG to fix this?

[amol-]

a0769d5 75273b2 and 29540be fixed this issue, making possible to access tg.request, tg.response and so on during streamed responses.

[kgk]

Thanks for fixing this.
What version is this fixed in 2.3?

[amol-]

yes, it's in 2.3