zhuyeqing / rupy Goto Github PK

Seems that size and limit are distinct terms, and mixed here. I once try to 
pipe with three arguments(ie, with out the limit), but i think of that a 100MB 
post in garbage can be nicely stoped, without further flood into the 
application code. However, that introduce a new startup argument "limit".

Query.java Line: 160
<if (Deploy.pipe(input, out, size, size) > 0)
>if (Deploy.pipe(input, out, size, limit) > 0)


By the way, Deploy.java, method pipe(,,,,), i move the line "total += read;" a 
few lines up, before the limit check.
And wondering if byte[] data, can be promoted to as class level private static 
buffer, performance better?

public static int pipe(InputStream in, OutputStream out, int length,
        int limit) throws IOException {
    byte[] data = new byte[length];
    int total = 0, read = in.read(data);
    while (read > -1) {
        total += read;
        if (limit > 0 && total > limit) {
            throw new IOException("Max allowed bytes read. (" + limit + ")");
        }
        out.write(data, 0, read);
        read = in.read(data);
    }
    return total;
}

Now "timeout", the property for Daemon, is defined as 
" session timeout in seconds or 0 to disable sessions."

I have found, actually the Session objects are still created, however, they are 
scheduled removed every second (Heart bit) in this case.


I wondering any live case for this is being used?
If yes, might 'constant' Session objects are created, the number of object is 
equal to threads(workers) number. and bypass remove action in heart bit loop. 
And at least session key should NOT sent to client in reply in this 
implementation.


How can i persist sessions?
unluckly, here ZERO is not defined for this purpose. :)

That uses .html and <!-- // --> instead of .jsp and <% // %> while keeping 
rupy footprint minimal.

The code base is a horrible mess. No packages, no proper logging. The threads 
are unstoppable "while(true)", there is not even a shutdown method at all.
It's a great idea, but the code itself is a shame and not integrable into 
other tools.

What steps will reproduce the problem?
1. Using Firefox 2.0.0.15 and Internet Explorer 6.0.2900.2180 SP2 on
Windows XP SP2.
2. Run the "talk" bat script from the command line and get rupy running and
then load the initial demo page: http://localhost:8000/talk.html in both IE
6 and FF2.
3. In the IE 6 browser type something into the chat textbox and press
enter. Now, in FF2 type something into the chat textbox and pay attention
to the rupy command prompt (output).  There will be an IOException error.

- Any further typing in either browser creates an "IOException: Reply still
processing".

What is the expected output? What do you see instead?


C:\rupy\talk>talk.bat

C:\rupy\talk>java -classpath
"../http/bin/http.jar;../http/lib/activation.jar;li
b/json.jar" se.rupy.http.Daemon -pass secret
RMY8 listen {RMY8=0}
F24H listen {F24H=1, RMY8=0}
BQDD broadcast {"id":1,"text":"test IE","channel":0,"key":"BQDD"}
F24H push
F9HB push
F24H listen {F24H=1, RMY8=0}
F9HB listen {F24H=1, RMY8=0, F9HB=2}
F24H broadcast {"id":2,"text":"test FF","channel":0,"key":"F24H"}
java.io.IOException: Reply already complete.F24H push

        at se.rupy.http.Reply.wakeup(Reply.java:134)
        at Broadcast.act(Broadcast.java:53)
        at Broadcast.filter(Broadcast.java:12)
        at se.rupy.http.Chain.filter(Chain.java:38)
        at se.rupy.http.Event.service(Event.java:256)
        at se.rupy.http.Event.read(Event.java:204)
        at se.rupy.http.Worker.run(Worker.java:90)
        at java.lang.Thread.run(Unknown Source)
F9HB push
F9HB listen {F24H=1, RMY8=0, F9HB=0}
F24H listen {F24H=1, RMY8=0, F9HB=0}


What version of the product are you using? On what operating system?
OS: Windows XP Service Pack 2
Browser 1: Firefox 2.0.0.15
Browser 2: Internet Explorer 6.0.2900.2180 SP2

(Screenshot attached)

What steps will reproduce the problem?
1. base64 encoded string are padded with '=', StringTokenizer consider the 
padding char as key-value seperator as well.
2. btoa("rupy")="cnVweQ==", example: "search=cnVweQ==&output=xml"
3.

What is the expected output? What do you see instead?
parsed parameters should be:
{
search : "cnVweQ=="
output : "xml"
}
instead of
{
search : "cnVweQ"
output : "xml"
}

What version of the product are you using? On what operating system?
0.4, xp, java 1.508

Please provide any additional information below.
my rewrited lines:
LINE 159
            while (amp.hasMoreTokens()) {
                String keyvalue = amp.nextToken();
                int pos = keyvalue.indexOf('=');

                String key = keyvalue.substring(0, pos);
                String value = keyvalue.length() == pos + 1? "" : keyvalue.substring(pos + 1);
                value = decoder.decode(value, "UTF-8");

                put(key, value);
            }

What is the expected output? What do you see instead?
rupy can be used with any websockets client
check this out for more info: http://www.websockets.org/ and 
http://jwebsocket.org/

Please provide any additional information below.
This enhancement would be having multiple benefits:
1. More efficient communication compared to Comet
2. Great visibility, not many servers do actually implement websockets at the 
moment, but this is getting hot

Is it possible to stop the daemon? I'd like to be able to start/stop the 
daemon (possibly) many times.

Thanks for the hard work! Looks great!

What steps will reproduce the problem?
1. Should work fine, but needs to be tested!

What is the expected output? What do you see instead?


Please use labels and text to provide additional information.
Using apache-commons to provide a tiny HTTP upload plugin, if you are up 
to the task, send me a mail!

What steps will reproduce the problem?
1. When I run 500 concurrent clients sending 400 requests per second server 
throws following exception:

java.lang.NullPointerException
        at java.util.LinkedList.clear(LinkedList.java:294)
        at se.rupy.http.Session.remove(Session.java:51)
        at se.rupy.http.Session.remove(Session.java:46)
        at se.rupy.http.Daemon$Heart.run(Daemon.java:761)
        at java.lang.Thread.run(Thread.java:662)

What version of the product are you using? On what operating system?
the latest available (0.4.2)

A:
i want to use rupy server in a sns Web,we have hundreds of thousands of
times per second the request. i do a test , use 300 threads to run the rupy
server.

when all Workers are running, the Event will do  workers.next() 300 times,
it's not a good design. i modify the employ() function to this:

private Map<Integer,Worker> waitWorkers;


synchronized Worker employ(Event event) {

        Worker w = this.notifyWaitWorker();

        if(w == null){
            synchronized (this.queue) {
                queue.add(event);
            }
            return null;
        }else{
            event.worker(w);
            w.event(event);
            w.wakeup();

            return w;
        }
}

public void addWaitWorkers(Worker w){
        synchronized(this.waitWorkers){
            this.waitWorkers.put(w.index(), w);
        }
    }

    public Worker notifyWaitWorker(){
        synchronized(this.waitWorkers){
            if(this.waitWorkers.isEmpty()){
                return null;
            }else{
                int key = this.waitWorkers.keySet().iterator().next();
                return this.waitWorkers.remove(key);

            }
        }
    }



and modify the Worker.java 

void snooze(long delay) {
        synchronized (thread) {
            try {
                if (delay > 0) {
                    thread.wait(delay);
                } else {
                    thread.wait();
                }
                this.daemon.addWaitWorkers(this);
            } catch (InterruptedException e) {
            }
        }
    }



B:
By the way, i find another problem:
on my test, when the server's connections is less than 100000, the server's
CUP occupancy rate is less than 10%, when the server's connections is more
than 20000, the CUP occupancy rate is between 100%~110%. i think it will
any bugs in the rupy server code. i hope the feedback is helpful of you

What steps will reproduce the problem?
1. You want 2 root services on the same rupy instance with different host 
names

What is the expected output? What do you see instead?
This doesent work now, would be nice and really simple to add!

Please use labels and text to provide additional information.

What steps will reproduce the problem?
An analysis with Java PathFinder revealed the rupy logs many 
NullPointerExceptions when multiple connections are handled simultaenously.

What is the expected output? What do you see instead?

After adding an assertion in Worker.reset to track NullPointerExceptions, I was 
able to see that a race condition on Worker.event was responsible.

--- a/src/examples/se/rupy/http/Worker.java Wed May 15 08:54:40 2013 +0900
+++ b/src/examples/se/rupy/http/Worker.java Wed May 15 09:56:15 2013 +0900
@@ -205,6 +205,10 @@
    }

    protected void reset(Exception ex) {
+       if (ex instanceof NullPointerException) {
+           ex.printStackTrace();
+           assert(false);
+       }
        if(event != null) {
            event.disconnect(ex);
        }


What version of the product are you using? On what operating system?

The bug still seems to be present in 0.4.5, although an older version was 
tested (the researcher who uploaded the code for analysis forgot to include the 
version number but it is probably 0.4.4).

Please provide any additional information below.

The patch is a possible fix. I have added "synchronized" to every access of 
Worker.event. Of course this fix makes the program a bit less efficient, but it 
eliminates the null pointer issues found before.

A fix using less synchronization may be possible (please contact me if 
interested in trying at c DOT artho AT aist.go.jp.

What steps will reproduce the problem?
1. Run rupy with Java 1.6
2.
3.

What is the expected output? What do you see instead?
The servers Selector.select() method will throttle after a while, this is 
caused by the 1.6 native NIO implementation:

1.4, 1.5 - PollArrayWrapper
1.6 - EPollArrayWrapper

Please use labels and text to provide additional information.

We will try to workaround this, but for now only use Rupy with 1.4 - 1.5 
on production servers.

What steps will reproduce the problem?
1. Download and extract talk-0.7.jar on windows xp with recent jdk and ant.
2. run.bat executes properly (it appears)
3. ant deploy (while in talk folder) fails with:

---------------------------------------------------
C:\javaprogs\rupy\talk-0.7\talk>ant deploy
Buildfile: C:\javaprogs\rupy\talk-0.7\talk\build.xml

check-http:

http:

check-page:

page:

check:

build:

deploy:
     [java] java.lang.NoClassDefFoundError: se/rupy/http/Deploy
     [java] Caused by: java.lang.ClassNotFoundException: se.rupy.http.Deploy
     [java]     at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
     [java]     at java.security.AccessController.doPrivileged(Native Method)
     [java]     at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
     [java]     at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
     [java]     at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)

     [java]     at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
     [java] Could not find the main class: se.rupy.http.Deploy.  Program will ex
it.
     [java] Exception in thread "main"
     [java] Java Result: 1

BUILD SUCCESSFUL
Total time: 2 seconds
---------------------------------------------------

What is the expected output? What do you see instead?
Expected output is "talk.jar deployed" or similar.

What version of the product are you using? On what operating system?
talk 0.7, winxp, recent jdk and ant

Please provide any additional information below.

To solve:
Extract rupy-0.4.zip to the same folder as talk-0.7.zip. So the directory 
structure should look like:

c:\myprogs\http
c:\myprogs\talk

THEN, you may use "ant deploy" in the talk folder to deploy talk (be sure to 
execute "talk\run.bat" first in a separate command window).

This matters because the build.xml ant script in "talk" looks for the http 
folder on the path next to it, and the included http.jar doesn't appear to 
satisfy its package requirements (se...Deploy) even though it appears that the 
intention was for talk to be self-contained.

Please remark if I'm missing something here. I'm very excited about this 
project. Thanks Marc,Jo..,Erik.

What steps will reproduce the problem?
1. Add a large file / service to the deployable
2. Request it.
3. The server will write until the socket buffer is exauhsted but not 
further.

What is the expected output? What do you see instead?


Please use labels and text to provide additional information.

Add SSL Sockets...

Small enhancement suggestion. Allows for bla/* paths. It requires public on
Chain and Chain get(String);


public class WildCardHttpDaemon extends Daemon {

    public WildCardHttpDaemon(Properties properties) {
        super(properties);
    }

    public Chain get(String path) {
        Chain chain = null;
        while (path != null && chain == null) {
            chain = super.get(path);
            path = wildCardPath(path);
        }
        return chain;
    }

    private String wildCardPath(String path) {
        if (path.endsWith("/*")) path = path.substring(0,path.length()-2);

        int lastSlash = path.lastIndexOf('/');
        if (lastSlash < 0)
            return null;
        int nextToLastSlash = path.lastIndexOf('/', lastSlash);
        if (nextToLastSlash < 0)
            return null;
        String string = path.substring(0, nextToLastSlash) + "/*";
        System.out.println(string);
        return string;
    }
}

What steps will reproduce the problem?
1. implement service
2. test with JMeter (e.g. 20 threads / 100 iterations)


What is the expected output? What do you see instead?
> After roughly 100-300 fast requests some threads throw a 
CancelledKeyException in Event.register(). After that no requests are possible 
anymore.

What version of the product are you using? On what operating system?

> rupy 0.4

Please provide any additional information below.

> I tried catching that exception in two places, but the thread still stops and 
no requests are possible anymore. It also seems that the server recovers after 
some time, but that would take too long in production.

With Rupy 0.4.3 on Linux, I'm getting a whole bunch of these in error.txt.

12-06-02 19:05:55.702 null null
java.lang.Exception: Threadlock 5693 (null)
        at se.rupy.http.Worker.busy(Worker.java:138)
        at se.rupy.http.Daemon$Heart.run(Daemon.java:907)
        at java.lang.Thread.run(Thread.java:662)

Not entirely sure why. Any ideas?

What steps will reproduce the problem?
1. Build a services that is broken.
2. Run it from a browser.

What is the expected output? What do you see instead?

You should get the stack trace in the browser ...

Please use labels and text to provide additional information.

I can start rupy easily. I just wanted an easy way to browse my html which uses 
Flot chart libraries, over Rupy. I dont see any help either.
Even there is no google-group to discuss this issue, hence logging it here.