md5 at the proxy about swift

[thiagodasilva]

@icorderi I think this patch might help: https://review.openstack.org/#/c/124903/
It just wasn't clear why you are trying to bypass the etag calculation. what did i miss?

[icorderi]

@thiagol11 Well, in the gluster case or any system plugin in behind that needs only 1 replica it doesn't matter. But for systems that still want to let swift do replication doing n md5()'s is a waste of CPU.
If the Proxy calculates the md5 once, and gives the etag down to the locally running object-servers, you save n-1 md5 calculations.

[thiagodasilva]

@icorderi, ok, that's what i figured but just wanted to make sure...I think sam's patch is going to help with that, but of course there are issues as dfg pointed out...

on another note, i got your changes (very nice btw!) and i'm working to clean up the code a bit and add another method of initializing the object server. should i just push those changes on top of yours?

[icorderi]

@thiagol11 I'm not sure we need the footer approach. In our case the object-servers are running on the same process as we are, that means we can just calculate the md5() and give it directly to each object-server instance.

eventlet.event.Event can be used as a lazy value.
I was thinking on having the object-server code do a:

etag = environ['Lazy-Etag'].wait() # on the bypass code-path

And have the proxy "clean" code do:

if etag: # send etag to object-servers
    for c in conns:
        if hasattr(c, 'etag'): 
            c.etag = etag # our custom conn object does evt.send(value). 

The etag calculation at the proxy can be turned on by any conn having:

hasattr(conn, 'requires_etag') and conn.requires_etag

What do you think?

[thiagodasilva]

@icorderi, sounds pretty clever, just a little unclear on the object server side the wait() would happen in line 433 or 451: https://github.com/icorderi/swift/blob/feature/no-http/swift/obj/server.py
Also

[thiagodasilva]

@icorderi, when you have a chance, checkout the change I pushed. It changed things a bit in terms of how to call the object-server, now it is not bypassing all the middleware, which I think might be better.

[icorderi]

@thiagol11 the wait() would be in L467

[icorderi]

@thiagol11 I don't see any push. Which branch was it?

[thiagodasilva]

@icorderi, had made a mistake pushing to another branch, fixed it now. Besides the change in wsgi.py to instantiate the app, the other interesting change is this: https://github.com/icorderi/swift/blob/feature/no-http/swift/proxy/controllers/obj.py#L423,L425