jcuga / golongpoll Goto Github PK
View Code? Open in Web Editor NEWgolang long polling library. Makes web pub-sub easy via HTTP long-poll servers and clients :smiley: :coffee: :computer:
License: MIT License
golang long polling library. Makes web pub-sub easy via HTTP long-poll servers and clients :smiley: :coffee: :computer:
License: MIT License
Since this is written using Golang and I'd expect many users to use Golang for their clients if they're writing backend code, it would be nice to have a client library to consume what this server produces.
Makes it much faster/simpler to integrate and allows to maintain best practices in the client that are aligned with the ones of the server.
It could also be used as an example to write clients in other languages.
I've written a client for one of my other projects, not sure if you're interested in bringing this into your codebase, I can open a PR if that's interesting for you. At the same time I'd polish up the godoc + write a proper readme
Code is here:
https://github.com/julsemaan/golongpoll-client
BTW, really like this library, just so simple and effective, really great work.
As a candy, that would fix #12 at the same time and lets be honest, who doesn't like candies!
100 Go Mistakes (T. Harsanyi)'s mistake 76 points out that using time.After
in a loop or an http handler leads to the underlying channel waiting the full duration before being closed.
Update uses of time.After
to use time.NewTimer
and it's Reset()
and/or Stop()
functions where appropriate.
This looks to be an issue in longpoll.go
and fileaddon.go
. (This likely exists in many of the test files as well, but less concerned about those...).
I'm using this library for publishing events of multiple connected meters (constantly being polled at 9600 baud). I've noticed that just publishing those events longpoll manager- even without connected clients- raises CPU load to > 50% on Raspi3:
func (f *Firehose) Run() {
for {
select {
case snip := <-f.in:
f.lpManager.Publish("meterupdate", snip)
case statupdate := <-f.statstream:
f.lpManager.Publish("statusupdate", statupdate)
}
}
}
...
go firehose.Run()
Does this behaviour make sense to you or should a low CPU load be expected when publishing without clients attached?
Hi, i'm using this package to implement notifications (it's awesome by the way)
i use one category for each user
after the user gets the notifications i'm not interested in keeping them in the buffer, so i'm deleting the user buffer after every publish (i did this already, i would do a pull request if you like)
also i don't want to keep the buffer if the user left the site for a long time
so, how i could do that?
Do you guys have any idea or plan for GIN package support for long polling? I would love to work on the same.
Thanks.
Line 652 in 033d1fa
I see there is no lock when remove and add item from map, this will case crash.
Latest tag is v.1.1.0
, and I think it should be v1.1.0
. Having it as v.1.1.0
messes up SemVer support with package managers like Glide.
Can you retag?
disconnectNotify := r.Context().Done()
--- a/longpoll.go
+++ b/longpoll.go
@@ -270,7 +270,7 @@ func getLongPollSubscriptionHandler(maxTimeoutSeconds int, subscriptionRequests
// event that a client crashes or the connection goes down. We don't
// need to wait around to fulfill a subscription if no one is going to
// receive it
- disconnectNotify := w.(http.CloseNotifier).CloseNotify()
+ disconnectNotify := r.Context().Done()
select {
case <-time.After(time.Duration(timeout) * time.Second):
// Lets the subscription manager know it can discard this request's
This looks pretty useful for my use case.
I have a restful interface at the moment, and need to push things to the Web Client.
I want to avoid Web sockets and too much complexity.
So, i am wondering about If the Restful Server i have , (based on Gin) needs to be a separate service from the Long Poller. I ask this because for my Restful interface i have alot of routing URLS already setup, and i am not sure if i can just all the Long Poller as a path on the standard Gin Router or not.
I know this is maybe hard to answer and i should just give it a go, but thought it was worth asking first.
thanks in advance.
Interested in polling multiple categories in one call.
Was not sure how to add support to poll multiple categories other than make multiple poll requests.
Hi,
as I understand..long polling need to timeout the polling so the server could free up the holding request.
How do you handle user when user leave page without information (page crash or internet down) then user refresh the page and re-request the long polling?
You hold previous request for nothing.
I can implement it if it makes sense.
I have an application that can be offline sometimes, so I'm passing the timestamp to get past events, but in this scenario it fails:
since_time
value because it didn't have a timestamp from the last event (as it did not receive any).I could calculate the timestamp on the client, but I don't think it would solve this, the date/time of the client could be wrong for example.
It would work better if the timeout reply returned a timestamp, in this case:
{
"timeout": "no events before timeout",
"timestamp": 1447218359843
}
Then I could update the timestamp after the first timeout, and even if the client loses the connection It will be able to get past events since the last timeout.
What do you think?
Hello,
I really like the project, and I am using it for some internal tools. I wanted to share the little js client "library" I have created over the weekend. Maybe you can use it.
https://gist.github.com/bluebrown/53e5144ea5c7e8fb1c77c20187844273
Anyway, thank you for the package.
Regards,
Nico
When two events are published to the LongpollManager
in the same millisecond, the behavior I observed is that clients will receive the first event only. The other events in the same millisecond can be seen by trying a request again with the same since_time
, but if clients listen again using the timestamp
received (as advised in https://github.com/jcuga/golongpoll#http-subscription-handler), they will never see those events.
For example, here is sample output from a client listening to a simple golongpoll
app that is hardcoded to publish two events in quick succession every 10s:
# Initial call
> curl -i "localhost:8081/events?category=connectivity&timeout=60&since_time=1588816900233"
HTTP/1.1 200 OK
Cache-Control: no-cache, no-store, must-revalidate
Content-Type: application/json
Expires: 0
Pragma: no-cache
Date: Thu, 07 May 2020 02:01:50 GMT
Content-Length: 97
{"events":[{"timestamp":1588816910237,"category":"connectivity","data":{"message":"message 1"}}]}
# If calling again with the same since_time, I see the event with duplicate timestamp:
> curl -i "localhost:8081/events?category=connectivity&timeout=60&since_time=1588816900233"
HTTP/1.1 200 OK
Cache-Control: no-cache, no-store, must-revalidate
Content-Type: application/json
Expires: 0
Pragma: no-cache
Date: Thu, 07 May 2020 02:01:51 GMT
Content-Length: 182
{"events":[{"timestamp":1588816910237,"category":"connectivity","data":{"message":"message 1"}},{"timestamp":1588816910237,"category":"connectivity","data":{"message":"message 2"}}]}
# If calling again with the received timestamp, the "message 2" message is never received.
> curl -i "localhost:8081/events?category=connectivity&timeout=60&since_time=1588816910237"
HTTP/1.1 200 OK
Cache-Control: no-cache, no-store, must-revalidate
Content-Type: application/json
Expires: 0
Pragma: no-cache
Date: Thu, 07 May 2020 02:02:00 GMT
Content-Length: 97
{"events":[{"timestamp":1588816920241,"category":"connectivity","data":{"message":"message 1"}}]}
Is this behavior expected? I am planning to work around this by forcing a 1ms delay between publishes, but perhaps this could be handled in the LongpollManager
by aggregating all events in the same millisecond and sending them together?
do golongpoll has some success in production?
Hi,
While I was monitoring network traffic on my device I saw that every 15 seconds there are acks being sent between the longpoll client and server.
Is this the intentional behavior?
The activity does not need to be ‘silent’ until a timeout occurs or a disconnect or a publish?
@jcuga
Currently event buffers are all in-memory--in many ways this is a performance gain and is simpler code.
However, if you want the event buffer to persist across restarts, or to load balance across multiple instances, data(base) backend support would be useful.
My current workaround for scaling is to design data needs up front to be deterministic so I can segment traffic consistently to the same running instance and each instance has it's own sub-universe of data. There is no workaround for persisting data across program restarts 😢
At some point I will try and add support for data backends, but cannot promise when that will be.
This is nice for very low amount of users of the server, how one can approach a c10k problem with this library ? maybe a way to connect those to a redis backed ? same as nginx nchan is doing ?
Won't it be better for the handler function to stop waiting for a timeout if the client has been disconnected?
Steps to reproduce the issue:
Create a new manager and register the handler:
manager, _ = golongpoll.CreateCustomManager(120, 1, true)
http.HandleFunc("/events", func(w http.ResponseWriter, r *http.Request) {
manager.SubscriptionHandler(w, r)
fmt.Println("client disconnected")
})
Connect to the server from a browser (it will start waiting for the response)
Manager's log: Adding Client (Category: "abc112" Client: c6650762-f3d7-4876-410c-c6df1ea1dfc0)
Without publishing anything disconnect the client (close browser's window or something)
The SubscriptionHandler is still running
I guess it'd be logical to terminate the handler's execution at once after the disconnect event.
I have 3 replicas of pods,A pod pubs a message and B pod subs, but B cant not accept message.How can I fix this?
Awesome Library!!!
I am using v1.1.0 and now want to upgrade it to latest version v.1.3.0.
Thanks in advance
For some reason, I keep getting EOF error when trying to connect using Go HTTP client.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.