machinezone / cobra Goto Github PK
View Code? Open in Web Editor NEWCobra is a realtime messaging server using Python3, WebSockets and Redis
License: BSD 3-Clause "New" or "Revised" License
Cobra is a realtime messaging server using Python3, WebSockets and Redis
License: BSD 3-Clause "New" or "Revised" License
Cobra can receive data in a weird encoding, we can use chardet to try to detect it and 'recover'.
Problem could also be linked to the fact that TEXT is sent while historically cobra has received BINARY data (websocket type).
We should close all connections, ideally with an explicit close reason sent to connected clients.
According to this we receive a SIGTERM when kubernete restart your app.
Dev note: test withkill -s TERM <pid>
2019-08-21 11:58:08 ERROR Unhandled exception in async future
future: <Task finished coro=<redisSubscriber() done, defined at /Users/bsergeant/src/foss/cobra/cobras/server/redis_subscriber.py:44> exception=TypeError('respond() takes 3 positional arguments but 4 were given')>
Traceback (most recent call last):
File "/Users/bsergeant/src/foss/cobra/cobras/server/redis_subscriber.py", line 60, in redisSubscriber
await messageHandler.on_init(connection)
File "/Users/bsergeant/src/foss/cobra/cobras/server/handlers/pubsub.py", line 204, in on_init
await self.state.respond(self.ws, self.app, response)
TypeError: respond() takes 3 positional arguments but 4 were given
^C
Which kinds of errors should be send to sentry is also up in the air.
[2020-08-26 21:23:31.243] [ws] [info] messages received 0 4562752 sent 0 4562752
[2020-08-26 21:23:32.243] [ws] [info] messages received 0 4562752 sent 0 4562752
[2020-08-26 21:23:32.908] [ws] [info] Subscriber closed: {}Close code 1011 reason Ping timeout
[2020-08-26 21:23:33.007] [ws] [info] Subscriber connected
[2020-08-26 21:23:33.007] [ws] [info] Connection: upgrade
[2020-08-26 21:23:33.007] [ws] [info] Date: Wed, 26 Aug 2020 21:23:33 GMT
[2020-08-26 21:23:33.007] [ws] [info] Sec-WebSocket-Accept: Kwn3MLpeFbF3b78mGf/Ld3ew8P0=
[2020-08-26 21:23:33.007] [ws] [info] Sec-WebSocket-Extensions: permessage-deflate; server_max_window_bits=15; client_max_window_bits=15
[2020-08-26 21:23:33.007] [ws] [info] Sec-WebSocket-Protocol: json
[2020-08-26 21:23:33.007] [ws] [info] Server: nginx/1.15.10
[2020-08-26 21:23:33.007] [ws] [info] Strict-Transport-Security: max-age=15724800; includeSubDomains
[2020-08-26 21:23:33.007] [ws] [info] Upgrade: websocket
[2020-08-26 21:23:33.007] [ws] [info] X-Cobra-Node: cobra-subscribers-250-d9b5w
[2020-08-26 21:23:33.007] [ws] [info] X-Cobra-Version: 2.9.97
[2020-08-26 21:23:33.010] [ws] [info] Subscriber closed: {}Close code 1011 reason Ping timeout
[2020-08-26 21:23:33.103] [ws] [info] Subscriber connected
[2020-08-26 21:23:33.103] [ws] [info] Connection: upgrade
[2020-08-26 21:23:33.103] [ws] [info] Date: Wed, 26 Aug 2020 21:23:33 GMT
[2020-08-26 21:23:33.103] [ws] [info] Sec-WebSocket-Accept: SNt31vb4U809r1RWthd/MB78PEc=
[2020-08-26 21:23:33.103] [ws] [info] Sec-WebSocket-Extensions: permessage-deflate; server_max_window_bits=15; client_max_window_bits=15
[2020-08-26 21:23:33.103] [ws] [info] Sec-WebSocket-Protocol: json
[2020-08-26 21:23:33.103] [ws] [info] Server: nginx/1.15.10
[2020-08-26 21:23:33.103] [ws] [info] Strict-Transport-Security: max-age=15724800; includeSubDomains
[2020-08-26 21:23:33.103] [ws] [info] Upgrade: websocket
[2020-08-26 21:23:33.103] [ws] [info] X-Cobra-Node: cobra-subscribers-250-d9b5w
[2020-08-26 21:23:33.103] [ws] [info] X-Cobra-Version: 2.9.97
[2020-08-26 21:23:33.105] [ws] [info] Received websocket pong: ixwebsocket::heartbeat::30s::374
[2020-08-26 21:23:33.148] [ws] [info] Subscriber authenticated
[2020-08-26 21:23:33.148] [ws] [info] Subscribing to blah_id
[2020-08-26 21:23:33.148] [ws] [info] Subscribing at position 1598476980499-0
[2020-08-26 21:23:33.148] [ws] [info] Subscribing with filter
[2020-08-26 21:23:33.207] [ws] [info] Subscriber: subscribed to channel blah_id
[2020-08-26 21:23:33.244] [ws] [info] messages received 0 4562752 sent 0 4562752
[2020-08-26 21:23:34.244] [ws] [info] messages received 34 4562786 sent 34 4562786
[2020-08-26 21:23:35.244] [ws] [info] messages received 876 4563662 sent 875 4563661
Dummy position: 1519190184:559034812775
cobra/cobras/server/handlers/pubsub.py
Line 205 in d9cb898
Position validation:
POSITION_PATTERN = re.compile('^(?P<id1>[0-9]+)-(?P<id2>[0-9]+)')
def validatePosition(position):
if position is None or position == '$':
return True
return POSITION_PATTERN.match(position)
Hello!
I hope you are doing well!
We are a security research team. Our tool automatically detected a vulnerability in this repository. We want to disclose it responsibly. GitHub has a feature called Private vulnerability reporting, which enables security research to privately disclose a vulnerability. Unfortunately, it is not enabled for this repository.
Can you enable it, so that we can report it?
Thanks in advance!
PS: you can read about how to enable private vulnerability reporting here: https://docs.github.com/en/code-security/security-advisories/repository-security-advisories/configuring-private-vulnerability-reporting-for-a-repository
For
/Users/bsergeant/src/foss/cobra/venv/lib/python3.7/site-packages/asyncio_redis/protocol.py:549: DeprecationWarning: `formatargspec` is deprecated since Python 3.5. Use `signature` and the `Signature` object directly
signature = formatargspec(* self.specs[:6])
This could be used: sqlalchemy/alembic@e3c5fc3
SELECT returns an empty object when a field is missing because extractAttributeFromDict defaults the subtree to an empty dict instead of None.
cobra/cobras/common/algorithm.py
Lines 22 to 26 in 8c9d8b6
Exception 1
[20ffff89::pubsub] < {"action": "rtm/subscribe", "body": {"subscription_id": "football", "channel": "football", "fast_forward": true, "filter": null, "position": "0-0"}, "id": 3}
2019-08-09 00:55:49 cobra-79cd6f74b9-jhp7n asyncio[1] ERROR Future exception was never retrieved
future: <Future finished exception=ConnectionClosedError('Reader at end of file')>
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/aioredis/commands/transaction.py", line 208, in _send_pipeline
result_fut = conn.execute(cmd, *args, **kw)
File "/usr/local/lib/python3.7/site-packages/aioredis/connection.py", line 319, in execute
raise ConnectionClosedError(msg)
aioredis.errors.ConnectionClosedError: Reader at end of file
2019-08-09 00:55:49 cobra-79cd6f74b9-jhp7n asyncio[1] ERROR Future exception was never retrieved
Exception 2
[c4805c5d::pubsub] > {"action": "auth/authenticate/ok", "id": 1, "body": {}}
[c4805c5d::pubsub] < {"action": "rtm/subscribe", "body": {"subscription_id": "football", "channel": "football", "fast_forward": true, "filter": null, "position": "0-0"}, "id": 3}
[c4805c5d::pubsub] > {"action": "rtm/subscribe/ok", "id": 3, "body": {"position": "1519190184:559034812775", "subscription_id": "football", "redis_node": "redis-2-service"}}
[c4805c5d::pubsub] < {"action": "rtm/publish", "body": {"channel": "football", "message": {"data": {"user": "john", "text": "cadsc"}}}}
'NoneType' object has no attribute 'write'
Generic Exception caught in Traceback (most recent call last):
File "/cobras/server/app.py", line 49, in cobraHandler
await processCobraMessage(state, websocket, app, message)
File "/cobras/server/protocol.py", line 588, in processCobraMessage
await handler(state, ws, app, pdu, msg)
File "/cobras/server/protocol.py", line 201, in handlePublish
batchPublish)
File "/cobras/server/pipelined_publisher.py", line 51, in push
await self.publishNow(job)
File "/cobras/server/pipelined_publisher.py", line 47, in publishNow
await pipe.execute()
File "/usr/local/lib/python3.7/site-packages/aioredis/commands/transaction.py", line 180, in execute
return_exceptions=return_exceptions)
File "/usr/local/lib/python3.7/site-packages/aioredis/commands/transaction.py", line 187, in _do_execute
return_exceptions=True)
File "/usr/local/lib/python3.7/site-packages/aioredis/commands/transaction.py", line 214, in _send_pipeline
yield result_fut
File "/usr/local/lib/python3.7/contextlib.py", line 119, in __exit__
next(self.gen)
File "/usr/local/lib/python3.7/site-packages/aioredis/connection.py", line 300, in _buffered
self._writer.write(buf)
AttributeError: 'NoneType' object has no attribute 'write'
[c4805c5d::pubsub] cancelling #1 subscriptions
[c4805c5d::pubsub] (close) uptime 0:00:02 msgcount 4 connections 1
[c4805c5d::pubsub] Cancelling redis subscription
[c4805c5d::pubsub] Closing redis subscription
[4b166ffa::pubsub] UNBLOCKED force unblock from blocking operation, instance state changed (master -> replica?)
[4b166ffa::pubsub] Generic Exception caught in Traceback (most recent call last):
File "/cobras/server/redis_subscriber.py", line 52, in redisSubscriber
latest_ids=[lastId])
File "/usr/local/lib/python3.7/site-packages/aioredis/util.py", line 55, in wait_convert
result = await fut
aioredis.errors.ReplyError: UNBLOCKED force unblock from blocking operation, instance state changed (master -> replica?)
[4b166ffa::pubsub] Closing redis subscription
> 2019-08-19 12:14:57 INFO Recovering and creating a new connection
> 2019-08-19 12:15:03 ERROR Task exception was never retrieved
future: <Task finished coro=<runClient() done, defined at /Users/bsergeant/src/foss/cobra/cobras/bavarde/client/client.py:103> exception=InvalidStatusCode('server rejected WebSocket connection: HTTP 403')>
Traceback (most recent call last):
File "/Users/bsergeant/src/foss/cobra/cobras/bavarde/client/client.py", line 166, in runClient
await args['connection'].publish(channel, message)
KeyError: 'connection'
Maybe we can just try / catch that error.
Is it possible to add features like channel search, channel admin roles ( add subscriber, ban subscriber, grant admin rights )
(https://user-images.githubusercontent.com/57979884/74179621-dc512480-4c63-11ea-8e4e-84cc7aa18c03.png)
2020-08-27 18:37:10 WARNING publish: cannot connect to redis ERR Protocol error: unbalanced quotes in request
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.