GithubHelp home page GithubHelp logo

asyncins / aiowebsocket Goto Github PK

View Code? Open in Web Editor NEW
318.0 5.0 55.0 1.07 MB

Async WebSocket Client. Advantage: Flexible Lighter and Faster

Python 100.00%
asyncio websocket async python socket websockets aio aiowebsocket aiowebsockets

aiowebsocket's Introduction

aiowebsocket: Asynchronous websocket client

AioWebSocket is an asynchronous WebSocket client that

follows the WebSocket specification and is lighter and faster than other libraries.

AioWebSocket是一个遵循 WebSocket 规范的 异步 WebSocket 客户端,相对于其他库它更轻、更快。

images

Why is it Lighter?
Code volume just 30 KB
Why is it Faster?
it is based on asyncio and asynchronous

Installation

pip install aiowebsocket

Usage

The relationship between WSS and WS is just like HTTPS and HTTP.

ws and wss

Now it can automatically recognize WS and WSS

import asyncio
import logging
from datetime import datetime
from aiowebsocket.converses import AioWebSocket


async def startup(uri):
    async with AioWebSocket(uri) as aws:
        converse = aws.manipulator
        message = b'AioWebSocket - Async WebSocket Client'
        while True:
            await converse.send(message)
            print('{time}-Client send: {message}'
                  .format(time=datetime.now().strftime('%Y-%m-%d %H:%M:%S'), message=message))
            mes = await converse.receive()
            print('{time}-Client receive: {rec}'
                  .format(time=datetime.now().strftime('%Y-%m-%d %H:%M:%S'), rec=mes))


if __name__ == '__main__':
    remote = 'wss://echo.websocket.org'
    # remote = 'ws://echo.websocket.org'
    try:
        asyncio.get_event_loop().run_until_complete(startup(remote))
    except KeyboardInterrupt as exc:
        logging.info('Quit.')

custom header

aiowebsocket just build a request header that meets the websocket standard, but some websites need to add additional information so that you can use a custom request header,like this:

import asyncio
import logging
from datetime import datetime
from aiowebsocket.converses import AioWebSocket


async def startup(uri, header):
    async with AioWebSocket(uri, headers=header) as aws:
        converse = aws.manipulator
        message = b'AioWebSocket - Async WebSocket Client'
        while True:
            await converse.send(message)
            print('{time}-Client send: {message}'
                  .format(time=datetime.now().strftime('%Y-%m-%d %H:%M:%S'), message=message))
            mes = await converse.receive()
            print('{time}-Client receive: {rec}'
                  .format(time=datetime.now().strftime('%Y-%m-%d %H:%M:%S'), rec=mes))


if __name__ == '__main__':
    remote = 'ws://123.207.167.163:9010/ajaxchattest'
    header = [
        'GET /ajaxchattest HTTP/1.1',
        'Connection: Upgrade',
        'Host: 123.207.167.163:9010',
        'Origin: http://coolaf.com',
        'Sec-WebSocket-Key: RmDgZzaqqvC4hGlWBsEmwQ==',
        'Sec-WebSocket-Version: 13',
        'Upgrade: websocket',
        ]
    try:
        asyncio.get_event_loop().run_until_complete(startup(remote, header))
    except KeyboardInterrupt as exc:
        logging.info('Quit.')

union header

Consider: because AIO provides the basic request header, and sometimes does not need to replace all the request headers, but only need to add or replace a field in the request header. So with the union_header parameter added, you can replace or add fields in the request header, such as Origin.

import asyncio
import logging
from datetime import datetime
from aiowebsocket.converses import AioWebSocket


async def startup(uri, union_header):
    async with AioWebSocket(uri, union_header=union_header) as aws:
        converse = aws.manipulator
        message = b'AioWebSocket - Async WebSocket Client'
        while True:
            await converse.send(message)
            print('{time}-Client send: {message}'
                  .format(time=datetime.now().strftime('%Y-%m-%d %H:%M:%S'), message=message))
            mes = await converse.receive()
            print('{time}-Client receive: {rec}'
                  .format(time=datetime.now().strftime('%Y-%m-%d %H:%M:%S'), rec=mes))


if __name__ == '__main__':
    remote = 'ws://123.207.167.163:9010/ajaxchattest'
    union_header = {'Origin': 'http://coolaf.com'}
    try:
        asyncio.get_event_loop().run_until_complete(startup(remote, union_header))
    except KeyboardInterrupt as exc:
        logging.info('Quit.')

union_header must be dict.

With union, the request header information becomes:

# union_header = {'Origin': 'http://coolaf.com'}

# before
GET /chat HTTP/1.1
Host: server.example.com
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==
Origin: Python/3.7
# after
GET /chat HTTP/1.1
Host: server.example.com
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==
Origin: http://coolaf.com

Todo list

  • 整体测试:虽然在开发过程中做了很多测试,但是没有使用 TestCase 进行功能性测试,后期有时间会专门编写 aiowebsocket 的 Testase。
  • 动作预处理:create/close connection 以及 open 等动作的预处理暂未设定,在 websockets 源码中有预处理的**痕迹,我认为这是非常好的,值得我学习。
  • 问题修正:记录使用过程中出现的问题,并腾出时间进行调优和修正。

版本记录

  • 2019-03-07 aiowebsocket 1.0.0 dev-2 发布,新增自动识别和处理 ssl 能力、单个请求头字段添加/替换功能,优化数据帧读取逻辑。
  • 2019-03-05 aiowebsocket 1.0.0 dev-1 发布,dev-1 版本具备 ws 和 wss 协议的连接能力,并且支持自定义 header。

作者信息

参考资料

开发故事

在开发 aiowebsocket 库之前,我参考了 websocket-client 和 websockets 这两个库,在阅读过源码以及使用过后觉得 WebSocket 的连接不仅仅要像它们一样方便,还要更轻、更快、更灵活,在代码结构上还可以更清晰。所以我在完全不懂 WebSocket 的情况下通过阅读文章、调试源码以及翻阅文档,最终用了 7 天时间完成 aiowebsocket 库的设计和开发。

目前 aiowebsocket 支持 ws 和 wss 这两种协议,同时允许使用自定义请求头,这极大的方便了使用者。下图是 aiowebsocket 库文件结构以及类的设计图:

images

相比 websockets 库的结构,aiowebsocket 库的文件结构非常清晰,并且代码量很少。由于 websockets 库用的是 asyncio 旧语法,并且通过继承StreameProtocol 实现自定义协议,加上功能设计不明确(有很多不明确的预处理和 pending task 存在),所以导致它的结构比较混乱。

整个 websockets 库的源码图我没有画出,但是在调试时候有绘制改进图,WebSocketsCommonProtocol 协议(改进草图)类似下图:

images

这是仅仅协议的改进草稿,还不包括其他模块。实际上源码的逻辑更为混乱,这也是导致我费尽心力设计一个新库的原因之一。

WebSocket 及协议相关知识

什么是 WebSocket、WebSocket的优势、Python Socket、WebSocket 协议

请求头与握手连接、数据帧、Data Frame、Control Frame、掩码 Mask、平公公与彭公公

以上列出的知识,可以阅读我在掘金社区发表的文章 WebSocket 从入门到写出开源库

WebSocket status Code tools.ietf.org

状态码 名称 含义描述
0~999 保留使用
1000 CLOSE_NORMAL 正常关闭; 无论为何目的而创建, 该链接都已成功完成任务.
1001 CLOSE_GOING_AWAY 终端离开, 可能因为服务端错误, 也可能因为浏览器正从打开连接的页面跳转离开.
1002 CLOSE_PROTOCOL_ERROR 由于协议错误而中断连接.
1003 CLOSE_UNSUPPORTED 由于接收到不允许的数据类型而断开连接 (如仅接收文本数据的终端接收到了二进制数据).
1004 保留. 其意义可能会在未来定义.
1005 CLOSE_NO_STATUS 保留. 表示没有收到预期的状态码.
1006 CLOSE_ABNORMAL 保留. 用于期望收到状态码时连接非正常关闭 (也就是说, 没有发送关闭帧).
1007 Unsupported Data 由于收到了格式不符的数据而断开连接 (如文本消息中包含了非 UTF-8 数据).
1008 Policy Violation 由于收到不符合约定的数据而断开连接. 这是一个通用状态码, 用于不适合使用 1003 和 1009 状态码的场景.
1009 CLOSE_TOO_LARGE 由于收到过大的数据帧而断开连接.
1010 Missing Extension 客户端期望服务器商定一个或多个拓展, 但服务器没有处理, 因此客户端断开连接.
1011 Internal Error 客户端由于遇到没有预料的情况阻止其完成请求, 因此服务端断开连接.
1012 Service Restart 服务器由于重启而断开连接.
1013 Try Again Later 服务器由于临时原因断开连接, 如服务器过载因此断开一部分客户端连接.
1014 由 WebSocket标准保留以便未来使用.
1015 TLS Handshake 保留. 表示连接由于无法完成 TLS 握手而关闭 (例如无法验证服务器证书).
1000–2999 保留用于定义此协议,其未来的修订版和在。中指定的扩展名永久和随时可用的公共规范。
3000–3999 保留供使用库/框架/应用程序。这些状态代码是直接在IANA注册。这些代码的解释该协议未定义。
4000–4999 保留供私人使用因此无法注册。这些代码可以由先前使用WebSocket应用程序之间的协议。解释这个协议未定义这些代码。

aiowebsocket's People

Contributors

asyncins avatar newscli avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

aiowebsocket's Issues

Todo list: typo error

整体测试:虽然在开发过程中做了很多测试,但是没有使用 TestCase 进行功能性测试,后期有时间会专门编写 aiowebsocket 的

Testase
Testcase?

请求地址带token如何自动更换

1.在谷歌浏览器中查看WSS,token会每隔一段时间请求获取新的token,然后变更WSS地址,重新交互数据,请问这种情况怎么办?

2.交互的数据如何保存在桌面.txt

3.或者还有什么方式能将交互数据推送到外部,比如web

添加自定义头后不能运行,不添加自定义头会被封

import asyncio
import logging
from datetime import datetime
from aiowebsocket.converses import AioWebSocket
async def startup(uri,problem):
header=[
'GET wss://www.hfwh.top/ HTTP/1.1',
'Host: www.hfwh.top',
'Origin: https://www.woxunbudao.cn',
'Upgrade: websocket',
'Sec-WebSocket-Version: 13',
'Connection: Upgrade',
'Sec-WebSocket-Accept: g+8vNmGT8HaLHnj3QcQ/+0Nr5rM=',
]
async with AioWebSocket(uri,headers=header) as aws:
converse = aws.manipulator
Problem=problem
message = '{"type":1,"chars":"'+Problem+'"}'
print(message)
await converse.send(message)#发送数据
print('Client send: {message} \n'.format(message=Problem))
mes = await converse.receive()#接收数据
print('Client receive: {rec} \n'.format(rec=mes))
return mes
if name=='main':
remote='wss://www.hfwh.top/'
problem='科举中"殿试"的主考官是'
print(problem)
print(remote)
try:
asyncio.get_event_loop().run_until_complete(startup(remote,problem))
except KeyboardInterrupt as exc:
logging.info('Quit.')

证书一直提示出错

ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:1091)
证书的问题一直解决不了,查了资料都是https协议的

ping check expired

错误信息:每当出现下面这条消息后,
2019-11-12 10:17:21-Client receive: b'\x03\xebping check expired, session: 5f9d157d-15ec-4b6a-b30d-681daaa06dd7'
就会抛出错误

Traceback (most recent call last):
  File "/usr/local/Caskroom/miniconda/base/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3326, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-9-9a06e2fb4ad2>", line 26, in <module>
    asyncio.get_event_loop().run_until_complete(startup(remote))
  File "/usr/local/Caskroom/miniconda/base/lib/python3.7/asyncio/base_events.py", line 584, in run_until_complete
    return future.result()
  File "<ipython-input-9-9a06e2fb4ad2>", line 14, in startup
    mes = await converse.receive()
  File "/usr/local/Caskroom/miniconda/base/lib/python3.7/site-packages/aiowebsocket/converses.py", line 103, in receive
    single_message = await self.frame.read(text, mask)
  File "/usr/local/Caskroom/miniconda/base/lib/python3.7/site-packages/aiowebsocket/freams.py", line 238, in read
    fin, code, rsv1, rsv2, rsv3, message = await self.unpack_frame(mask, maxsize)
  File "/usr/local/Caskroom/miniconda/base/lib/python3.7/site-packages/aiowebsocket/freams.py", line 209, in unpack_frame
    frame_header = await reader(2)
  File "/usr/local/Caskroom/miniconda/base/lib/python3.7/asyncio/streams.py", line 677, in readexactly
    raise IncompleteReadError(incomplete, n)
asyncio.streams.IncompleteReadError: 0 bytes read on a total of 2 expected bytes

不知aiowebsocket的心跳响应是如何运作的,有没有内置的解决方法?或者是否支持自定义处理?

想请问下,发送的数据能否是16进制字节

如, await converse.send(message),用 websocket-client 库的发送是,

x = [8, 1, 26, 7, 8, 184, 232, 190, 199, 220, 44]
ws.send(x, websocket.ABNF.OPCODE_BINARY)

那么aiowebsocket 要怎么实现??

await converse.receive() 抛出Task exception was never retrieved

async def startup(uri):
    async with AioWebSocket(uri=uri) as aws:
        converse = aws.manipulator
        jsonTxt = {
            'id': 1,
            'jsonrpc': '2.0',
            'method': "subscribe",
            'params': {
                'channel': "pools"
            }
        }
        msg = json.dumps(jsonTxt)
        await converse.send(msg)

        while True:
            receive = await converse.receive()
            print(receive.decode())
            continue


if __name__ == '__main__':
    tasks = [
        startup("ws://host:8080/a"),
        startup("ws://host:8080/b"),
        startup("ws://host:8080/c")
    ]

    loop = asyncio.get_event_loop()
    loop.run_until_complete(asyncio.wait(tasks))

我的代码逻辑大概如上,目的是建立多个websocket连接。但是有一个连接在接受到第一次数据之后,服务器在很长一段时间内都无法吐回数据,然后就出现了如下的异常,异常的原因是服务器太长时间没有回吐数据了

PS D:\my_code\my_crpyto_tool\LPMoniter> python .\wsGet.py
{"jsonrpc":"2.0","result":{"status":"ok","data":"ready"},"id":1}
Task exception was never retrieved
future: <Task finished coro=<startup() done, defined at .\wsGet.py:29> exception=IncompleteReadError('0 bytes read on a total of 2 expected bytes')>
Traceback (most recent call last):
  File ".\wsGet.py", line 58, in startup
    receive = await converse.receive()
  File "D:\Program\Anaconda3\lib\site-packages\aiowebsocket\converses.py", line 103, in receive
    single_message = await self.frame.read(text, mask)
  File "D:\Program\Anaconda3\lib\site-packages\aiowebsocket\freams.py", line 238, in read
    fin, code, rsv1, rsv2, rsv3, message = await self.unpack_frame(mask, maxsize)
  File "D:\Program\Anaconda3\lib\site-packages\aiowebsocket\freams.py", line 209, in unpack_frame
    frame_header = await reader(2)
  File "D:\Program\Anaconda3\lib\asyncio\streams.py", line 677, in readexactly
    raise IncompleteReadError(incomplete, n)
asyncio.streams.IncompleteReadError: 0 bytes read on a total of 2 expected bytes

服务的逻辑是,如果客户端发送 "ping“ 能主动回应 "pong”,我有试过在捕获异常然后调用 await converse.send(ping) 从而在下一次while 循环的时候拿到“pong”结果,从而保持连接,但是这个时候websocket连接已经断开了,这里能不能再出现异常之后继续保持连接呢?

while True:
    try:
        receive = await converse.receive()
    except Exception as e:
        print(e)
        await converse.send("ping")
        continue

receive()数据接收是否有长度限制

当数据长度比较长时,数据只能获取前面一部分,这个有哪里需要设置吗
class Converse:
"""Responsible for communication
between client and server
"""
def init(self, reader: object, writer: object, maxsize: int = 2**16):
self.reader = reader
self.writer = writer
self.message_queue = Queue(maxsize=maxsize)
self.frame = Frames(self.reader, self.writer)
看到您写的这个maxsize是用来限制数据长度的对吗,为什么加这样的限制呢?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.