amyangfei / gormw Goto Github PK
View Code? Open in Web Editor NEWGoReplay Middleware in Python
License: MIT License
GoReplay Middleware in Python
License: MIT License
Hello!
I'm trying to use goreplay & middleware while processing pcap. I just simple want to log requests/responses to stderr. I slightly modified your example in readme.
run command:
gor --input-raw my.pcap:80 -output-http "http://server:8080" --output-http-track-response --middleware "middleware_gor.py" --input-raw-track-response --input-raw-engine "pcap_file"
Middleware file is bellow:
# coding: utf-8
import sys
from gor.middleware import AsyncioGor
def log(msg):
"""
Logging to STDERR as STDOUT and STDIN used for data transfer
@type msg: str or byte string
@param msg: Message to log to STDERR
"""
try:
msg = str(msg) + '\n'
except:
pass
sys.stderr.write(msg)
sys.stderr.flush()
def on_request(proxy, msg, **kwargs):
log("fromtype:1 id: {}".format(msg.id))
proxy.on('response', on_response, idx=msg.id, req=msg)
def on_response(proxy, msg, **kwargs):
log("fromtype:2 id: {}".format(msg.id))
proxy.on('replay', on_replay, idx=kwargs['req'].id, req=kwargs['req'], resp=msg)
def on_replay(proxy, msg, **kwargs):
log("fromtype:3 id: {}".format(msg.id))
if __name__ == '__main__':
proxy = AsyncioGor()
proxy.on('request', on_request)
proxy.run()
What am I doing wrong?
如果接口是上传文件,比如上传图片,body内容在decode的时候,不能转成字符串,就会报错了
" 'utf-8' codec can't decode byte 0x89 in position 715: invalid start byteTraceback (most recent call last):
File "/Users/rc/Library/Python/3.9/lib/python/site-packages/gor/base.py", line 70, in parse_message
payload = binascii.unhexlify(line.strip()).decode()
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x89 in position 715: invalid start byte
python 3.4版本以上都是tornado 6.x的版本,都有这个问题
step 1:
I use the command to start goplay
gor --input-raw :8000 \
--output-stdout \
--output-file request.log \
--input-raw-track-response \
--output-http-track-response \
--prettify-http \
--output-tcp diferencia:8081 \
--http-disallow-header 'User-Agent: Gor'
step 2:
and use the demo code to start the goplay middleware
# coding: utf-8
import sys
from gor.middleware import AsyncioGor
def on_request(proxy, msg, **kwargs):
proxy.on('response', on_response, idx=msg.id, req=msg)
def on_response(proxy, msg, **kwargs):
proxy.on('replay', on_replay, idx=kwargs['req'].id, req=kwargs['req'], resp=msg)
def on_replay(proxy, msg, **kwargs):
replay_status = proxy.http_status(msg.http)
resp_status = proxy.http_status(kwargs['resp'].http)
if replay_status != resp_status:
print('replay status [%s] diffs from response status [%s]\n' % (replay_status, resp_status))
else:
print('replay status is same as response status\n')
sys.stderr.flush()
if __name__ == '__main__':
proxy = AsyncioGor()
proxy.on('request', on_request)
proxy.run()
when I send http message to goplay by curl,But I can`t get message from the middleware?
I need help
payload = binascii.unhexlify(line.strip()).decode()
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x8b in position 4100: invalid start byte
Note that issue is seen even with goreplay's --prettify-http
flag. Ref: buger/goreplay#400 (comment)
最近在看goreplay的middleware部分,发现了这个包,感觉还是比较好用的,在使用过程中,我感觉还有一些方法可以提供,现在的获取header,获取查询参数都需要加name参数,作者有考虑过写几个直接返回所有header,所有参数的方法吗,在使用过程中,可能会需要将请求划分为url,params,headers,method这样的部分,所以有一些能够直接将msg解析为通用请求结构的方法会好一点
ref ci job with python3.10: https://app.travis-ci.com/github/amyangfei/GorMW/jobs/578993409
=============================== warnings summary ===============================
tests/test_asyncio_gor.py::TestAsyncioGor::test_run
/home/travis/build/amyangfei/GorMW/gor/asyncio_impl.py:56: DeprecationWarning: There is no current event loop
self.io_loop = asyncio.get_event_loop()
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
Message callback is append only, there is no GC mechanism currently.
Line 57 in c8cf1cf
Similar NodeJS library GC strategy: buger/goreplay@debb265
# coding: utf-8
import sys
from gor.middleware import AsyncioGor
def on_request(proxy, msg, **kwargs):
proxy.on('response', on_response, idx=msg.id, req=msg)
def on_response(proxy, msg, **kwargs):
proxy.on('replay', on_replay, idx=kwargs['req'].id, req=kwargs['req'], resp=msg)
def on_replay(proxy, msg, **kwargs):
replay_status = proxy.http_status(msg.http)
resp_status = proxy.http_status(kwargs['resp'].http)
if replay_status != resp_status:
sys.stderr.write('replay status [%s] diffs from response status [%s]\n' % (replay_status, resp_status))
else:
sys.stderr.write('replay status is same as response status\n')
sys.stderr.flush()
if __name__ == '__main__':
proxy = AsyncioGor()
proxy.on('request', on_request)
proxy.run()
replay_resp appears earlier than origin_resp, and on_replay depends on on_response. on_replay method is not registered at the time of replay_resp's appearance
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.