Comments (5)
Thank you for raising an issue before starting your work! ๐ฐ
Generally happy to add AMF parsing, my only concern here is that I'd like to avoid adding a dependency on an additional library. In particular, I'm very strongly -1 on anything that involves file format parsing in C. Both from a memory safety perspective as well as a "needs a compiler to install" perspective. Is there maybe already an existing Kaitai Struct definition for AMF somewhere? Not sure how complex AMF is, but if this can be done with Kaitai this would be by far my preferred option. :)
from mitmproxy.
Hey, status update: I managed to get some amf0 parsing and it works well against the https://github.com/sile/amf/tree/master/src/testdata dataset. Here's the Kaitai Struct YAML file:
amf0.v1.ksy.txt, (I couldn't find amf ksy files online, maybe I'm bad at searching) and here's what parsing amf0-ref-test.bin
gives:
It's pretty verbose, maybe there are KS tricks to reduce the amount of layers.
I didn't have a chance yet to plug that in mitmproxy to observe my own intercepted data. I'm specifically wondering if it's raw amf0/amf3 objects or if they're packed in the custom amf-packet format. Because both could be used, and there's no way to tell amf-packet data from amf0/3 data without attempting to parse it. I guess I'll have to offer both, even if having amf-packet
and amf
as types feels superfluous, I don't foresee a viable alternative.
Are there examples of Kaitai Struct -parsed formats in mitmproxy I could borrow code from ?
from mitmproxy.
from mitmproxy.
Hey, status update:
- amf0 : more or less OK. could be enhanced: boolean, undefined, null, references, times
- amf-packet : OK (!) (or is it ? :D)
- amf3 : just started, I'm struggling with the U29 integer format they invented, pretty much UTF-8-for-uint32. There will be the same possible enhancements as for amf0 on data types, and I'll also have to sort out some bitfield issues. (Wouldn't be fun if these bitfield flags were not applied to U29 integers, yay.)
I got some pointers from the peeps developing ruffle.rs, the flash runtime. My own dataset is amf-packet with headers packing amf0 objects which sometimes are amf3 objects. I've been pointed at some random flash games that do generate amf-packet data as well. Unless I see raw amf0 objects on the wire then the parsing root will be amf-packet.
from mitmproxy.
Status update
I managed to implement the U29 format, but am stuck at amf3_object_type
, and can't reasonably invest more energy & time into this initiative ยซ at this time ยป.
Blockers
- I couldn't get the meaning of the specification for
amf3_object_type
or the few implementations I could find. I tried readingsile/amf
,ruffle-rs/rust-flash-lso
andhydralabs/pyamf
and still could not make sense of the amf3 object type, notably when compared to real-world data. - The resources I can invest in this are too scarce. Real life. This, or I'm too stupid to read a spec. I guess I'll never know ๐คท :D.
Remaining work
- Ideally, you just have to sort the amf3_object_type structure, possibly array and dictionary will require similar handling.
- Then, plugging the spec in plugging it in
mitmproxy
might not be that hard. I don't know hehe.
Contribution so far
- amf-packet : implemented
- amf0 : implemented, and tested to some extent against the
sile/amf
dataset and some custom flash app. - Limitation: "null", "undefined" and other JavaScript-ish objects are not well handled by Kaitai Struct so we just skip them
- Limitation: so far, there's no support for references, I'm not sure if it's possible (or worth it) to perform bookkeeping like that in Kaitai Struct.
- I managed to sort out that u29 format, which isn't the vlq_base128 found in the Kaitai Struct standard set.
Enclosed .KSY file
Anyways, here's my amf.ksy
file so far, I'm releasing it as public domain for whomever wants to finish tackling this down. I'd be very honored if anyone managed to reuse my work to some extent.
Thanks for mitmproxy, it really is a great tool I keep recommending anytime I get a chance.
from mitmproxy.
Related Issues (20)
- Internet not working without mitmproxy enabled
- won't log localhost traffic on windows HOT 10
- Random SIGPIPE error when writing to a closed socket on a surge of requests HOT 8
- Async `client_connected` hook is broken in 10.2.4 HOT 9
- Update aioquic dependency to >= 1.0.0, < 2.0.0
- Breaking change in addon API inherited from (`cryptography`?) HOT 3
- Standalone Binaries: linux-arm64 (aarch64) HOT 8
- mitmdump stdout never reaches parent process HOT 25
- Can't handle a connection without server or with timeout HOT 5
- Unexpected event type at TCPLayer.relay_messages: Expected DataReceived|ConnectionClosed|TcpMessageInjected, got Start({}) HOT 13
- Deprecation Warning for not_valid_after in certs.py
- different output from io.FlowReader and io.FlowWrite HOT 1
- ERR_CONNECTION_RESET on android emulator HOT 2
- Cannot type header filter expression alone in mitmweb but no problem with the terminal mitmproxy
- Request to add a "clear all" button to the web interface HOT 2
- 403 access forbidden for www.youtube.com
- (trapped) error reading bcrypt version HOT 1
- Mitmdump does not exit HOT 1
- WireGuard mode in mitmproxy where find client config file? How work with it? Any documentation? HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. ๐๐๐
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google โค๏ธ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from mitmproxy.