mitmproxy / mitmproxy

An interactive TLS-capable intercepting HTTP proxy for penetration testers and software developers.

mitmproxy

mitmproxy is an interactive, SSL/TLS-capable intercepting proxy with a console interface for HTTP/1, HTTP/2, and WebSockets.

mitmdump is the command-line version of mitmproxy. Think tcpdump for HTTP.

mitmweb is a web-based interface for mitmproxy.

Installation

The installation instructions are here. If you want to install from source, see CONTRIBUTING.md.

Documentation & Help

General information, tutorials, and precompiled binaries can be found on the mitmproxy website.

The documentation for mitmproxy is available on our website:

If you have questions on how to use mitmproxy, please use GitHub Discussions!

Contributing

As an open source project, mitmproxy welcomes contributions of all forms.

mitmproxy's Issues

Replacement of entire file

Hello, congrats for mitmproxy :)

It's possibile to completely replace a response body with a local file? I tried this way but I failed:

--replace=:site.css:.*:~/site.css

thanks

decoded decorator leads to content encoding failures.

As already discussed yesterday, employing the decoded decorator in handle_response breaks some responses. I have not found a page where this is always reproducible, but it occasionally happens to me on www.spiegel.de

Here is a mitmproxy dump containing two requests to the same resource of which the first one fails. These dumps were produced before applying decorated to the response:

            if self.o.wfile:
                self.fwriter.add(flow)
            with decoded(flow.response):
                pass
            if self.o.wfile2:
                self.fwriter2.add(flow)

This file contains the same request after the decoded decorator has been applied. One possible solution might be caching the encoded version in HTTPMsg.decode. While this is definitely not a very nice and clean solution, it would guarantee that every flow get passed as-is.

Crash on openSUSE 12.1

If I run mitmproxy and set Firefox up to use it, I get the following crash:

Traceback (most recent call last):
File "/usr/local/bin/mitmproxy", line 64, in
m.run()
File "/usr/local/lib/python2.7/site-packages/libmproxy/console/init.py", line 538, in run
self.ui.run_wrapper(self.loop)
File "/usr/local/lib64/python2.7/site-packages/urwid/raw_display.py", line 242, in run_wrapper
return fn()
File "/usr/local/lib/python2.7/site-packages/libmproxy/console/init.py", line 747, in loop
changed = self.tick(self.masterq)
File "/usr/local/lib/python2.7/site-packages/libmproxy/flow.py", line 1421, in tick
return controller.Master.tick(self, q)
File "/usr/local/lib/python2.7/site-packages/libmproxy/controller.py", line 77, in tick
self.handle(msg)
File "/usr/local/lib/python2.7/site-packages/libmproxy/controller.py", line 95, in handle
m(msg)
File "/usr/local/lib/python2.7/site-packages/libmproxy/flow.py", line 1499, in handle_clientconnect
self.add_event("Connect from: %s:%s"%cc.address)
File "/usr/local/lib/python2.7/site-packages/libmproxy/console/init.py", line 948, in add_event
self.eventlist.set_focus(len(self.eventlist))
File "/usr/local/lib64/python2.7/site-packages/urwid/listbox.py", line 178, in set_focus
raise IndexError, "No widget at position %s" % (position,)
IndexError: No widget at position 1

I also get a crash if I try to bring up the eventlog:

Traceback (most recent call last):
File "/usr/local/bin/mitmproxy", line 64, in
m.run()
File "/usr/local/lib/python2.7/site-packages/libmproxy/console/init.py", line 538, in run
self.ui.run_wrapper(self.loop)
File "/usr/local/lib64/python2.7/site-packages/urwid/raw_display.py", line 242, in run_wrapper
return fn()
File "/usr/local/lib/python2.7/site-packages/libmproxy/console/init.py", line 766, in loop
k = self.view.keypress(size, k)
File "/usr/local/lib64/python2.7/site-packages/urwid/container.py", line 1103, in keypress
return self.body.keypress( (maxcol, remaining), key )
File "/usr/local/lib/python2.7/site-packages/libmproxy/console/flowlist.py", line 208, in keypress
self.master.toggle_eventlog()
File "/usr/local/lib/python2.7/site-packages/libmproxy/console/init.py", line 430, in toggle_eventlog
self.view_flowlist()
File "/usr/local/lib/python2.7/site-packages/libmproxy/console/init.py", line 585, in view_flowlist
self.body = flowlist.BodyPile(self)
File "/usr/local/lib/python2.7/site-packages/libmproxy/console/flowlist.py", line 76, in init
self.focus = 0
AttributeError: can't set attribute

They both seem to be related, but I'm not sure - any suggestions on how to resolve?

This was installed using pip earlier this evening. The system is x86_64 openSUSE 12.1 with the current updates (as of now) applied.

mitmdump seems to work OK.

Multiple scripts and script arguments.

mitmproxy is one hell of a tool but I've come across some drawbacks. Looking at the code template and usage it seems that only one script can be active at a time. A nice addition would be to allow multiple scripts per mitmproxy process that would process responses in reverse order to request, ie, the first script would get request first but response last. This would enable multiple scripts per process to be chained.

Passing arguments to scripts would be usefull as well, perhaps a command line parameter. This would enable slightly tweaking the functionality since I'm finding myself copying scripts over and making slight modifications and then chaining proxies together at the moment.

I'd be happy to help out but would prefer some guidence of how you would like these features implemented. No used in submitting a patch that doesn't get accepted. Any suggestions?

crash when editing the response to an intercepted request

When you try to create a response from the UI, mitmproxy crashes.

Steps to reproduce:

1. set an intercept option
2. choose the pending request ()
3. switch to the response ()
4. edit some field (, )

Result:

    Traceback (most recent call last):
      File "./mitmproxy", line 67, in <module>
        m.run()
      File "/usr/src/mitmproxy/libmproxy/console/__init__.py", line 583, in run
        self.ui.run_wrapper(self.loop)
      File "/usr/lib/python2.7/dist-packages/urwid/raw_display.py", line 242, in run_wrapper
        return fn()
      File "/usr/src/mitmproxy/libmproxy/console/__init__.py", line 806, in loop
        self.prompt_execute(k)
      File "/usr/src/mitmproxy/libmproxy/console/__init__.py", line 740, in prompt_execute
        msg = p(txt, *args)
      File "/usr/src/mitmproxy/libmproxy/console/flowview.py", line 340, in edit
        self.flow.response = flow.Response(self.flow.request, 200, "OK", flow.ODictCaseless(), "", None)
    TypeError: __init__() takes at least 8 arguments (7 given)

Stick window focus to bottom when new requests are intercepted

Production

Hi,

Quick question re mitmproxy. Would you consider it to be suitable for use in a production environment?

Best Regards,

Carl

domain_match not found in python 2.7 cookielib, sticky cookie not working

Hi Aldo,

I'm wondering if I'm doing things wrong but I don't manage to use sticky cookie feature and I noticed that domain_match is not present in python 2.7 cookielib so it may explain my issue, am I missing something ?

Regards,

Jerome

Python 2.6 and argparse

Good afternoon,

Your README file states that python 2.6 is supported, but python 2.6 does not contain argparse, just the deprecated optparse module.

Not sure what the route forward is, but I wanted to bring it to your attention.

Thanks,
Logan

Just not working

I am new to all of these concepts, so I mgiht be doing something wring, but the mitmproxy just isn't working. I set it up using pip (although there was an unimaginable about of warnings that came up), and since then whenever I follow the tutorial of launching mitmproxy, then typing in i then ~q, nothing ever shows up in the screen. Anybody have any ideas? (Maybe the pip install doesn't work on mountain lion?)

HTTP CONNECT method (for ssl) not supported?

I'm trying to proxy a connection that uses the HTTP CONNECT method and the request never shows up in the command window (although I know it's happening). Is this currently not supported in Mitmproxy? Thanks!

AttributeError: 'NoneType' object has no attribute 'acked'

I'm receiving the following traceback, while intercepting:

Traceback (most recent call last):
  File "./mitmproxy", line 61, in <module>
    m.run()
  File "/tmp/git/mitmproxy/libmproxy/console.py", line 1275, in run
    self.ui.run_wrapper(self.loop)
  File "/Library/Python/2.7/site-packages/urwid-1.0.1-py2.7-macosx-10.7-intel.egg/urwid/raw_display.py", line 237, in run_wrapper
    return fn()
  File "/tmp/git/mitmproxy/libmproxy/console.py", line 1750, in loop
    self.view.keypress(size, k)
  File "/Library/Python/2.7/site-packages/urwid-1.0.1-py2.7-macosx-10.7-intel.egg/urwid/container.py", line 641, in keypress
    return self.body.keypress( (maxcol, remaining), key )
  File "/tmp/git/mitmproxy/libmproxy/console.py", line 247, in keypress
    return urwid.ListBox.keypress(self, size, key)
  File "/Library/Python/2.7/site-packages/urwid-1.0.1-py2.7-macosx-10.7-intel.egg/urwid/listbox.py", line 766, in keypress
    key = focus_widget.keypress((maxcol,),key)
  File "/tmp/git/mitmproxy/libmproxy/console.py", line 193, in keypress
    self.master.view_flow(self.flow)
  File "/tmp/git/mitmproxy/libmproxy/console.py", line 1321, in view_flow
    self.body = ConnectionView(self, self.state, flow)
  File "/tmp/git/mitmproxy/libmproxy/console.py", line 302, in __init__
    self.view_request()
  File "/tmp/git/mitmproxy/libmproxy/console.py", line 359, in view_request
    self.w = self.wrap_body(VIEW_FLOW_REQUEST, body)
  File "/tmp/git/mitmproxy/libmproxy/console.py", line 326, in wrap_body
    if self.flow.intercepting and not self.flow.response.acked:
AttributeError: 'NoneType' object has no attribute 'acked'

MacOSX 10.7.3
Python 2.7.1

editing gzipped data

How is mitmproxy to handle the editing of gzipped data in an external editor?

Would it be better to let the editor bear the burden of uncompressing/recompressing, or would it be better if mitmproxy did this automatically?

If it's the former, I think adding the relevant extension to the file would help the editor identify it as zipped.

exception throw on request

Running from the latest in git master, I get the following exception flashed briefly, then disappear.

File "/usr/local/lib/python2.7/dist-packages/netlib/tcp.py", line 163, in request_thread
self.handle_connection(request, client_a File "/usr/local/lib/python2.7/dist-packages/netlib/tcp.py", line 163, in request_thread
self.handle_connection(request, client_address)
File "/usr/local/lib/python2.7/dist-packages/libmproxy/proxy.py", line 352, in handle_connection
h.handle()
File "/usr/local/lib/python2.7/dist-packages/libmproxy/proxy.py", line 118, in handle
while self.handle_request(cc) and not cc.close:
File "/usr/local/lib/python2.7/dist-packages/libmproxy/proxy.py", line 139, in handle_request
request = self.read_request(cc)
File "/usr/local/lib/python2.7/dist-packages/libmproxy/proxy.py", line 293, in read_request
method, scheme, host, port, path, httpversion = http.parse_init_proxy(line)
TypeError: 'NoneType' object is not iterable
ddress)
File "/usr/local/lib/python2.7/dist-packages/libmproxy/proxy.py", line 352, in handle_connection
h.handle()
File "/usr/local/lib/python2.7/dist-packages/libmproxy/proxy.py", line 118, in handle
while self.handle_request(cc) and not cc.close:
File "/usr/local/lib/python2.7/dist-packages/libmproxy/proxy.py", line 139, in handle_request
request = self.read_request(cc)
File "/usr/local/lib/python2.7/dist-packages/libmproxy/proxy.py", line 293, in read_request
method, scheme, host, port, path, httpversion = http.parse_init_proxy(line)
TypeError: 'NoneType' object is not iterable

multipart post not supported?

I am debugging an oauth client and mitmproxy can catch all GET request and application/x-www-form-urlencoded POST request but multipart/form-data POST request no show.

fatal error: 'libxml/xmlversion.h' file not found

I'm getting this error when trying to install it with pip in Mac OS X Mountain Lion.

This is the complete pip.log:
https://gist.github.com/4125461

I've all the latest XCode Command Line Tools installed.

Requires urwid

After a fresh install of version 0.6 from PyPi, when running mitmproxy I get the following error:

Traceback (most recent call last):
File "/usr/local/bin/mitmproxy", line 19, in
from libmproxy import proxy, controller, console, utils, flow, cmdline
File "/usr/local/lib/python2.7/dist-packages/libmproxy/console.py", line 19, in
import urwid
ImportError: No module named urwid

setup.py should probably reference urwid (I see it does in the latest code on github).

Allow JUNK HTTP method

When testing certain configurations it is not uncommon to get unusual results with a completely invalid HTTP method such as "JUNK".

Can "junk" be added to the changing of request methods to generate a JUNK request? EG:

JUNK /some.jsp HTTP/1.1
Host: target.tld
...

P

Replace optparse with argparse

Hi cortesi,

mitmproxy currently uses the deprecated optparse for handling console argument parsing. Would you welcome a patch that replaces optparse with argparse or are there any special reasons for sticking with optparse?
I am asking because we'd love to have support for argument files in HoneyProxy.

Thanks!

Max

requires PIL

the PIL requirement is not listed in the README
PIL is available at http://www.pythonware.com/products/pil/

no dummy wildcard SSL certs

Looks like the dummy SSL cert's CN is generated from the HOST header which doesn't really work with sites using wildcard certs. It would be more convenient if the dummy's CN was generated based on the pre-fetched real cert's CN.

Does it sound feasible?

proxy alters http requests to https also mangles https

some strange magic going on here, all http requests sent to the proxy are being converted into https by the proxy. Also all https requests are failing do to some mangling of the SSL handshake, Firefox error -12263 (SSL record was too long). My config is the following:

Ubuntu 10.04
Python 2.6.5
OpenSSL 0.9.8k

Debug isn't showing nearly enough info to troubleshoot.. At this point I'm weeding through libmproxy with print statements but if you have a quicker fix let me know.

Thanks

Service unavailable errors

I'm using arpspoof and mitmproxy on debian testing. When I try to access some sites.(like: eurosport.pl) (on proxied computer) browser tells Service Unavailable. When I disable arpspoof and put mitmproxy ip and port in browser configuration - it works well - no errors.

Can someone tell me what I am doing wrong? Or maybe it is a bug in mitmproxy?

Best regards,

Colors not suited for white terminal background

(Python 2.7.2 , OpenSUSE Linux 12.1, Gnome )

When launching mitmproxy I get very soft colours on a white background, which is not readable at all. I guess mitmproxy should force a black background color or add a command-line switch to allow the user to specify the background color.

Preview HTML using lynx

I'm pretty sure the issue is me not getting it.

I edited my ~/.mailcap, tried both those lines:

text/html; lynx %s; nametemplate=%s.html
text/html; lynx %s

Reopened my shiny 0.8 mitmproxy, and sadly, still no lynx while using 'v' on response pages (open in external viewer, if I'm not mistaken).

Am I doing something wrong?

Not working on OS X 10.8 - Symbol not found: _PyInt_Check

Any way I install it, I get:

 ✔ Marvin ⮀ mitmproxy-0.8.1 ⮀ mitmproxy                                                                                                                                                                                     (env: mitmproxy) 
Traceback (most recent call last):
  File "/Users/filosottile/Envs/mitmproxy/bin/mitmproxy", line 19, in <module>
    from libmproxy import proxy, console, cmdline
  File "/Users/filosottile/Envs/mitmproxy/lib/python2.7/site-packages/libmproxy/proxy.py", line 24, in <module>
    import utils, flow, certutils
  File "/Users/filosottile/Envs/mitmproxy/lib/python2.7/site-packages/libmproxy/flow.py", line 24, in <module>
    import controller, version, certutils
  File "/Users/filosottile/Envs/mitmproxy/lib/python2.7/site-packages/libmproxy/certutils.py", line 4, in <module>
    import OpenSSL
  File "/Users/filosottile/Envs/mitmproxy/lib/python2.7/site-packages/OpenSSL/__init__.py", line 36, in <module>
    from OpenSSL import crypto
ImportError: dlopen(/Users/filosottile/Envs/mitmproxy/lib/python2.7/site-packages/OpenSSL/crypto.so, 10): Symbol not found: _PyInt_Check
  Referenced from: /Users/filosottile/Envs/mitmproxy/lib/python2.7/site-packages/OpenSSL/crypto.so
  Expected in: flat namespace
 in /Users/filosottile/Envs/mitmproxy/lib/python2.7/site-packages/OpenSSL/crypto.so

Problem to proxify a request on a large file

Size file is about 5 Go and give an error with the following traceback :

Traceback (most recent call last):
File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/SocketServer.py", line 558, in process_request_thread
self.finish_request(request, client_address)
File "/Users/admin/Desktop/mitmproxy-0.6/libmproxy/proxy.py", line 422, in finish_request
self.RequestHandlerClass(self.config, request, client_address, self, self.masterq)
File "/Users/admin/Desktop/mitmproxy-0.6/libmproxy/proxy.py", line 224, in init
SocketServer.StreamRequestHandler.init(self, request, client_address, server)
File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/SocketServer.py", line 615, in init
self.handle()
File "/Users/admin/Desktop/mitmproxy-0.6/libmproxy/proxy.py", line 230, in handle
self.handle_request(cc)
File "/Users/admin/Desktop/mitmproxy-0.6/libmproxy/proxy.py", line 259, in handle_request
response = server.read_response()
File "/Users/admin/Desktop/mitmproxy-0.6/libmproxy/proxy.py", line 208, in read_response
content = read_http_body(self.rfile, self, headers, True)
File "/Users/admin/Desktop/mitmproxy-0.6/libmproxy/proxy.py", line 63, in read_http_body
content = rfile.read(int(headers["content-length"][0]))
File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/socket.py", line 351, in read
data = self._sock.recv(left)
OverflowError: signed integer is greater than maximum

Please, do you know a workaround ?
Thanks
O.

Add to homebrew?

Might be nice to install it with just brew install mitmproxy on Mac OS X.

Crash upon viewing JSON response

Trying to view a JSON response, content type application/json, crashes with the following stacktrace:

Traceback (most recent call last):
  File "./mitmproxy", line 68, in <module>
    m.run()
  File "/Users/mark/Code/mitmproxy/libmproxy/console.py", line 1145, in run
    self.ui.run_wrapper(self.loop)
  File "/usr/local/Cellar/python/2.7.1/lib/python2.7/site-packages/urwid/raw_display.py", line 234, in run_wrapper
    return fn()
  File "/Users/mark/Code/mitmproxy/libmproxy/console.py", line 1605, in loop
    self.view.keypress(size, k)
  File "/usr/local/Cellar/python/2.7.1/lib/python2.7/site-packages/urwid/container.py", line 614, in keypress
    return self.body.keypress( (maxcol, remaining), key )
  File "/Users/mark/Code/mitmproxy/libmproxy/console.py", line 454, in keypress
    self.view_response()
  File "/Users/mark/Code/mitmproxy/libmproxy/console.py", line 343, in view_response
    self.state.view_body_mode
  File "/Users/mark/Code/mitmproxy/libmproxy/console.py", line 323, in _conn_text
    viewmode
  File "/Users/mark/Code/mitmproxy/libmproxy/utils.py", line 503, in wrap
    ret = f(self, *args)
  File "/Users/mark/Code/mitmproxy/libmproxy/console.py", line 1030, in _cached_conn_text
    txt.extend(self._find_pretty_view(content, hdrItems))
  File "/Users/mark/Code/mitmproxy/libmproxy/console.py", line 1001, in _find_pretty_view
    return self._view_conn_json(lines)
TypeError: _view_conn_json() takes exactly 3 arguments (2 given)

[Enhancement] stream response to client

hi !

Sounds not convenient to implement but it would be nice to forward server response to client on the fly to prevent streaming issues (ex: on youtube, have to download full video to start playing).

Cheers

mitmproxy crashes when client connects

Hi,

When I run mitmproxy everything seems fine, until a client connects and requests data. Then it Fails with this:

Traceback (most recent call last):
  File "/Users/nilsriedemann/bin/mitmproxy", line 68, in <module>
    m.run()
  File "/Users/nilsriedemann/bin/src/mitmproxy-0.6/libmproxy/console.py", line 1249, in run
    self.ui.run_wrapper(self.loop)
  File "/Library/Python/2.7/site-packages/urwid-1.0.1-py2.7-macosx-10.7-intel.egg/urwid/raw_display.py", line 237, in run_wrapper
    return fn()
  File "/Users/nilsriedemann/bin/src/mitmproxy-0.6/libmproxy/console.py", line 1568, in loop
    changed = self.tick(self.masterq)
  File "/Users/nilsriedemann/bin/src/mitmproxy-0.6/libmproxy/flow.py", line 1204, in tick
    return controller.Master.tick(self, q)
  File "/Users/nilsriedemann/bin/src/mitmproxy-0.6/libmproxy/controller.py", line 77, in tick
    self.handle(msg)
  File "/Users/nilsriedemann/bin/src/mitmproxy-0.6/libmproxy/controller.py", line 95, in handle
    m(msg)
  File "/Users/nilsriedemann/bin/src/mitmproxy-0.6/libmproxy/flow.py", line 1268, in handle_clientconnect
    self.add_event("Connect from: %s:%s"%cc.address)
  File "/Users/nilsriedemann/bin/src/mitmproxy-0.6/libmproxy/console.py", line 1794, in add_event
    self.eventlist.set_focus(len(self.eventlist))
  File "/Library/Python/2.7/site-packages/urwid-1.0.1-py2.7-macosx-10.7-intel.egg/urwid/listbox.py", line 144, in set_focus
    raise IndexError
IndexError
^C[1]    22725 terminated  mitmproxy

transparent proxy redirect to https

when i setup mitmproxy for transparent proxy using iptables(NAT redirect to port), mitmproxy always catch request with https instead http.

e.g

GET http://news.ycombinator.com/ HTTP/1.1

in mitmproxy

GET https://news.ycombinator.com/ HTTP/1.1

Full-width bars are less than full-width under tmux/screen

This wasn't an issue under mitmproxy 0.8 and urwid 1.0.2, but the issue occurred after upgrading mitmproxy to 0.8 as well as in the current master (09c29e8). Also, urwid 1.1.1 shows the same.

Cannot “save request/response body” to file

When I’m browsing a connection, I want to save the request and response body to a file (with the b key).

However, when I press b, I get the "Save body request:" prompt line, but after entering a filename (eg. foo.txt), I get a Python traceback that is pretty hard to copy/paste since it's merged with the current lines on the terminal.

The last lines of the traceback are these:

path = os.path.expanduser(path)
   NameError: global name 'os' is not defined

However, I am able to save a "flow" (with the W key) to foo.txt just fine.

Dictionary comprehensions are not supported in Python <2.7

The following line causes a syntax error for pythons older than 2.7:

https://github.com/cortesi/mitmproxy/blob/master/libmproxy/authentication.py#L104

Traceback (most recent call last):
  File "./mitmproxy", line 19, in <module>
    from libmproxy import proxy, console, cmdline, version
  File "/home/geo/0/mitmproxy/libmproxy/proxy.py", line 21, in <module>
    import authentication
  File "/home/geo/0/mitmproxy/libmproxy/authentication.py", line 104
    self.usernames = {username:token for username,token in valid_entries}
                                       ^
SyntaxError: invalid syntax

mitmproxy alters headers in transit

Strictly speaking, HTTP headers are supposed to be case-insensitive. So theoretically it shouldn't matter that you convert all the headers in the HTTP response to lower case. But crappy HTTP clients exist, and one of them is the ActiveSync client on the Nokia N900. Using mitmproxy breaks it.

Crash when the PAGER is `less -R`

> echo $PAGER
less -R

Then if I use < or > to navigate, I will get the following exception.

› mitmproxy -nr game_subway-bmx

Traceback (most recent call last):
  File "/usr/local/bin/mitmproxy", line 67, in <module>
    m.run()
  File "/Library/Python/2.7/site-packages/libmproxy/console/__init__.py", line 583, in run
    self.ui.run_wrapper(self.loop)
  File "/Library/Python/2.7/site-packages/urwid/raw_display.py", line 242, in run_wrapper
    return fn()
  File "/Library/Python/2.7/site-packages/libmproxy/console/__init__.py", line 812, in loop
    k = self.view.keypress(size, k)
  File "/Library/Python/2.7/site-packages/urwid/container.py", line 1103, in keypress
    return self.body.keypress( (maxcol, remaining), key )
  File "/Library/Python/2.7/site-packages/libmproxy/console/flowview.py", line 546, in keypress
    self.master.spawn_external_viewer(conn.content, t)
  File "/Library/Python/2.7/site-packages/libmproxy/console/__init__.py", line 546, in spawn_external_viewer
    subprocess.call(cmd, shell=shell)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 493, in call
    return Popen(*popenargs, **kwargs).wait()
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 679, in __init__
    errread, errwrite)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 1228, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory

AttributeError when trying to launch an editor with the "v" shortcut

This occurs on the master branch when I attempt to edit a flow's request's content with the "v" shortcut.

Traceback (most recent call last):
  File "/home/bryan/.virtualenvs/envname/bin/mitmproxy", line 7, in <module>
    execfile(__file__)
  File "/home/bryan/.virtualenvs/envname/src/mitmproxy/mitmproxy", line 67, in <module>
    m.run()
  File "/home/bryan/.virtualenvs/envname/src/mitmproxy/libmproxy/console/__init__.py", line 583, in run
    self.ui.run_wrapper(self.loop)
  File "/home/bryan/.virtualenvs/envname/lib/python2.7/site-packages/urwid/raw_display.py", line 242, in run_wrapper
    return fn()
  File "/home/bryan/.virtualenvs/envname/src/mitmproxy/libmproxy/console/__init__.py", line 812, in loop
    k = self.view.keypress(size, k)
  File "/home/bryan/.virtualenvs/envname/lib/python2.7/site-packages/urwid/container.py", line 1103, in keypress
    return self.body.keypress( (maxcol, remaining), key )
  File "/home/bryan/.virtualenvs/envname/src/mitmproxy/libmproxy/console/flowview.py", line 545, in keypress
    self.master.spawn_external_viewer(conn.content, t)
  File "/home/bryan/.virtualenvs/envname/src/mitmproxy/libmproxy/console/__init__.py", line 546, in spawn_external_viewer
    subprocess.call(cmd, shell=shell)
  File "/usr/lib/python2.7/subprocess.py", line 493, in call
    return Popen(*popenargs, **kwargs).wait()
  File "/usr/lib/python2.7/subprocess.py", line 679, in __init__
    errread, errwrite)
  File "/usr/lib/python2.7/subprocess.py", line 1249, in _execute_child
    raise child_exception
AttributeError: 'NoneType' object has no attribute 'rfind'

Installing on Mac OS X 10.7.3

Hi,

i downloaded the source and installed mitmproxy with the following command:
sudo python setup.py install

If i try to start mitmproxy with:
./mitmproxy -p 8899

I get the following errors:

Traceback (most recent call last):
File "./mitmproxy", line 19, in
from libmproxy import proxy, console, cmdline
File "/Users/carl.jahn/Downloads/cortesi-mitmproxy-bc3ba4c/libmproxy/proxy.py", line 24, in
import utils, flow, certutils
File "/Users/carl.jahn/Downloads/cortesi-mitmproxy-bc3ba4c/libmproxy/certutils.py", line 2, in
from pyasn1.type import univ, constraint, char, namedtype, tag
ImportError: No module named pyasn1.type

How can i fix the errors?

Add option to skip processing some responses

Mitmproxy processes all requests and parses all responses, sending them to the client when the response is complete. For big responses (e.g. streaming movies) this introduces unnecessary delays and/or timeouts.

I'd like to be able (in a script request method) to mark some requests so as the responses to them will be forwarded straight to client without parsing. They might even be not stored in the dump file.

Mac OSX install problems

Hey there!

I tried to install mitmproxy to test it with my firefoy and my ios device.
But after my installation of python and urwid (via macports) the gui of mitmproxy is still blank.

I googled and tried the following instructions: http://r00tsec.blogspot.com/2011/04/howto-installation-mitmproxy.html
But without success...

Do I have to initiate the SSL connection in a special way?

Thanks for help!

support for upstream proxies

wrapping in proxychains works however DNS is still done locally instead of the proxychained, mitmproxy.

so would love to have the following happen:

HTTP request for www.example.com -> mitmproxy forwards request to upstream HTTP proxy -> HTTP Proxy performs DNS and request, response sent to mitmproxy -> mitmproxy adds a few bytes-> modified response provided

mitmproxy-0.8.tar.gz gzipped twice

Hi,

I noticed that the current stable download (at http://mitmproxy.org/download/mitmproxy-0.8.tar.gz) is gzipped twice:

$ tar -xzf mitmproxy-0.8.tar.gz 
tar: This does not look like a tar archive
tar: Skipping to next header
tar: Exiting with failure status due to previous errors
$ file mitmproxy-0.8.tar.gz
mitmproxy-0.8.tar.gz: gzip compressed data, from Unix
$ gunzip mitmproxy-0.8.tar.gz
$ file mitmproxy-0.8.tar
mitmproxy-0.8.tar: gzip compressed data, was "dist/mitmproxy-0.8.tar", last modified: Mon Apr  9 06:44:19 2012, max compression
$ tar -xzf mitmproxy-0.8.tar
$

AssertionError in m.run() (looks to be urwid related)

So i'm not sure why this is happening, I've downloaded and compiled the newest version of urwid, and pulled from you git repo, for some reason this keeps freaking out whenever I go to get more detail about a connection.

Ubuntu 10.04LTS x86_64 running python 2.6

Version 0.6 seems to work fine, although I was getting a traceback with something last night I don't remember...

Traceback (most recent call last):
File "./mitmproxy", line 61, in
m.run()
File "/tmp/mitmproxy/libmproxy/console/init.py", line 541, in run
self.ui.run_wrapper(self.loop)
File "/usr/lib/pymodules/python2.6/urwid/raw_display.py", line 234, in run_wrapper
return fn()
File "/tmp/mitmproxy/libmproxy/console/init.py", line 733, in loop
size = self.drawscreen()
File "/tmp/mitmproxy/libmproxy/console/init.py", line 717, in drawscreen
self.ui.draw_screen(size, canvas)
File "/usr/lib/pymodules/python2.6/urwid/raw_display.py", line 601, in draw_screen
for row in r.content():
File "/usr/lib/pymodules/python2.6/urwid/canvas.py", line 555, in content
yield shard_body_row(sbody)
File "/usr/lib/pymodules/python2.6/urwid/canvas.py", line 787, in shard_body_row
row.extend(content_iter.next())
File "/usr/lib/pymodules/python2.6/urwid/canvas.py", line 378, in content
assert trim_left >= 0 and trim_left < maxcol
AssertionError

Host name in URL not shown for https requests

The current release displays the URL just fine:

https://encrypted.google.com/search?q=foo

The current development version, however, only displays the IP:

https://173.194.70.101/search?q=foo

HTTP requests are not affected. The issue appears first in 9130cd6.

Crash when saving edited flow

I was trying to use mitmproxy (version 0.8.1 installed with pip on Mac OS 10.7.4) in server-replay mode to fake a response, which I thought would be easier than hacking the server for the test I was running. Turns out it wasn't.

I edited the response body and saved the flow with no problem, except that in doing so I changed the length of the response. Unfortunately mitmproxy did not change the Content-Length: header to match, so I had to edit that, too. Then when I tried to save the flow I got this error:

Traceback (most recent call last):
File "/usr/local/bin/mitmproxy", line 64, in
m.run()
File "/Library/Python/2.7/site-packages/libmproxy/console/init.py", line 538, in run
self.ui.run_wrapper(self.loop)
File "/Library/Python/2.7/site-packages/urwid/raw_display.py", line 237, in run_wrapper
return fn()
File "/Library/Python/2.7/site-packages/libmproxy/console/init.py", line 762, in loop
self.prompt_execute()
File "/Library/Python/2.7/site-packages/libmproxy/console/init.py", line 689, in prompt_execute
msg = p(txt, *args)
File "/Library/Python/2.7/site-packages/libmproxy/console/init.py", line 619, in save_one_flow
return self._write_flows(path, [flow])
File "/Library/Python/2.7/site-packages/libmproxy/console/init.py", line 613, in _write_flows
fw.add(i)
File "/Library/Python/2.7/site-packages/libmproxy/flow.py", line 1557, in add
tnetstring.dump(d, self.fo)
File "/Library/Python/2.7/site-packages/libmproxy/tnetstring.py", line 103, in dump
file.write(dumps(value,encoding))
File "/Library/Python/2.7/site-packages/libmproxy/tnetstring.py", line 93, in dumps
_rdumpq(q,0,value,encoding)
File "/Library/Python/2.7/site-packages/libmproxy/tnetstring.py", line 174, in _rdumpq
size = _rdumpq(q,size,v,encoding)
File "/Library/Python/2.7/site-packages/libmproxy/tnetstring.py", line 174, in _rdumpq
size = _rdumpq(q,size,v,encoding)
File "/Library/Python/2.7/site-packages/libmproxy/tnetstring.py", line 165, in _rdumpq
size = _rdumpq(q,size,item,encoding)
File "/Library/Python/2.7/site-packages/libmproxy/tnetstring.py", line 165, in _rdumpq
size = _rdumpq(q,size,item,encoding)
File "/Library/Python/2.7/site-packages/libmproxy/tnetstring.py", line 182, in _rdumpq
raise ValueError("must specify encoding to dump unicode strings")
ValueError: must specify encoding to dump unicode strings

At which point there was no console display but mitmproxy was still running. The only way I found to kill it was to switch to another terminal window and kill the process.

I was unsuccessful at trying to edit the saved flow with vi or emacs, either: after editing, mitmproxy would not load them. When I tried to load them via the UI, the UI showed "Invalid data format." When I tried to load the flow with mitmproxy -S flowfile it crashed:

Traceback (most recent call last):
File "/usr/local/bin/mitmproxy", line 63, in
m = console.ConsoleMaster(server, opts)
File "/Library/Python/2.7/site-packages/libmproxy/console/init.py", line 392, in init
self.server_playback_path(options.server_replay)
File "/Library/Python/2.7/site-packages/libmproxy/console/init.py", line 451, in server_playback_path
self.statusbar.message(ret)
AttributeError: ConsoleMaster instance has no attribute 'statusbar'

Pretty-printing is not work?

How can I make html or js to look better with mitmproxy? js-beautifier has built in mitmproxy, why is not work?

Problem with upstream-cert

Hi, I'm runnin mitmproxy with --upstream-cert, and when I enter into https://www.google.com i'm getting this error:

---

Exception happened during processing of request from ('192.168.XX.XX', 56121)
Traceback (most recent call last):
File "/usr/lib/python2.6/SocketServer.py", line 560, in process_request_thread
self.finish_request(request, client_address)
File "/usr/local/lib/python2.6/dist-packages/libmproxy/proxy.py", line 506, in finish_request
self.RequestHandlerClass(self.config, request, client_address, self, self.masterq)
File "/usr/local/lib/python2.6/dist-packages/libmproxy/proxy.py", line 296, in init
SocketServer.StreamRequestHandler.init(self, request, client_address, server)
File "/usr/lib/python2.6/SocketServer.py", line 617, in init
self.handle()
File "/usr/local/lib/python2.6/dist-packages/libmproxy/proxy.py", line 302, in handle
self.handle_request(cc)
File "/usr/local/lib/python2.6/dist-packages/libmproxy/proxy.py", line 311, in handle_request
request = self.read_request(cc)
File "/usr/local/lib/python2.6/dist-packages/libmproxy/proxy.py", line 390, in read_request
certfile = self.find_cert(host, port),
File "/usr/local/lib/python2.6/dist-packages/libmproxy/proxy.py", line 361, in find_cert
sans = cert.altnames
File "/usr/local/lib/python2.6/dist-packages/libmproxy/certutils.py", line 211, in altnames
altnames.append(i[0].asOctets())

AttributeError: IA5String instance has no attribute 'asOctets'

Thanks

Editing dumped file then client-replay

I am using mitmdump to dump some network activity. Then I use textedit on osx to edit then (literally changing one line), but when I try client replay it gives Could not load file: Invalid data format. How do I fix this???? I also tried simple replay w/o editing it and it gives the issue

mitmproxy ios certificate

Cerifitcate expires in 1902 and is not working.
http://imageshack.us/m/405/2772/img0974.png

Download cert here:
http://www.mediafire.com/?gcosyenvi03m733