Tell me more ×
Stack Overflow is a question and answer site for professional and enthusiast programmers. It's 100% free, no registration required.
up vote 2 down vote favorite

I am trying to record HTTP GET/POST requests sent by my browser using the library scotch.

I am using their sample code: http://darcs.idyll.org/~t/projects/scotch/doc/recipes.html#id2

 import scotch.proxy
 app = scotch.proxy.ProxyApp()

 import scotch.recorder
 recorder = scotch.recorder.Recorder(app, verbosity=1)

 try:

     from wsgiref.simple_server import WSGIServer, WSGIRequestHandler

     server_address = ('', 8000)
     httpd = WSGIServer(server_address, WSGIRequestHandler)

     httpd.set_app(app)

     while 1:
        httpd.handle_request()

 finally:

    from cPickle import dump
    outfp = open('recording.pickle', 'w')
    dump(recorder.record_holder, outfp)
    outfp.close()

    print 'saved %d records' % (len(recorder.record_holder))

So I ran above code, went over to google chrome, and visited a few sites to see if that would get recorded.

However, I do not see how the code should terminate. It seems that there has to be an error in httpd.handle_request() for the code to terminate.

I tried a variation of the code where I removed the try and finally syntax, and changed the while condition so that the loop ran for 30 seconds. However, that seems to be running forever as well.

Any ideas on how to get this working? I am also open to using other python libraries available for what I am trying to do: record my browser's GET/POST requests, including logons, and replay this within python.

Thanks.

share|improve this question
Did you change the proxy settings in the browser? Do you see that this code is running? – Alex Kreimer yesterday
Hey, I did not change any proxy settings on the browser (I guess because there were no instructions on this). I guess my knowledge is lacking in regards to how this library works. And yes, the code was running on pyscripter according to the status bar. Any help would be appeciated, or a point in the right direction where I can read about proxies and how this all works. – jacksnyder yesterday

1 Answer

up vote 0 down vote accepted

Correct me if I'm wrong, but you're trying to log the activity of your local browser by setting a local proxy. If this is the case your browser needs to go through your proxy in order for your proxy server to log the activity.

The code that you've provided sets a proxy server at localhost:8000, so you need to tell your browser about this. The actual setting depends on the browser, I'm sure you'd be able to google it easily.

When I've asked to check if the code is running I actually mean whether your local proxy accepts some kind of request from the browser. Do you see the 'saved records' print out of your code at some point?

share|improve this answer
Hey Alex, yes you are right about what I am trying to do. I changed firefox's proxy to 8000 (tools -> options -> advanced -> network tab -> settings) ~ screenshot. It seems the code is getting stuck on this line: httpd.handle_request(). I guess because it isn't seeing any activity on port 8000? I refactored the code to make it simpler to debug: pastebin.com/frSu0edL (this is what I am running atm and it runs forever). – jacksnyder 6 hours ago
Hey, it works! I added the localhost address 127.0.0.1 for the HTTP Proxy address and I get some of my activity recorded. I'm sure with fiddling around I can figure out the rest. Thanks a lot bro! I would upvote but not enough karma atm. Will upvote when I am able. – jacksnyder 6 hours ago

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.