How do I send a HTTP POST value to a (PHP) page using Python?

When testing, or automating websites using python, I enjoy using twill. Twill is a tool that automatically handles cookies and can read HTML forms and submit them.

For instance, if you had a form on a webpage you could conceivably use the following code:

from twill import commands
commands.go('http://example.com/create_product')
commands.formvalue('formname', 'product', 'new value')
commands.submit()

This would load the form, fill in the value, and submit it.

I find http://www.voidspace.org.uk/python/articles/urllib2.shtml to be a good source of information about urllib2, which is probably the best tool for the job.

import urllib
import urllib2

url = 'http://www.someserver.com/cgi-bin/register.cgi'
user_agent = 'Mozilla/4.0 (compatible; MSIE 5.5; Windows NT)'
values = {'name' : 'Michael Foord',
          'location' : 'Northampton',
          'language' : 'Python' }
headers = { 'User-Agent' : user_agent }

data = urllib.urlencode(values)
req = urllib2.Request(url, data, headers)
response = urllib2.urlopen(req)
the_page = response.read()

The encoding is actually done using urllib, and this supports HTTP POST. There is also a way to use GET, where you have to pass the data into urlencode.

Don't forget to call read() though, otherwise the request won't be completed.

Yes. urllib2 is a nice Python way to form/send HTTP requests.

Just an addendum to @antileet's procedure, it works similarly if you're trying to do a HTTP POST request with a web-service-like payload, except you just omit the urlencode step; i.e.

import urllib, urllib2

payload = """
<?xml version='1.0'?>
<web_service_request>
    <short_order>Spam</short_order>
    <short_order>Eggs</short_order>
</web_service_request>
""".strip()
query_string_values = {'test': 1}
uri = 'http://example.com'

# You can still encode values in the query string, even though
# it's a POST request. Nice to have this when your payload is
# a chunk of text.
if query_string_values:
    uri = ''.join([uri, '/?', urllib.urlencode(query_string_values)])
req = urllib2.Request(uri, data=payload)
assert req.get_method() == 'POST'
response = urllib2.urlopen(req)
print 'Response:', response.read()