Constants

They are better defined outside of functions; there is no need in reinitializing them each call.

Also they are not balanced, fixing it will avoid a weird body = '<tr>\n' + ....

Files management

Just in case the path of the input file contained a dot, I recommend you use rsplit('.', 1).

Also you should try to open the input file before the ouput one. Just in case an error happen during the access to the file, you won't create an empty output file.

And since you're dealing with CSV files, why not use the csv module? It will allow you to handle edge-case data (such as fields containing comas or newlines) easily.

Wrap stuff into tags

Since you're outputing HTML, it may be a better idea to build it as such: by wrapping stuff into tags.

You could define a PATTERN = "<{tag}>{{}}</{tag}>" and a formatter:

def formatter(tag):
    return PATTERN.format(tag=tag).format

map that formatter at will to wrap stuff into tags.

Proposed improvements

import webbrowser
import sys
import csv

PATTERN = "<{tag}>{{}}</{tag}>"

def formatter(tag):
    return PATTERN.format(tag=tag).format

def csv_to_html(csv_file):
    reader = csv.reader(csv_file)

    header = formatter('tr')(
        '\n'.join(map(formatter('th'), next(reader)))
    )

    data = '\n'.join(
        map(formatter('tr'),
            '\n'.join(map(formatter('td'), row))
            for row in reader
        )
    )

    return formatter('table')('\n'.join((header, data)))

if __name__ == "__main__":
    csv_filename = sys.argv[1]
    html_filename = csv_filename.rsplit('.', 1)[0] + '.html'
    with open(csv_filename) as in_file:
        with open(html_filename, "w+") as out_file:
            out_file.write(csv_to_html(in_file))
    webbrowser.open(html_filename)

You can't just go about doing string substitutions like that. Every serialization format has some kind of escaping mechanism. Examples:

• SQL strings are delimited by '. To include a literal apostrophe, you use ''. If you interpolate strings into your SQL queries without escaping, then you end up with a Bobby Tables situation (a.k.a sql-injection).
• HTML has special characters <, >, ", and &, which should be written as <, > ", and &, or numerically like ., or in hexadecimal like &. If you neglect to perform HTML escaping, then you might end up with arbitrary HTML, including possibly a <script> tag injected into the document.
• CSV is only loosely specified, but RFC 4180 is a rough summary of the de facto practices. If there are literal commas in your data, then the field can be "quoted, like this".

These are just a few examples, but if you think about it, every data format has such special characters (YAML, HTTP URLs, LDAP search filters, Javascript string literals, C printf() format strings, …). Ignore that fact, and you have a security hole.

To parse CSV, use the csv module. To escape HTML, use the cgi.escape() function.