Python Programming/Tips and Tricks

There are many tips and tricks you can learn in Python:

Strings

• Triple quotes are an easy way to define a string with both single and double quotes.
• String concatenation is expensive. Use percent formatting and str.join() for concatenation:

(but don't worry about this unless your resulting string is more than 500-1000 characters long)

print "Spam" + " eggs" + " and" + " spam"               # DON'T DO THIS
print " ".join(["Spam","eggs","and","spam"])            # Much faster/more
                                                        # common Python idiom
print "%s %s %s %s" % ("Spam", "eggs", "and", "spam")   # Also a pythonic way of
                                                        # doing it - very fast

Module choice

• cPickle is a faster, C written module for pickle. cPickle is used to serialize python program. Other modules have C implementations as well, cStringIO for the StringIO module, and cProfile for the profile module.

import cPickle # You may want to import it as P for convenience.

• These can even fall back to the slower pure-python version if they fail to import.

try:
  import cPickle as pickle
except ImportError:
  import pickle

List comprehension and generators

• List comprehension and generator expressions are very useful for working with small, compact loops. Additionally, it is faster than a normal for-loop.

directory = os.listdir(os.getcwd())       # Gets a list of files in the
                                          # directory the program runs from
filesInDir = [item for item in directory] # Normal For Loop rules apply, you
                                          # can add "if condition" to make a
                                          # more narrow search.

• List comprehension and generator expression can be used to work with two (or more) lists with zip or itertools.izip

[a - b for (a,b) in zip((1,2,3), (1,2,3))]  # will return [0, 0, 0]

Data type choice

Choosing the correct data type can be critical to the performance of an application. For example, say you have 2 lists:

list1 = [{'a': 1, 'b': 2}, {'c': 3, 'd': 4}, {'e': 5, 'f': 6}]
list2 = [{'e': 5, 'f': 6}, {'g': 7, 'h': 8}, {'i': 9, 'j': 10}]

and you want to find the entries common to both lists. You could iterate over one list, checking for common items in the other:

common = []
for entry in list1:
    if entry in list2:
        common.append(entry)

For such small lists, this will work fine, but for larger lists, for example if each contains thousands of entries, the following will be more efficient, and produces the same result:

set1 = set([tuple(entry.items()) for entry in list1])
set2 = set([tuple(entry.items()) for entry in list2])
common = set1.intersection(set2)
common = [dict(entry) for entry in common]

Sets are optimized for speed in such functions. Dictionaries themselves cannot be used as members of a set as they are mutable, but tuples can. If one needs to do set operations on a list of dictionaries, one can convert the items to tuples and the list to a set, perform the operation, then convert back. This is often much faster than trying to replicate set operations using string functions.

Other

• Decorators can be used for handling common concerns like logging, db access, etc.
• While Python has no built-in function to flatten a list you can use a recursive function to do the job quickly.

def flatten(seq, list = None):
    """flatten(seq, list = None) -> list
 
    Return a flat version of the iterator `seq` appended to `list`
    """
    if list == None:
        list = []
    try:                          # Can `seq` be iterated over?
        for item in seq:          # If so then iterate over `seq`
            flatten(item, list)      # and make the same check on each item.
    except TypeError:             # If seq isn't iterable
        list.append(seq)             # append it to the new list.
    return list

• To stop a Python script from closing right after you launch one independently, add this code:

print 'Hit Enter to exit'
raw_input()

• Python already has a GUI interface built in: Tkinter, based on Tcl's Tk. More are available, such as PyQt4 and pygtk3.

• Ternary Operators:

[on_true] if [expression] else [on_false]
 
x, y = 50, 25
 
small = x if x < y else y

• Booleans as indexes:

b = 1==1
name = "I am %s" % ["John","Doe"][b]
#returns I am Doe