Overflow in exp in scipy/numpy in Python?

In your case, it means that b is very small somewhere in your array, and you're getting a number (a/b or exp(log(a) - log(b))) that is too large for whatever dtype (float32, float64, etc) the array you're using to store the output is.

Numpy can be configured to

1. Ignore these sorts of errors,
2. Print the error, but not raise a warning to stop the execution (the default)
3. Log the error,
4. Raise a warning
5. Raise an error
6. Call a user-defined function

See numpy.seterr to control how it handles having under/overflows, etc in floating point arrays.

When you need to deal with exponential, you quickly go into under/over flow since the function grows so quickly. A typical case is statistics, where summing exponentials of various amplitude is quite common. Since the numbers are very big/smalls, one generally takes the log to stay in a "reasonable" range, the so-called log domain:

exp(-a) + exp(-b) -> log(exp(-a) + exp(-b))

Problems still arise because exp(-a) will still underflows up. For example, exp(-1000) is already below the smallest number you can represent as a double. So for example:

log(exp(-1000) + exp(-1000))

gives -inf (log (0 + 0)), even though you can expect something like -1000 by hand (-1000 + log(2)). The function logsumexp does it better, by extracting the max of the number set, and taking it out of the log:

log(exp(a) + exp(b)) = m + log(exp(a-m) + exp(b-m))

It does not avoid underflow totally (if a and b are vastly different for example), but it avoids most precision issues in the final result

Isn't exp(log(a) - log(b)) the same as exp(log(a/b)) which is the same as a/b?

>>> from math import exp, log
>>> exp(log(100) - log(10))
10.000000000000002
>>> exp(log(1000) - log(10))
99.999999999999957

2010-12-07: If this is so "some values in the array b are intentionally set to 0", then you are essentially dividing by 0. That sounds like a problem.