current community

your communities

more stack exchange communities

company blog
Stack Exchange Inbox Reputation and Badges
tour help
Code Review Stack Exchange is a question and answer site for peer programmer code reviews. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top
up vote 3 down vote favorite
1

I'm currently using SciPy's mode function to find the most occurring item in different iterable objects. I like the mode function because it works on every object type I've thrown at it (strings, floats, ints).

While this function seems reliable, it is very slow on bigger lists:

from scipy.stats import mode

li = range(50000)
li[1] = 0

%timeit mode(li)
1 loops, best of 3: 7.55 s per loop

Is there a better way to get the mode for a list? If so, would the implementation be different depending on the item type?

share|improve this question
up vote 4 down vote accepted

Originally I thought you must have been doing something wrong, but no: the scipy.stats implementation of mode scales with the number of unique elements, and so will behave very badly for your test case.

As long as your objects are hashable (which includes your three listed object types of strings, floats, and ints), then probably the simplest approach is to use collections.Counter and the most_common method:

In [33]: import scipy.stats

In [34]: li = range(50000); li[1] = 0

In [35]: scipy.stats.mode(li)
Out[35]: (array([ 0.]), array([ 2.]))

In [36]: timeit scipy.stats.mode(li)
1 loops, best of 3: 10.7 s per loop

but

In [37]: from collections import Counter

In [38]: Counter(li).most_common(1)
Out[38]: [(0, 2)]

In [39]: timeit Counter(li).most_common(1)
10 loops, best of 3: 34.1 ms per loop
share|improve this answer
    
Wow! What an improvement! – ChrisArmstrong Dec 29 '12 at 22:00

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.