Count reoccurring elements in multiple lists in Python

Question

I have a number of names divided into several lists. I am printing all the names which occur in more than one list, sorted by the number of occurrences.

What is a better/easier/more pythonic way of doing this? I feel like my solution is overly complicated.

import numpy as np


list_1 = ['John Cleese', 'Terry Gilliam']
list_2 = ['Eric Idle', 'Terry Jones', 'Michael Palin']
list_3 = ['Graham Chapman', 'Sir Lancelot the Brave', 'Terry Jones']
list_4 = ['Arthur, King of the Britons', 'Terry Jones', 'John Cleese']
list_5 = ['Michael Palin', 'Sir Robin the Not-Quite-So-Brave-as-Sir-Lancelot']

all = sorted(np.unique(list_1+list_2+list_3+list_4+list_5))


def in_how_many(name):
    result = 0
    if name in list_1:
        result += 1
    if name in list_2:
        result += 1
    if name in list_3:
        result += 1
    if name in list_4:
        result += 1
    if name in list_5:
        result += 1

    return result


names_list = []

for name in all:
    if in_how_many(name) > 1:
        name_dict = {'name': name, 'value': in_how_many(name)}
        names_list.append(name_dict)

for person in sorted(names_list, key=lambda k: k['value'], reverse=True):
    print '\'%s\' is in %s lists' % (person['name'], person['value'])

This prints:

'Terry Jones' is in 3 lists
'John Cleese' is in 2 lists
'Michael Palin' is in 2 lists

Answer 1

What you want is collections.Counter, and the Counter.most_common method.

A naive implementation could be:

import collections

lists = [list_1, list_2, list_3, list_4, list_5]
counter = collection.Counter(sum(lists))

for name, amount in counter.most_common():
    print('\'%s\' is in %s lists' % (name, amount))

This however will mean if list_1 is ['a'] * 5 it will say 'a' is in five lists when this isn't true. A dictionary comprehension is all that's needed for this building a dictionary from the lists, setting the value to 1. And you can take advantage of the fact collection.Counter has addition defined.

counter = sum(
    collection.Counter({name: 1 for name in names})
    for names in lists
)

---

Without using collections.Counter I would advise you use set. Firstly it removes the need for using np.unique and emphasis the fact that sorted is redundant, as sets are unordered.

It also means that making all is simple and reduces name in list_1 from an \$O(n)\$ operation to an \$O(1)\$ operation. Leading to faster code.

As shown earlier it's easier to work with lists, rather than a list of variable names and so lists is defined the same as above.

sets = [set(names) for names in lists]
all = sum(sets)


def in_how_many(value, sets):
    return sum(
        value in set_
        for set_ in sets
    )

---

No matter which of the above two solutions you use you still have a couple of other problems:

• Don't put code in global scope, have a main function. This makes it harder to do bad things or mess up.
• You should use if __name__ == '__main__' to only allow the code in main to run if it's the 'main' file.
• In relation to the above two, notice that I pass sets to in_how_many. This is because you shouldn't rely on global scope as it makes reusing the same code harder to do.
• Modulo formatting was deprecated for a while, because it's generally worse than str.format and is more susceptible to errors. I suggest using str.format or upgrading to Python 3.7 to take advantage of f-strings.
• Import print_function form __future__ to make print a function. This makes it easier to upgrade to Python 3.

from __future__ import print_function
import collections
import functools
import operator


def main(lists):
    counter = functools.reduce(
        operator.add,
        (
            collections.Counter(set(names))
            for names in lists
        )
    )

    for name, amount in counter.most_common():
        print('{name!r} is in {amount} lists'.format(name=name, amount=amount))
        # print(f'{name!r} is in {amount} lists')  # Python 3.7 f-string.


if __name__ == '__main__':
    main(...)

Answer 2

This exercise can be solved using the tools in the Python standard library, namely the collections module and the itertools module.

• First, you want to ensure that just because a name appears twice in the same list, it is not double-counted. Use map and set for that (or a comprehension).
• Then you want to parse all names from all lists, use itertools.chain.from_iterable for that.
• Finally you need to count how often each name appeared, which you can use collections.Counter for (just like in the other answer by @Peilonrayz).

from collections import Counter
from itertools import chain

lists = [list_1, list_2, list_3, list_4, list_5]

no_of_lists_per_name = Counter(chain.from_iterable(map(set, lists)))

for name, no_of_lists in no_of_lists_per_name.most_common():
    if no_of_lists == 1:
        break # since it is ordered by count, once we get this low we are done
    print(f"'{name}' is in {no_of_lists} lists")
# 'Terry Jones' is in 3 lists
# 'John Cleese' is in 2 lists
# 'Michael Palin' is in 2 lists

If you are learning Python now, don't learn Python 2, unless you really have to. It will become unsupported in less than a year. And Python 3(.6+) has nice f-strings, which make formatting a lot easier (I used them in the code above).

Note that you can mix string quotation marks. I.e. if you need single-quotes ' inside your string, use double-quotes "" to make the string (and vice-versa). This way you don't need to escape them.