Pythonic way for double iteration (list all files with extensions)

Question

Given a base path and a list with extensions the task is to list all files:

Two of my solutions are:

from glob import glob
from os import path

EXTENSIONS = ['*.zip', '*.jar', '*.pdf']
DOC_PATH = '/path/to/files'

# Solution1:
files = []         
for ext in EXTENSIONS:
    files.extend(glob(path.join(DOC_PATH, ext)))
# works but looks very clumsy

# Solution2:
files = reduce(lambda x,y: x+y,
               [glob(path.join(DOC_PATH, ext)) for ext in EXTENSIONS])
# Also functional but looks like a misuse of reduce

Have you got any other ideas?

Not quite sure how idiomatic/efficient it is : sum((glob(path.join(DOC_PATH, ext)) for ext in EXTENSIONS), []) . — Josay, yesterday

otus · Accepted Answer · 2014-07-07 12:30:54Z

up vote 3 down vote accepted

If you only need to iterate over them (once) and not an actual list, you could use itertools.chain and glob.iglob:

files = chain(*(iglob(path.join(DOC_PATH, ext)) for ext in EXTENSIONS))

If you do need an actual list, you can further call list(files), of course.

answered yesterday

otus
67414

chain is quite a good idea – ProfHase85 yesterday

add comment

Gareth Rees · Answer 2 · 2014-07-07 13:07:41Z

Using os.listdir and os.path.splitext:

import os
EXTENSIONS = 'zip jar pdf'.split()
EXTENSION_SET = set('.' + e for e in EXTENSIONS)
files = [f for f in os.listdir(DOC_PATH) if os.path.splitext(f)[1] in EXTENSION_SET]

Using os.listdir and re.search:

import os
import re
EXTENSIONS = 'zip jar pdf'.split()
EXTENSION_RE = re.compile(r'\.({})$'.format('|'.join(EXTENSIONS))
files = [f for f in os.listdir(DOC_PATH) if EXTENSION_RE.search(f)]
# or files = list(filter(EXTENSION_RE.search, os.listdir(DOC_PATH)))

The advantage of these approaches is that you only iterate over the files in the directory once (rather than once for each extension, as in the glob case).

asked	yesterday
viewed	355 times
active	yesterday

current community

your communities

more stack exchange communities

Pythonic way for double iteration (list all files with extensions)

2 Answers

Your Answer

Not the answer you're looking for? Browse other questions tagged python or ask your own question.

Hot Network Questions

current community

your communities

more stack exchange communities

Pythonic way for double iteration (list all files with extensions)

2 Answers

Your Answer

Sign up or log in

Post as a guest

Not the answer you're looking for? Browse other questions tagged python or ask your own question.

Related

Hot Network Questions