How to find files with same name but different line count in two directories?

Question

I have a directory sub1 with the following files:

$ wc -l *

5 file1.csv
5 file3.csv
1 file4.csv

In sub2, I have the following:

$ wc -l *
5 file1.csv
5 file2.csv
1 file3.csv
5 file4.csv
1 file5.csv

In the first directory, I might have files with added lines, which then go to the second dir. In this example, I might need to update file3 in sub2.

How do I get a list of the files with differences?

---

I did some tests with diff and grep, but it doesn't work because the directories have different files (and hence the lines are different):

~/dir1/$ wc -l >> wc.luis

~/dir1/$ wc -l * | awk '{ gsub(/\/home.*dir1\//,""); print $0 }' 
                 | diff --side-by-side wc.luis -
                 | grep \|

Ideally, I would get a list like this:

5 file3.csv | 1 file3.csv
1 file4.csv | 5 file4.csv

Any help is appreciated!

---

Notes:

• I cannot check on the date, because all files were updated, with or without changes.

• Sometimes the newest files lack some lines, for which reason I cannot just take the bigger one.

Answer 1

Here is a quick and dirty shell "one-liner" with example output:

$ join -j2 <(cd sub1; wc -l *) <(cd sub2; wc -l *) | awk '$2!=$3'
file3.csv 5 1
file4.csv 1 5
total 11 17

The total line is an artifact from the output of wc. It can be removed with another filter:

$ join -j2 <(cd sub1; wc -l *) <(cd sub2; wc -l *) | awk '$2!=$3' | head -n-1
file3.csv 5 1
file4.csv 1 5

Explanation:

join will join two files based on a common column. In this case we join based on the second column (-j2). In the output of wc the second column is the filename. This will only print files which are common in both directories.

The wc invocations are done in process substitutions with working directory changed to sub1 or sub2 so the filenames are printed without directory name. This is so that join can find the common files.

The awk command compares the value in the second and third column and only prints the line if the values differ. This will filter out files with the same line count.

head -n-1 will print all lines but not the last line. This will filter out the last total line from wc.