current community

your communities

more stack exchange communities

Stack Exchange
tour help
Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems. It's 100% free, no registration required.
up vote 4 down vote favorite

I am trying to insert line breaks on a file based on the same names which happens to be the second field in my file. So, my input file is like below. 

17412193|name1|organization
43979400|name1|organization
1405541|name2|organization
53595498|name2|organization
50439202|name2|organization
54678379|name3|Not Found
21757330|name3|organization

So I am trying to get the output like,

17412193|name1|organization
43979400|name1|organization
###linebreak inserted here
1405541|name2|organization
53595498|name2|organization
50439202|name2|organization
###linebreak inserted here
54678379|name3|Not Found
21757330|name3|organization
###linebreak inserted here

I am using this line break to split the input file to group the similar names together. I would later apply a similarity algorithm on these grouped names. So, for the above example, after the output I would apply the similarity algorithm on 3 pairs of names.

share|improve this question

4 Answers 4

up vote 8 down vote accepted

Assuming your file is sorted/grouped by the 2nd field already

awk -F '|' 'NR>1 && $2 != prev {print ""} {prev=$2; print}' file
share|improve this answer
    
Thank you. This is what I needed. :) –  Ramesh Mar 26 '14 at 15:48
    
It would be helpful to us, if you explain your command. –  Avinash Raj Mar 29 '14 at 6:57
up vote 4 down vote

Here's a case insensitive Perl solution:

perl -F'\|' -lape 'unless($F[1]=~/^$pre$/i || $.==1){print ""}; $pre=$F[1]' file

Explanation:

  • The -a splits the input lines into the @F array, making perl act like awk.
  • The -F is the field separator
  • -p means print each input line
  • -l adds a \n to each print call, so print "" prints a newline.
  • unless($F[1]=~/^$pre$/i || $.==1) : Unless the second field is the same as that of the previous line (the i in //i makes the match case insensitive), or unless this is the first line.
  • $pre=$F[1] : save this lines second field as $pre.

@GlennJackman suggested a slightly different version in the comments which will probably be faster for larger files:

perl -F'\|' -lape 'unless(lc($F[1]) eq lc($pre) || $.==1){print ""}; $pre=$F[1]' file
share|improve this answer
    
thanks for the Perl solution. :) –  Ramesh Mar 26 '14 at 16:05
    
@Ramesh you're welcome, I added an explanation of how it works. –  terdon♦ Mar 26 '14 at 16:10
    
It is really an elegant solution. Thanks again :) –  Ramesh Mar 26 '14 at 16:13
    
This seems like regex overkill: $F[1]=~/^$pre$/i -- I wonder if this would be simpler: lc($F[1]) eq lc($pre) –  glenn jackman Mar 26 '14 at 21:05
2  
Also, -l automatically chomps each incoming line, so print $_ outputs the right number of newlines (one). –  glenn jackman Mar 26 '14 at 21:06
up vote 4 down vote

A little shorter Perl solution:

perl -pe 'print "\n" if ($l =~ /name\d+/ && $_ !~ /$&/);$l=$_;' input
  • if last line ($l) was name\d+ and current line not last match, then print new line
  • assign current line to $l

A more general solution

perl -pe 'print "\n" if ($l =~ /\|([^\|]+)/ && $_ !~ /$1/);$l=$_;'  input
share|improve this answer
1  
Heh, +1 for the clever matching but I don't think the names the OP has will actually be in the form of name\d+. More likely they will be arbitrary strings. –  terdon♦ Mar 26 '14 at 23:32
    
thanks, yes that's right, I'm thinking about /\|.*?\|/, but it seems not to work. –  user55518 Mar 26 '14 at 23:53
    
You need to quote the match because of the |: perl -pe 'print "\n" if ($l =~ /\|.*?\|/ && $_ !~ /"$&"/);$l=$_;' –  terdon♦ Mar 27 '14 at 0:04
    
thank you. it's really weird. –  user55518 Mar 27 '14 at 0:13
up vote 0 down vote 
#!/bin/sh
#shell basics, POSIX compliant
(set -f ;IFS='|
' ; set -- $(cat) ; while [ -n "$3" ] ;do {
[ "${t=$2}" != "$2" ] && echo && t=$2
printf '%s|%s|%s\n' "$1" "$2" "$3" ; shift 3 
} ; done )<<\SAMPLE
17412193|name1|organization 
43979400|name1|organization 
1405541|name2|organization 
53595498|name2|organization 
50439202|name2|organization 
54678379|name3|Not Found 
21757330|name3|organization
SAMPLE

Output

17412193|name1|organization 
43979400|name1|organization 

1405541|name2|organization 
53595498|name2|organization 
50439202|name2|organization 

54678379|name3|Not Found 
21757330|name3|organization
share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.