Tagged Questions
12
votes
6answers
5k views
Best way to follow a log and execute a command when some text appears in the log
I have a server log that outputs a specific line of text into its log file when the server is up. I want to execute a command once the server is up, and hence do something like the following:
tail -f ...
10
votes
2answers
16k views
How do I delete the first n lines of an ascii file using shell commands?
I have multiple files that contain ascii text information in the first 5-10 lines, followed by well-tabulated matrix information. In a shell script, I want to remove these first few lines of text so ...
10
votes
2answers
993 views
Simplest way to comment/uncomment certain lines using command line
Is there a way to comment/uncomment a shell/config/ruby script using command line?
for example:
$ comment 14-18 bla.conf
$ uncomment 14-18 bla.conf
this would add or remove # sign on bla.conf on ...
9
votes
6answers
826 views
Text Manipulation Across multiple lines
I Have a file that has text like this:
AAAA
BBBB
CCCC
DDDD
1234
5678
9012
3456
EEEE
7890
etc...
And i want to match up the Alphabetic lines with the Numeric lines so they are like this:
...
9
votes
4answers
125 views
Script to extract selected entries from a bibtex file
I have a large bibtex file with many entries where each entry has the general structure
@ARTICLE{AuthorYear,
item = {...},
item = {...},
item = {...},
etc
}
(in some cases ARTICLE might be a ...
8
votes
4answers
2k views
How to prepend a license header recursively for all .h and .cpp files in a directory
I'm trying to add a license header to all header files and source files in a project directory using a for loop. This is not working, is there any other approach using sed?
7
votes
7answers
770 views
Slick one-liner to convert a list like “1: 2, 3, 4, 5” to “1.2, 1.3, 1.4, 1.5”
Let's say I have a file that looks something like this:
23: a, b, c, d
24: b, d, f
25: c, g
and I want to get output like this:
23.a
23.b
23.c
23.d
24.b
24.d
24.f
25.c
25.g
Of course it's not ...
7
votes
5answers
394 views
List out strings which are substrings of other strings in the list
I have a list of names like so:
dog_bone
dog_collar
dragon
cool_dragon
lion
lion_trainer
dog
I need to extract out names that appear in other names like so:
dragon
lion
dog
I looked through the ...
7
votes
2answers
341 views
Strip // Comments From Files
What's the best way to strip all code comments in a given directory? I'd like to strip out all // ... EOL comments, and /* blah \*/ (or /** ... \*/) comments as well.
This is a PHP project, and I'd ...
6
votes
10answers
592 views
Copy only Specific text of a file to another
I have a file abc.txt the contents are
<classpathentry kind="src" path="Sources"/>
<classpathentry kind="con" path="WOFramework/ERExtensions"/>
<classpathentry kind="con" ...
6
votes
7answers
226 views
Count total number of lines before/after a pattern match
I am having a long list of IP addresses, which are not in sequence. I need to find how many IP addresses are there before/after a particular IP address. How can I achieve this?
6
votes
2answers
508 views
Find all files with a Python Shebang
I'm trying to run a PEP8 check against a large source tree. The tree is comprised of a mix of files in various languages. The idea is to check all the Python scripts without having to explicitly list ...
6
votes
5answers
3k views
Compare two files with first column and remove duplicate row from 2nd file in shell script
I will ask my question with an example. I have 2 files:
File #1:
118D FC300_R5_TP FX.B 32775 2112 6 2038 6 2112 0
118E FC300_R5_TP FX.B 32775 2136 7 ...
6
votes
3answers
72 views
Print a multiline variable in a column
I have a script that collects values from a cluster. The values, in some cases, have multiple lines. I have a printf format that specifies how the data should be positioned however, it doesn't take ...
5
votes
4answers
1k views
how do I trim each line in bash?
I would like to remove all leading and trailing spaces and tabs from each line in an output.
Is there a simple tool like trim I could pipe my output into?
5
votes
3answers
259 views
removing redundancy from output columns
What is a *NIX way of removing redundancy in a case where I have pairwise comparisons like these in two columns
A B
B A
A C
A D
C A
D A
B C
C B
A B and B A ...
5
votes
4answers
2k views
Why is sed giving me an error about an unterminated `s'?
I have a set of sed replacements in a bash script and am getting an error about an unterminated `s' command. Here's what the sed line looks like:
sed -n -e "s/TMPFOO1/$FOO1/" -e "s/TMPFOO2/$FOO2/" ...
5
votes
3answers
6k views
How do I append text to the beginning and end of multiple text files in Bash?
I have a directory full of text files. My goal is to append text to the beginning and end of all of them. The text that goes at the beginning and end is the same for each file.
Based on code I got ...
5
votes
4answers
8k views
Text between two tags
I wanna retrieve whatever is between these two tags – <tr> </tr> – from an html doc.
Now I don't have any specific html requirements that would warrant for an html parser. I just plain ...
5
votes
4answers
1k views
Tool to create text files from a template
I have to regularly create 100+ text files from templates.
I currently use an overly complicated shell script. I think there is a smarter way to handle this, but I don't know how.
I have a "database" ...
5
votes
3answers
56 views
How to put the output of the trailing lines of grep in on-line?
Assuming that I have this file:
Thu May 8 15:32:07 2014
User-Name = "Mark"
Framed-IP-Address = 0.0.0.0
Acct-Status-Type = Interim-Update
Acct-Input-Octets = 95684
...
5
votes
2answers
161 views
Make a directory for each row within the given text file?
I'm trying to write a bash shell script that will read the data from a given (as an argument) file, and for each row in the file, make a sub-directory under the given (as an argument) directory.
The ...
5
votes
3answers
216 views
How to extract URIs from files
How can I extract from a certain number of files all the URIs like magnet: file:? I'd like to have one URI per line as the output.
I'd like to use bash to achieve this, but any suggestion is welcome.
...
5
votes
2answers
206 views
Shell script (bash) to add or replace the license text contained within .c , .h and makefiles?
I have a set of *.c , *.h and Makefiles in a folder where some files contain the license text and some files don't have any license text. So I need a shell script where I can add license text if the ...
5
votes
1answer
160 views
counting multiple patterns in a single pass with grep?
I've written a grep loop to iteratively count DNA trinucleotides within a gzipped DNA fasta file containing DNA sequences e.g.
declare -a tri=(AAA AAC AAG AAT CAA .. etc)
for i in ${tri[@]}
do
...
5
votes
3answers
2k views
Editing config file via a bash script
I've been trying to write a simple bash script which I'll be using to install an application and update it's config file. I'm having hard time to have get it's config file modified.
# ...
5
votes
2answers
65 views
Extract tomcat parameters from ps output
I'm trying to write a little diagnostic tool and one of the functions I want from it is that I want to run is:
ps axu | grep tomcat | grep -v grep | awk '{ print $<list of desired fields> }'
...
5
votes
1answer
395 views
How to append multiple lines to a file without last newline?
I have a very long string that is split in chunks. I want to append them to a file without putting the newline character using bash.
Example:
First append
cat >> abc.sh << EOL
echo "bla ...
4
votes
7answers
322 views
How can I groups numbers in a file
I have a file with numbers in float format.
I can review them via sort -rn numbers.txt | less
I would like though to be able to "group" them. I.e. easily see how many are in the same range.
To give an ...
4
votes
4answers
449 views
Joining two files with unique identifier
I have two files with approximately 12900 and 4400 entries respectively, that I want to join. The files contain location information for all landbased weather observing stations around the globe.
The ...
4
votes
2answers
948 views
How to output file & ignoring lines that start with “?”?
I do svn status --show-updates and then I want to either
Q1:
ignore (not to display) lines that start with ?
Q2:
display only lines that start with * Note that there are few spaces before * occurs. ...
4
votes
3answers
113 views
Frequency of words in non-English language text: how can I merge singular and plural forms etc.?
I'm sorting French language words in some text files according to frequency with a focus on insight rather than statistical significance. The challenge is about preserving accented characters and ...
4
votes
3answers
194 views
How can I delete a trailing newline in bash?
I'm looking for something that behaves like Perl's chomp. I'm looking for a command that simply prints its input, minus the last character if it's a newline:
$ printf "one\ntwo\n" | ...
4
votes
5answers
4k views
How can I convert tab delimited data to comma delimited data?
I'm requesting a list of ec2 snapshots via amazon's ec2 command line tool:
ec2-describe-snapshots -H --hide-tags > snapshots.csv
The data looks something like this:
SnapshotId VolumeId ...
4
votes
3answers
264 views
Parsing string token with bash
I have this output (from a previous command):
aaa something
bbb someother
ccc blabla
For each line, I would like to create a file whose name is the first token, and whose ...
4
votes
3answers
770 views
Change the text on a single line number in multiple text files
Suppose that I have ten bash shell scripts: script1.sh, script2.sh, ..., script10.sh. Initially, all ten files are identical.
Now, I would like to make two changes in each script:
In each file, ...
4
votes
4answers
557 views
Count lines matching pattern and matching previous line
I want to count the number of times that a role is successfully deleted. The problem is that one line of the log file will show that the script is about to delete the role:
Prepare to remove role X
...
4
votes
3answers
676 views
Find files that have words in common
What would be the best way to create a list of files that have common words with a given file. For example, if I had:
$ ls
mainFile file1 file2 file file4
$ cat mainFile
exquisite malicious ...
4
votes
1answer
2k views
How to find out common elements between two files?
For an example, I have 2 files having following info:
File #1:
12
13
14
15
File #2:
12 1
13 2
14 2
15 6
16 7
17 8
Output File:
1
2
2
6
In the output file, I want only the second column ...
4
votes
3answers
150 views
conditional coloring in grep
I have a simple bash script. One line of my script matches a "pattern" with grep and colors the match
grep -i --color=always -- "$1"
I need to modify it, so that it only colors the match, if the ...
4
votes
1answer
2k views
awk: forcing a return status?
This is a followup to my earlier question.
I am validating the number of fields in /etc/passwd using this handy snippit. In the following example, the users 'fieldcount1' and 'fieldcount2' have the ...
4
votes
3answers
672 views
using sed to replace pattern with hash values
I want to search the file and replace specific pattern with its hash (SHA1) values.
For example, let file.txt has the following content:
one S56G one two three
four five V67X six
and I want to ...
4
votes
2answers
1k views
sed: multi-line replace of config block
I have some configuration files that basically look like
(...content...)
# BEGIN DYNAMIC BLOCK - DO NOT EDIT MANUALLY
(... more content ...)
# END DYNAMIC BLOCK
(... even more content ...)
Now, in ...
4
votes
2answers
2k views
How to remove lines included in one file from another file in bash?
I have two lists, a complete list and a partial list. I want a bash script that will search the complete list for any lines that are in the partial, if there are, they will be removed from the ...
4
votes
2answers
36 views
How to sort a collection of lines from different files?
I want to sort second line in each file by second column, and also print name of the corresponding file. I am doing it like this -
rm tmp;
for filename in file*; do
num=`head -2 $filename | tail ...
4
votes
2answers
117 views
Remove duplicates from basenames in two files
I have two long lists of filenames, and I want to merge these two by excluding duplicated filenames in the sense of basename. For example, I have something like
../data_folderA/file_1
...
4
votes
6answers
61 views
reading a list of lines from a file writing output to a utility
I have a file with a list of lines I want to read from another file. I want to output those lines to a utility (grep) that lets me read the entire line and pull information from it. The file with the ...
4
votes
3answers
405 views
Linux command line that will fail if any standard out is produced
Is there a simple utility which I can pipe output to on Linux that will:
Return a success code if there is no output on standard out (and / or standard error).
Return a failure code if output is ...
3
votes
3answers
3k views
How to search and replace text in all php-files in a directory and it's subdirectories
I am looking for a shell script that recursively traverses all .php files in a directory and performs a search & replace of a particular text pattern.
The search pattern is quite long ( > 5000 ...
3
votes
1answer
2k views
Why is 'while.. read.. <<EOL' performing variable expansion, yet <file and | don't?
Aside from the variable expansion mentioned in the question's heading, I also experience another alarming issue when reading inline data to <<EOL... When the data contains a backtick `, it ...
