Manipulation of text by programs, scripts, etc.
-1
votes
1answer
35 views
Match and combine records in Bash [duplicate]
Consider two csv files:
$ cat given.csv
123,John
246,Paul
369,George
987,Ringo
$ cat family.csv
246,McCartney
123,Lennon
987,Starr
369,Harrison
Note that the lines are not ordered! How might I use ...
2
votes
5answers
94 views
Extract lines from a file containg From <str1> To <str2>
Let's says there is file "abc.txt" that contains a paragraph/lines:
Hello, how are you doing sir?
when are you going to arrive at the SBAHN platform to catch the train?
How can I extract all the ...
2
votes
1answer
21 views
Comparing delimited fields
I have two fields in a file, students and teachers. They're delimited by a semicolon and I want to find which single students have two different teachers.
jdoe;ateacher
jdoe;bteacher
jsmith;cteacher
...
0
votes
3answers
56 views
Combinatorics of letters and words - preferably bash, will accept perl or other
I am trying to write a shell script to generate all possible words in the English language less than 20 characters. I doubt there is any truly efficient way to do this other than to brute force some ...
0
votes
2answers
34 views
Convert two rows to two columns
I would like to convert every two rows to two columns using awk. Any help appreciated.
input.txt:
# Query: gi|11465907|ref|NC_001872.1| Chlamydomonas eugametos genome, complete genome
# 0 hits found
...
0
votes
3answers
123 views
bash find using variable
I have an issue I don't understand. It is simple and it should work, but it doesn't.
=EDITED exactly what I can see from terminal=
I have a list of filenames:
[molni@archlinux picasa_album]$ cat LIST
...
2
votes
2answers
18 views
Replace line names with set of new names without disturbing the rest of the file
I have a large protein sequence file where each sequence name is identified using > and the corresponding sequence in the following lines.
Example (ignore quotation marks):
>YAL003W EFB1 ...
1
vote
3answers
81 views
How can I select columns based on a control file with invalid column numbers?
I have to generate the file dynamically from the source file based on the below control file.
control_file.txt
1,3,5,-1,8,-1,4
The control file contains the position of columns which I require ...
6
votes
2answers
255 views
Compare two files and print matches
Is there anyway to get this through Unix shell scripting?
I have a fileA with one column (1000 rows), and fileB with 26 columns(13000 rows).
I need to search each value of fileA with fileB and return ...
1
vote
0answers
18 views
Create files from templates, substitutions in middle of other words
Is there a way to convince m4 to replace a macro in the middle of a word?
I would like this file, day.m4:
define(Sat,Cat)dnl
Saturday
and this command:
m4 day.m4
to produce this output:
...
2
votes
3answers
43 views
Merging two tables including multiple ocurrence of column identifiers and unique lines
I asked a similar question yesterday (Merging two tables including multiple ocurrence of column identifiers) but ran into a problem with unique lines.
I would like to merge two tables based on column ...
1
vote
3answers
42 views
Replace word instances with its count attached to it
I have a file which contains word 'name' multiple times ( 250 times ).
I need to replace every instance of 'name' with its count attached to it as name_1, name_2, name_3....name_250.
Can I do this ...
4
votes
6answers
133 views
Sed — Replace first k instances of a word in the file
I want to replace only the first k instances of a word.
How can I do this?
Eg. Say file foo.txt contains 100 instances occurrences of word 'linux' .
I need to replace first 50 occurrences only.
2
votes
2answers
94 views
Merging two tables including multiple ocurrence of column identifiers
I would like to merge two tables:
File 1:
1 today
2 tomorrow
3 red
File 2:
1 a lot
1 sometimes
2 at work
2 at home
2 sometimes
3 new
Desired output (file 3):
1 today ...
1
vote
2answers
44 views
Count the maximum character length for all the data fields in a csv file and output to txt
Given a CSV with many data fields (>50), how can I count the maximum character length for each data field and then export all the counts to a txt file?
BTW, I want to ignore the first line of the file ...
0
votes
0answers
22 views
Shell Script for tracking number of times each command executes
How to have a shell script that takes input from /var/log/sudo.log and sends results to a spreadsheet that displays graph for each command execution? I want to record how many times each command ...
0
votes
2answers
26 views
How to find the occurrence of names in a file?
I want to know how many times users who have logged on the system in a single command.
I have used repetitive commands to find the how many times he have logged.
$ last | grep -c user1
$ last | grep ...
1
vote
3answers
40 views
sed match pattern N times
I want to split a file into chunks with 2 words each.
$cat tmp
word1 word2 word3 word4 word5 word6 word7
$sed -e 's/word. word. /&\n/g' tmp
word1 word2
word3 word4
word5 word6
word7
$sed -e ...
1
vote
1answer
40 views
Use case for /usr/bin/lessecho
I can not come up with an example of how I could/would use lessecho. The description in the man page is fairly straight forward but is not helping me come up with a use case:
DESCRIPTION
...
1
vote
4answers
150 views
Decimal conversion number of the form x.xx to 0x.xx, sed?
I have a file of the form
...
num 0.12 num num
num 25.53 num num
num 7.82 num num
...
and I want to convert the 2nd field of each line adding a "0" at the numbers >= 0 and < 10 so the output ...
-2
votes
1answer
36 views
Place the cut field in the desired position of ouput file [closed]
I am writing a script like in which, it will cut the fields from input file and append to output file.
cat $1|while read line
do
acc_no=`echo "$line" | cut -c29-43`
acc_type=`echo "$line"|cut ...
1
vote
1answer
42 views
Store output from one command and process it for another
I am working on Unix command line. I have two files. I want to cat file1.txt and grep the output in file2.txt
File1.txt
123A
223A
143A
153A
183A
123J
123P
File2.txt:
123A (TYU)
223A (RUT)
143A ...
1
vote
4answers
47 views
Parsing complex text file using Unix commands
I have the following text structure which I would like to parse:
>Cluster 423
0 56aa, >HWI-ST1448:257:C3V2HACXX:1:1106:19087:2550.1... at 92.86%
1 64aa, ...
2
votes
3answers
61 views
how to concatenate a file with multiple header into one
Can you suggest me how I can combine.
a.fasta with 10000 sequences having headers 1, 2, 3, .... 10000 .......................into just one header e.g
>1
AAATTTTGGGGCCC
>2
ACCCCGGGTTT
...
2
votes
0answers
17 views
Make GNU indent break long comments
Is there a way to convince GNU indent to break long comment and change it to multiline comment?
Something like this:
// Very long comment, longer than 80 characters. Just imagine that.
To this:
/*
...
3
votes
2answers
177 views
awk: Extracting a fixed number of rows where the last row number may vary
This question may look like a duplicate, but only at first glance.
Of course, I would no longer need help in how to code a one-liner that extracts a fixed number of continuous lines (e. g. 5 in this ...
1
vote
3answers
64 views
Print content of file from desired character and end also to desired character
I have this file with content written below.
I want output of my file stating with ".SUBCKT C032_THS_COMPARATOR" and end with ".ENDS":
.SUBCKT C032_THS_DPDNPROT_cdm_tdpdnw TOPROTECT VPLUS
.SUBCKT ...
-2
votes
2answers
54 views
Grep and cut the selected field
I am trying to cut the field if a line starts with a particular name,
cat $1|while read line
do
if [ "$1" = "custbills.cmp" ]; then
acc_no=`grep "^Custbills" $1 | cut -c29-43`
acc_type=`grep ...
5
votes
5answers
83 views
Replace data at specific positions in txt file using data from another file
I have a text file in the below format:
$data This is the experimental data
good data
This is good file
datafile
1 4324 3673 6.2e+11 7687 67576
2 3565 8768 8760 5780 8778 "This is line ...
3
votes
3answers
141 views
Replacing a pattern in line preceding another pattern of interest
I have a log file with many instances of statements as shown below:
14 Aug 19 16:30:11.506: <DATA> |POS|IDLE|1|01131844090|5950|$hostIp|$size |$data
20 Aug 19 16:30:12.439: <DATA> ...
5
votes
3answers
74 views
How to use awk through multiple files?
I want a solution for the general case (N folder)
I'm using awk to process a file and extract its content and put it in a variable then echo it:
This is the file:
H1 H2 H3 H4 H5 H6 H7 H8 H9
not ...
0
votes
4answers
46 views
Adding a character to every other text line
I have a huge (ca. 20G) text file which contains millions of passages (a.k.a. paragraphs) with headers. Headers and paragraphs are always one line each, e.g.,
Sunshine
This is a sunny day.
...
3
votes
2answers
78 views
Extract specific set lines matching a rule
I have a large file with the logs as shown below. There are about 30000 instance of such events logged. I need to extract those lines beginning with RINGING and CLOSE (included) and which does NOT ...
0
votes
2answers
47 views
Sum up the selected fields
I have a file with a numeric value in character positions 86 to 99. I want to sum up the entire amount present in the file, only for lines which contain RS at position 124-125.
I am trying to do like ...
4
votes
4answers
63 views
stream edit >GDRLR5L04EQY8D 3931|1166 into >3931
I have large text DNA sequence files with the following format
>name
AACTGGGCT
I need to parse the name line which currently has extra information and special characters, while keeping any line ...
1
vote
2answers
33 views
Error: ambiguous redirect while transferring output to command
Output to redirect in script:
511@ubuntu:~/Unix/test$ ls -ltr |awk '{print $9}'
default.txt
dfah.txt
fruit.txt
fruit_prices.txt
dfh.txt
header.txt
testfile.txt
topoutput.txt
Script written on shell ...
3
votes
2answers
53 views
Compare 2 tab delimited files and output differences with column header
I would like to compare 2 similar files on a common column. The files will have identical headers.
file1.txt
mem_id Date Time Building
aa bb cc dd
ee ff ...
1
vote
3answers
42 views
cat a directories files apending the file name to the row of text and removing the header row? [closed]
How can I cat a directories files apending the file name to the row of text and removing the header row? So as to only have 1 header row in the all.txt file.
3
votes
3answers
86 views
Determine maximum column length for every column in csv-file
To determine the maximum length of each column in a comma-separated csv-file I hacked together a bash-script. When I ran it on a linux system it produced the correct output, but I need it to run on OS ...
4
votes
1answer
40 views
Truncated result returned by JOIN
I'm using join command under linux, but the results vary between different machines.
I have two simple files:
cat 1.txt
a aaa,0.2
b bbb,0.3
c ccc,0.5
cat 2.txt
a aaa,0.2
b bbb,0.3
c ccc,0.6
I'm ...
1
vote
4answers
28 views
Loop through array
I have records in list.txt (INPUT) as
List of animals SET 1=Dog 2=Crow 3=Snake Pet,bird,reptile
List of Countries SET 1=France 2=Singapore 3=Columbia Europe,Asia,SouthAmerica
List of Parts SET ...
3
votes
3answers
184 views
Get the part of a line before the last slash
I'm trying to write a conky script that shows my MPD album art, a 'folder.jpg' in the album folder. My current plan is to use mpc -f %file%, which prints out the file name and path, and then cut out ...
1
vote
1answer
51 views
Why is the output of this paste command split on separate lines with indents?
I was trying to merge files using either of these commands:
paste data_{1..2}.txt > 1_2.txt
paste <(awk '{print $1}' data_2.txt ) <(awk '{print $1}' data_2.txt) > 1_2.txt
But the ...
5
votes
7answers
519 views
How to find a position of a character using grep?
I need to identify the postion of a character in string using grep command.
Example, the string is RAMSITALSKHMAN|1223333.
grep -n '[^a-zA-Z0-9\$\~\%\#\^]'
How do I find the position of | in the ...
4
votes
3answers
598 views
Replace “o” with “0” in large wordlist and save the original word?
I have a large text file with words. Every word is on a single line (typical wordlist).
I want to replace all characters "o" with the number "0" in every word, and the new formed word to be pasted on ...
0
votes
1answer
52 views
Extract values from HTML Files
I have 2 HTML files, where part of the content looks like this:
In FILE1:
<td width="48%" align="right" valign="top">
<b>mom. Wirkleistung P+ tot.: </b><br>
<b>mom. ...
3
votes
3answers
110 views
Finding groups of IP addresses in a file
I have a file with groups of IP addresses. The file looks like this:
London:
1.1.1.0-1.1.1.200
172.25.2.0-172.25.2.100
Germany:
2.2.2.0-2.2.2.100
192.168.1.0-192.168.1.200
172.25.2.0-172.25.2.200
...
1
vote
1answer
24 views
What does “position” mean in iconv error message?
I get error when converting a large file:
$ iconv -f GB2312 -t UTF-8 2001.txt -o 2001_u.txt
iconv: illegal input sequence at position 245256667
What does the position mean in this error? I tried it ...
2
votes
4answers
72 views
Filter Require Capital words from a file( not all capital words )
I want Output of this file only
AVDDPLL1V8
AGNDPLL1V8
DVDDPLL1V1
DGNDPLL1V1
Please help me in this to find solution.
Here is my input:
6.1.2 Power and Ground Pins
The following ...
0
votes
1answer
50 views
How to copy rows of different files and paste column-wise?
I want to create a new file by copying rows of each file and pasting columnwise according to their file name orders.
Here is the sample input and required output files below.
file1.txt
1234
5678
...
