Manipulation of text by programs, scripts, etc.
0
votes
1answer
26 views
Comparing two lines in BASH using awk
I have text file like this.
2015-11-24 12:59:37.112 128.206.6.136 source
2014-11-24 12:59:36.920 8.8.8.8 source
2014-11-24 14:59:38.112 23.234.22.106 destination
2014-11-24 13:59:37.113 23.234.22.106 ...
1
vote
3answers
33 views
How to use Unindented line as the Record Separate in awk cli
I have a log file that look like:
2016-05-31 09:54:36 (16667) heritage_w?
From: ip68-8-49-100.sd.sd.cox.net
User: user1wizard (wizard)
Agent: Mozilla/5.0 (Windows NT 10.0; WOW64) ...
1
vote
1answer
25 views
Print some data related to certain field
I want to grep some information from raw logs:
51.254.56.62 - - [01/Jun/2016:20:49:28 +0500] "GET /vendors/jquery.slimscroll.min.js HTTP/1.1" 404 - "http://networkconfig.net/" "Mozilla/5.0 (X11; ...
2
votes
0answers
35 views
Compare lines and upgrade two different files
I have a couple of text files with the following features:
$ cat file_1
Line A
Line B
Line C
Line D
$ cat file_2
Line A
Line added 1
Line B
Line D
Line added 2
They are such that file_1 has some ...
0
votes
2answers
17 views
Interchange two Columns
I am trying to swap the hits and domain column.
Actual Output: cat filename
Hits Domains
204492 greenlivingasc.in
37102 smcracks.uk
17410 bahriains.org
...
0
votes
1answer
25 views
How to append some string combined with the pattern at the end of line with the pattern
I want to append .com at the end of line for every line matching the pattern in /etc/hosts file.
Sample file content:
127.0.0.1 localhost
1.2.3.4 hostname1 hostname1.xyz hostname1.xyz.
...
1
vote
1answer
32 views
Compare string in two files And Merge its Output
I am trying to get the bandwidth of the domain from one file and its hits details from another file.
The format of the two files is as follows:
bandwidth.txt
aadrivingschool.ws 2840.36M
...
0
votes
0answers
35 views
How tcan I delete columns from a file without losing the format? [on hold]
I have a file with this information:
1 2 3 4 5
O O O O O
Eigenvalues -- -20.44628 ...
0
votes
1answer
25 views
How to move lines between patterns to the end of a file
I want to move several lines to the end of the file.
Example:
1
2
3
4
5
6 blkn XYZ indf
7
8
9
10
11 lnkn ABCYDF knnfd
12
13
14
15
I want to move lines 6 to 11 to the end of the file. The patterns ...
2
votes
6answers
417 views
How do I remove a words ending in the letter S if duplicates without an S on the end are on the same list?
I have a large list of words. Many of the words are only different because they have the letter s on the end. If a word on the list is the exact same as another word on the list, except one of the ...
2
votes
1answer
49 views
How to sum numbers inside many 2D arrays
I need to sum numbers located in a file like this one:
column1 column2 column3
row1 a(1,1) a(1,2) a(1,3)
row2 a(2,1) a(2,2) a(2,3)
row3 a(3,1) a(3,2) a(3,3)
row4 a(4,1) a(4,2) ...
1
vote
3answers
37 views
How to delete whitespace surrounding equal signs in a string?
I'm trying to determine a way to delete leading and trailing whitespace surrounding the equal sign from a string. The whitespace may (or may not) be present. There may be 0, 1 or more equal signs.
...
2
votes
4answers
79 views
All possible permutations of words in different files in pairs
I have multiple files, let's say file1, file2 etc. Each file has one word in each line, like:
file1 file2 file3
one four six
two five
three
What I want is to combine them in a new file4 in ...
-1
votes
1answer
22 views
Replace values with #<value># with date - '01-Jan-2015'
I want to replace values in file.
Replace :
values #startdate# with - '01-Jan-2015'
values #code# with - 'SA','CA'
1
vote
1answer
41 views
Need to filter the data on the basis of two column having many to one relationship using awk
I have a big file having 50s columns and 100K of rows delimited by |. Now $2(col 2) has multiple type of $1(col 1) value which means col 2 will be repeated. So I have sorted the file. I need now to ...
-1
votes
2answers
43 views
how to change record or field separator of a file [on hold]
How can I change the record separator of a file? For example, concatenating multiple lines into a single line, by changing record separator from newline to space. Another example is to split a line at ...
1
vote
3answers
53 views
Remove all the rows below a specific row number from one specific column
I've asked a question before on true row deletion, and got some great answers, but I'm now interested to know how you would remove all the rows below a specific row number from one specific column. ...
0
votes
1answer
22 views
extract two columns and add at a specific location to another file that is already full of columns
I have a file where I want to extract two columns and add at a specific location to another file that is already full of columns.
Could I cat file 1 and make it a variable and then cat file 2 and ...
6
votes
4answers
66 views
True column deletion, not selecting all columns that you want
I know this is a simple question and in many ways has been answered before, however every use I've seen of awk, sed, grep etc have all been selecting and printing the columns you want. In a case where ...
2
votes
2answers
52 views
add a file name to lines in this file
I have fasta files, coming from different samples eg: 01.fasta, 02.fasta, etc.
All of them contain OTU list with the following pattern:
>OTU1
CCTACGGGCGGCTGCAGT
>OTU2
CCTACGGGTGGCTGCAG
I ...
2
votes
2answers
41 views
I would like to replace a pattern with another pattern found in the same line
I would like to replace the second tab-delimited field with the first underscore delimited field found in the same line
Input
Cho1_01:C4WMNACXX:1:250326554 1 2 4 2 -9 1 1 4 3 1 4 1 -9 4 1 4 1 3 ...
1
vote
0answers
21 views
Amend perl script so that words are matched on a word for word basis [migrated]
I have been using this perl script (thanks to Jeff Schaller) to match 3 or more words in the title fields of two separate csv files.
Original question here:
Matching 3 or more words from fields in ...
0
votes
1answer
26 views
How to insert fields from one file into fields of another file at particular location?
Is there any quick way to inserts fields from one file into another.
For example, if the file contents of file1 and file 2 are like this :
File1:
=======
size1
size2
size3
File2:
=======
char ...
1
vote
4answers
30 views
cut column 2 from text file
My text file has no delimiter to specify separator just spaces, how do I cut out column 2 to output file,
39 207 City and County of San Francisc REJECTED MAT = 0
78 412 Cases and ...
3
votes
1answer
53 views
Join arbitrary lines to output as one line
I want to join the 3rd and 5th line of a file to the output of a script without trailing new lines.
Is it possible to implement this construction by using the code:
{
echo 'Some text'
echo 'Some ...
1
vote
1answer
26 views
How to append the first line at the end of each line of a file with Sed?
I'm trying to append the first line of a file at the end of each line with a single sed command:
Example:
line 1
line 2
line 3
line 4
should become:
line 1 line 1
line 2 line 1
line 3 line 1
line ...
4
votes
2answers
53 views
Find IP addresses visiting /page1 but not /page2 from nginx access logfile
I need help with selecting specific data's from log files.
I need two scripts:
I need to select all IP addresses that only visited /page1
I need to select all IP addresses that visited /page1 but ...
3
votes
2answers
39 views
Get a string and place it at the start of a line until the next string is found
I have a big non-standard xhtml file that I'm using sed (about 4 times) to iterate down to just the basics I need to throw the data into a MySQL database. My last bit I'm struggling with. The file is ...
3
votes
2answers
82 views
Organizing three dimensional data with awk/sed/perl
I have this file (a sparse matrix):
PC.354 OTU1 6
PC.354 OTU2 1
PC.356 OTU0 4
PC.356 OTU2 7
PC.356 OTU3 3
I want an output like this (dense matrix -classic .biom table):
OTU_ID PC.354 PC.355 ...
1
vote
1answer
18 views
Library database
I have a database of readers and I would like to lend them a book and write it into the database.
Example:
Reader
753 Leonardo DaVinci [email protected] 321654987 Lended Books:
grep ...
0
votes
2answers
27 views
Generating wordlist permutations from existing files
I have two wordlist files; the first contains:
John
Jerry
Jim
the second contains:
one
two
three
The files are several thousand lines each, not just the 3 words above. How can I generate ...
1
vote
2answers
35 views
Adding 'exception' words to the rule of perl matching titles script
I have been using this perl script (thanks to Jeff Schaller) to match 3 or more words in the title fields of two separate csv files as answered here:
Matching 3 or more words from fields in separate ...
0
votes
3answers
60 views
How can I retrieve and compare two values from a file? [closed]
How can I retrieve and compare two values from the first two lines of a file?
I have the following text:
05-24-2016, 2:59:32,0,0
05-24-2016, 2:59:37,0,0
05-24-2016, 2:59:42,0,0
05-24-2016, ...
0
votes
3answers
55 views
How to extract logs between the current time and the last 15 minutes
I want to extract the logs between the current time stamp and 15 minutes before and sent an email to the people configured. I developed the below script but it's not working properly; can someone help ...
2
votes
1answer
37 views
How to standardize the number of characters in a column?
I have a column (in my data-file database) where the number of characters is not standardized, and need to standardize. How can I do it?
Example:
Sample_ID
159
148
1110
1100
150
But my bank does ...
3
votes
4answers
56 views
Remove a certain word in a file
How do I remove just abcd11.gmail.com and *test1: from line 1
and abcd14.gmail.com and *fortest2: from line 2 and abcd19.gmail.com and *asatesteg: from line 2 from below leaving the remaining as is? I ...
0
votes
1answer
48 views
Using regexp_replace to replace single quotes around string but not within words e.g. can't
I'm using the below code from a bash script to replace single quotes from my title column in a postgres database with spaces.
psql -U postgres -d my_database -c "UPDATE my_table SET title = ...
1
vote
1answer
41 views
Sort text by specific location on each line
What I want to achieve is sorting lines of text by a specific point in the text (the lines of text have no spaces or columns). I've worked out a solution to this, but I want to know if there is a more ...
3
votes
1answer
43 views
Substitute text fields with values from other file
Alright, I cannot figure out how to do this.
I have a fileA which looks like this :
([7]RIMS_ID) : "CNR"
(refGain_A[7]) ...
0
votes
0answers
24 views
copying every 2nd line starting from 1 [duplicate]
The below has actually been asked before, which I hadn’t realised. but out of curiosity, how would you go about it if you wanted not just odds or evens, for example every 5th line
I'm trying to copy ...
1
vote
2answers
67 views
Removing spaces from the fields in a semicolon-delimited file using shell script
I have a csv file like this:
03139; 5; IT1234978208; 20150930 ; CTZ 13/31.12.15;
03137; 6; IT1234978206; 20151015 ; CTZ 13/31.11.18;
....
....
03134; 8; IT1234974406; ...
-1
votes
0answers
21 views
Word processing [closed]
When I backspace in a word doc to erase to the left, the cursor always jumps to the right end of the line and erases. Are all word processors like this and how do I avoid it? This is a real nuisance.
-3
votes
0answers
19 views
Evaluating a Question Generation System [closed]
I'm finishing a System that generates questions automatically from declarative sentences.
Example:
"The capital of Russia is Moscow"
The system may generate questions like:
What is the capital of ...
1
vote
2answers
54 views
find string from one file in another if not present then remove from original file
I'm trying to make a script that looks through each line of one file and if a line fails to match anywhere in any line of another text file then remove that line from the original file. so far I've ...
1
vote
1answer
53 views
How to find match with surrounding words?
I want to find match with surrounding words. I also want it to be fast.
Here's what I've got:
grep -o -P '(\w+ ){0,n}match( \w+){0,m}'
n is number of words before the phrase, m is number of words ...
2
votes
4answers
55 views
Create text corresponding to multiple variable assignments from a single variable
I have a variable that has a value like this
line=dog,/path1,/path2
What I want is to create a variable from the value; i.e dog should be the variable and want to get the below output:
...
3
votes
1answer
45 views
Modify values in a column based on number of colons it contains
I would manipulate the contents of the second column based on the number of colons it contains. If the second field contains more than one colon then I would require the content before first colon ...
3
votes
5answers
310 views
Increment index in file
Alright, it seems I cannot find a way to do what I need.
Let's say I have a text file A like this
(freqBiasL2[27])
(SatBiasL1[27])
(defSatBiasL2_L1[27])
(defSatBiasSlope[27])
...
3
votes
1answer
29 views
Use a list file to extract a second list from a master file (and comment out some lines)…HOW?
I have two files: master.tbl and sites.lst
The file master.tbl is a list of sites, with their port configurations, port names, IP addresses, etc. There are lines containing site descriptions, ...
0
votes
1answer
25 views
Each line into a file (XML as part of file)
Here is a sample of the content in the file:
<:abc>test file name <:/abc> XYZ 1122
XML content is pretty heavy and big.
Under the directory I am working in I want to create a ...
