Manipulation of text by programs, scripts, etc.
-1
votes
1answer
18 views
Removing new line characters around patterns [duplicate]
I have a fasta file of millions of paired sequences, that looks like this:
>7001289F:56:HKH3FBCXX:2:1101:1692:2074 1:N:0:CGATGT
...
0
votes
2answers
36 views
How to get only certain fields from a given file?
I have list of employees in following format
Name:
Phone No:
Email:
Name:
Address:
Phone No:
Email:
Name:
Country:
Address:
Phone No:
Email:
Name:
Address:
Name :
Email:
Address:
All I want ...
2
votes
2answers
38 views
Insert text at specific line number
I'm working on a bash script that will split the contents of a text document depending on the data in the line.
If the contents of the original file were along the lines of
01 line
01 line
02 line
...
3
votes
2answers
41 views
sed pattern and multiline substitution
I need to parse a text file and replace a certain portion of it. I believe sed is the way to do it but not sure how to handle multiple lines with it
The pattern I am looking is
set cells { \
cell1 ...
1
vote
4answers
56 views
Bash: How to print additional column matching IP Address to Name
I'm using netstat to collect information about connected IP's. And I have the following format:
netstat -tn 2>/dev/null | awk '/:80 / {print $5}' | sed 's/.*::ffff://' | sed 's/:.*//' | sor t | ...
3
votes
2answers
43 views
Sorting by two columns
I need help with this homework assignment:
Write a shell script named sales1 using nawk and sort commands to do the following:
1. print the file in two sections, east and west, with lines alphabetized ...
1
vote
1answer
34 views
What does :<i> code stand for in subtitles?
I downloaded a subtitle from subscene http://subscene.com/subtitles/the-5th-wave/english/1298314 and it contains some coding such as :<i> and </i> which is not being ...
1
vote
1answer
38 views
Extract random sample of N lines based on pattern
I have a file formatted like this:
train/t/temple/east_asia/00000025.jpg 94
train/t/temple/east_asia/00000865.jpg 94
...
train/s/swamp/00000560.jpg 92
train/s/swamp/00000935.jpg 92
....
...
1
vote
2answers
21 views
How to append a string to the first column of a matrix file?
I have a matrix file as follows.
The first column is a name list.
I want to add _i1 to the end of each name.
A1A A1B A1C A1D A1E
TR6764_c0_g2 0.00 0.02 0.00 0.00 0.00
...
3
votes
3answers
45 views
Replace strings in a file based on a list of strings and a list of corresponding replacements
I am trying to replace strings in a file A:
Hello Peter, how is your dad? where is mom?
where the strings to be replaced are in file B:
Peter
dad
mom
and their corresponding replacements are in ...
0
votes
0answers
10 views
Linking and combining text from different documents into dynamic document
I am taking notes on many, related topics from multiple sources. For instance, one of the sources is a textbook on machine learning. I type the notes in an .odt document (but whatever, it could be ...
3
votes
2answers
45 views
On applying commands to groups of lines from stdin
The Unix package datamash supports the application of several summarizing operations to groups of input lines. For example1, here datamash is used to compute the sums of column 2 for each value in ...
1
vote
1answer
33 views
Removing columns from a file
I'm trying to remove columns 75 through 85. The colrm command works, but the problem is that it stops whenever it finds an extended ASCII character in the file.
So, my question is this: is there ...
1
vote
1answer
40 views
Compare independent lines in two files [closed]
I have two files, each with 500k+ user accounts. I need to find the differences between the two (the missing accounts/delta).
Each file is in a simple format, each line contains an email address in ...
3
votes
1answer
34 views
Get contents of a file from first instance to last instance of some arbitrary string
How can I view in less from the first instance of some arbitrary string "foo" to the last instance?
This arbitrary string will be on most every line of the log. I don't want to do grep "foo" bar.log ...
1
vote
1answer
32 views
Sed delete line if nth character is not specific character
I have a csv file, and I want to delete lines if their 12th character is not ;.
So, for example, my file looks like this:
2266308;A;B;dfsgsfdg
2266309;A;BJHSADFK;gfsdg
2266310;A;B;dfg
and I ...
2
votes
2answers
56 views
Split File into 10 files with unique item in each file and maximum line number limit in each file
I have a huge file (more than 2M records in it). Here is my requirement:
Initially, break the one big file up in to 10 smaller files.
The files should be formatted as follows: ...
1
vote
1answer
28 views
How to set csplit output filenames by pattern?
Is there a way to set output names of csplit through the pattern?
I want to split the ssh-config by hosts to corresponding files.
If there is a solution without csplit I'm happy as well.
0
votes
2answers
38 views
Bash Script with Multiple Lines and Grep
I have output from a specific command that can provide the number of times an application has been marked as 'used' on an OS X system. The command is mdls.
The output looks similar to the following ...
0
votes
1answer
40 views
subset a file just on one column
I have a tab delim file, I want to make a subset on the basis of column 2, choosing "A".
435 A 2 27.92464882 8.63E-07 1 752721
564 F 2 30.12532071 2.87E-07 1 798959
654 A 2 ...
0
votes
1answer
27 views
sed find line in file and remove characters
I have a file that contains lines like this
proto udp
.
.
.
..
server 10.10.8.0/24 255.255.255.0
I need a way to remove only /24 from this line server 10.10.8.0/24 255.255.255.0
to become
server ...
0
votes
2answers
44 views
How do I find a range of numbers in the content of file and move files containing the range of numbers to another directory?
Okay so I have about 90 files, each with a body like
187942 78.9
I want to filter files that has the values between 70 to 90 on the second column and move them to another directory.
I'm ...
-2
votes
3answers
67 views
Extract sequences from a fasta file
I have a fasta file (not in right format) that contains hundreds of thousands of different lengths of DNA sequences like this:
...
3
votes
2answers
52 views
match columns and replace
I am newbie in linux.
I have two very large text files with space-delimited fields:
File1
527858 51 2 27.92464882 8.63E-07
570289 82 2 30.12532071 2.87E-07
571034 90 2 ...
0
votes
0answers
11 views
Building ASCAT input files [closed]
I am trying to build input matrices of LogR and B Allele Frequency (BAF) data (ASCAT). I have copy number files and allelic frequencies files
Table 1:
feature chromosome start end segments
...
1
vote
2answers
45 views
How to add strings from a name list file to another file?
I have a fasta file (seq.fa) which is a standard file format for genetic info, like so:
>TR1|c0_g1_i1
GTCGAGCATGGTCTTGGTCATCTTCCTTTCAAAGAA
>TR6|c0_g1_i1
GTGGAATATCGCCAGTGACCATCACTGATTAACCTG
I ...
1
vote
3answers
71 views
How to do following formatting using sed or awk or bash script on linux
Suppose, I have two files:
file1 having below contents.
message_aaa.wav
test 1 of 1
test Label : aaa
test Sentence : test aaa
test String: test message
Interpretation ...
1
vote
2answers
45 views
AWK: How to put field ($1) inside regular expression to select all prefixes?
Assume I have a file A:
fileA
fileB
Suppose I have now a file named:
fileA_someprefix_20160101.txt
Now I want to match all lines from A which prefix this filename, so I thought:
...
0
votes
2answers
51 views
Transforming a list of files separated by new lines into a variable
Goal: I'm trying to find all instances of *.clj or *.cljs files recursively within a directory, store them in a string variable (separated by new lines) and then transform them.
So if the following ...
-1
votes
1answer
18 views
Sorting and outputting in another file
How can I sort a file then output it into another file I never created or do I have to create the file I want to put in. Might not make sense how I am typing it so...
Let's say the file I have is ...
4
votes
3answers
64 views
String replacement using a dictionary
What is a good way to do string replacements in a file using a dictionary with a lot of substituend-substituent pairs? And by a lot, I actually mean about 20 – not much, but many enough so that I want ...
0
votes
3answers
75 views
What is the most concise script to list the missing ones in a digit sequence? [duplicate]
For example, we have the content
001
002
004
008
010
in a text file named file, how to extract the missing 3 5 6 7 9?
0
votes
2answers
30 views
Find and replace words in text file recursively
I have tree of folders inside these folders there are many text files. These files have the same name "log.txt" and the same format. For example inside every file we can see multiple lines like:
date
...
1
vote
1answer
26 views
pattern regex for apache rewrite rule
I have the following pattern on a few thousand lines I will like to use regex to convert to what I want
RewriteRule ^/omg/deadly-venom/?$ http://www.example.com/omg/deadly-venom [L,NC,R=301]
...
1
vote
1answer
43 views
Why don't the SHA's match?
I am trying to write a script and it uses the SHA of a date but I am getting two different results and for the life of me can't figure out why.
echo -n 03112016 | cut -d'.' -f4 | sha256sum | cut -d' ...
3
votes
3answers
34 views
Strings - Extract File Version Recursive
I have been using strings to extract and compare file versions between directories.
I don't seem to be able to do this recursively.
The method I have been using per file is:
strings -e l ...
-1
votes
3answers
52 views
Better way to parse string with bash?
I need to parse the below command to extract namespace & interface names using a bash script. I generally do this using IFS, but that seems tedious to me.
Is there a better way of doing this ...
-1
votes
2answers
30 views
First and last five lines of file [duplicate]
How to see first and last 5 lines of file?
Right now I am using
head filename ; tail filename
Any other way around and more efficient?
1
vote
4answers
44 views
Remove any occurrence of '-' from a field in shell script
below is my csv file,I want to remove all occurrences of - from retailer_id field and create new csv. Kindly help...
...
0
votes
2answers
43 views
Compare a file with a column of another file
I have a file File1 with data as
aaa
bbb
And another file File2 with data as:
2,aaa,234
w,bbb,589
4,ccc,675
I need to compare the File1 data with column2 data of File2 and print the matching ...
0
votes
3answers
51 views
Awk command is not working properly
I have a very simple problem, but for reasons it does not work properly..
I have these .txt files in the following format
2 250 1
4 250 1
5 250 1
I wanted to subtract 1 from the ...
-1
votes
3answers
56 views
Change all values with an absolute value less than 1 with 0
I have a large file with a lot of rows and columns; the value of the contents are between -2 and 2. I'm interested in replacing values which, in their absolute value form, are less than; I want to ...
3
votes
5answers
61 views
How to extract data from a file and construct a filename from one of the values
I'm trying to pull two values from a configuration file and write them to a new file. The hard part is that I want the name of the new file to be determined by one of the values in the input file.
...
1
vote
2answers
27 views
Joining columns from files if they contain a match in another column
I have two files:
File1.txt
30 40 A T match1 string1
45 65 G R match2 string2
50 78 C Y match3 string3
File2.txt
match1 60 add1 50 add2
...
3
votes
6answers
163 views
Add quotes and new delimiter around space delimited words
I have the following string
y10_zcis y10_nom y10_infl y20_zcis y20_infl y30_zcis
I would like to transform this to
"y10_zcis", "y10_nom", "y10_infl", "y20_zcis", "y20_infl", "y30_zcis"
I ...
1
vote
1answer
40 views
How to test if certain strings existing in file are present in another file?
I have a file sites.txt that is generated by a script. It's a list of host names.
I have another list of host names sites_old.txt.
I need to check whether any of the host names present in the ...
1
vote
2answers
32 views
Reorganizing lines in a text file
So I have the following file:
First Line
Second Line
Third Line
3 0.374
2 0.777
1 0.245
Fourth Line
4 0.687
...
And I want to reorganize it so the lines beginning with an int are appended to the ...
0
votes
4answers
35 views
Difference between open file & open STDIN, when to use each [duplicate]
In very lot of answers, mainly about text-processing commands, I saw commands such as sed, awk, grep, among other, being used with STDIN and the simple open of a file
e.g.
$ sed -e 's|foo|bar|g' ...
0
votes
3answers
40 views
I want to manipulate the group file entries [closed]
Suppose I have a file:
uid=1(a) groups=cat,dog,hen
uid=2(b) groups=cat,hen
uid=3(c) groups=hen
uid=4(d) groups=dog,hen,buffalo,horse
I am looking for:
cat a,b
dog a,d
hen a,b,c,d
buffalo d
horse d
...
1
vote
0answers
48 views
how to copy a piece of a text file byte-by-byte to another text file? dd, head, or?
I need to grab the first lines of a long text file for some bugfixing on a smaller file (a Python script does not digest the large text file as intended). However, for the bugfixing to make any sense, ...