Manipulation of text by programs, scripts, etc.
0
votes
2answers
50 views
compare two files and fetch data
I have two files:
File A sample
>> AT5G46880.1|PACid:19669936
# score bias c-Evalue i-Evalue hmmfrom hmm to alifrom ali to envfrom env to acc
--- ------ ----- --------- ...
1
vote
3answers
43 views
Remove special pattern ^M from script which got appended after FTP from windows to Unix
I send a windows file using FTP to an unix system and got appended ^M wherever a new line was intended, and I just want to remove them.
One method which I can opt is to run dos2unix command.
Can ...
0
votes
3answers
32 views
How to remove a string of characters after and before a specific character?
I have a file with thousands of lines that start with:
>Miriam132_38138 Otu32555|1
I need to remove 'Miriam*********' so that each lines begins with:
>Otu32555|1
The first string of ...
1
vote
2answers
38 views
Remove string from file name to leave variable?
I have the following list of files:
Dorn_Triatomine_A5201_sequence_1_unmappedforTdim_tdim.alleles.tsv
Dorn_Triatomine_A5201_sequence_1_unmappedforTdim_tdim.matches.tsv
...
3
votes
4answers
168 views
Replacing new line character with a pipe and a new line character
I am using ksh.
I have a file temp.txt with some pipe delimited data.
one|two|three|four|five
abc|def|pqr|lmn|xyz
As clear from example, the record ends with a new line character after the data ...
0
votes
3answers
32 views
How can I swap the order of each two character pair in a string?
I am trying to automate the swapping of characters 1 and 2, 3 and 4, etc, in a an arbitrary-length string of hex characters.
Example:
A627E39B
becomes:
6A723EB9
It seems like this should be ...
2
votes
2answers
40 views
Replace between patterns that start and end in different lines
I have a file with the following format:
Whatever1
Whatever2
Whatever3
binaries=(
text1
bin2
ohhh3
)
Whatever4
Whatever5
Whatever6
I need to replace whatever inside lines binaries=( and ) with a ...
-5
votes
2answers
55 views
PDF: Using Terminal [on hold]
I tried creating a PDF file using terminal . I wrote "vi New.pdf". It created the file. Now that when i trying to double click and open it . It said something like :-
The file "New.pdf" could not ...
2
votes
2answers
21 views
Utility which gives line from /proc/id/maps by specific address
Is there an utility that gives me specific line from /proc/id/maps when given specific address from this process?
For example when given 0x7fb4a84f6008 it returns:
7fb4a84f6000-7fb4a84fd000 r-xp ...
3
votes
4answers
202 views
How to truncate the second column to given length
Given input of the form
XY981743 foobarlkasdf saflkas asfZR!sgfad asdSAD asdsadf SAdfasdf46lk lksad bar foolkasjfdrte
how can I truncate only the second column? The delimiter is TAB and the ...
0
votes
3answers
56 views
Printing column value in the next line
My input is in TSV format:
C1
r1
r2
r3
C2
r1
r3
Desired output is:
C1 r1
r2
r3
C2 r1
r3
So, basically what I want is printing value of column 1 in next line, is it ...
0
votes
1answer
32 views
How can I append and prepend text to single line text output and save it all to one file?
I have a database dumped to sql with create table commands and I need to create a script to apply a given command to some of the objects, such as dropping the table as an example.
In the case of ...
4
votes
1answer
81 views
Problem with paste and standard output in linux
I have two files that I am trying to merge, one file is:
linux$ cat temp2
linear_0_A_B linear_0_B_A
103027.244444 102714.177778
103464.311111 102876.266667
103687.422222 103072.711111
103533.244444 ...
5
votes
6answers
288 views
split file into two parts, at a pattern
How to split a large file into two parts, at a pattern?
Given an example file.txt:
ABC
EFG
XYZ
HIJ
KNL
I want to split this file at XYZ such that file1 contains lines up-to XYZ and rest of the ...
0
votes
3answers
65 views
Reorder lines in each 3-line record
I have a file which contains lots of records in following manner.
Name
Address
Phone
This is the sequence for each record in that file. Now, I want to copy this file to other file with changed ...
2
votes
2answers
78 views
Extract substrings from each line. These substrings are in comma separated, “name=value” format
I have lines in a file that are comma separated. There are no column headers, it's mostly comma separated 'name=value' pairs.
Here is some test data:
...
2
votes
3answers
47 views
Reformat date string
I am trying to figure out how to reformat a file full of wrongly formatted dates. The source looks like this:
{"_id":"","timestamp":"Mon Apr 20 08:30:55 +0000 2015"}
{"_id":"","timestamp":"Mon Apr 20 ...
0
votes
1answer
45 views
How can I regex specific links from rss content in yahoo pipes?
From a RSS content I'd like to remove everything except specified href links
Example:
<div class='text'>
<div class="center"> <b><a rel="nofollow" target="_blank" ...
-1
votes
2answers
49 views
How to split output lines into multiple lines (only lines not files)? [duplicate]
My requirement is display output lines into mutliple lines
My code is:
#!/bin/bash
dir="$1"
echo -n "file size:"
du="$(du $dir -hab | sort -n -r | tail -1)"
printf "%s\n" "echo "$du""
It's ...
1
vote
2answers
67 views
How to check each line within a pipe delimited file to be same of different
I have a pipe-delimited file which can have millions of records. What I want to do is sort the file and remove if there are any duplicate lines there.
Then I need to check the first 32 columns and ...
1
vote
2answers
54 views
Search and replace multiple strings in a script using vim(search and replace) commands without sed?
A script that creates a file for weather precipitation is,
#!/bin/ksh
x=10
b=snow
c=rain
d=snow
e=hail
echo "$x,$b,$c,$d,$e" > weatherfile
exit 0
given
3,8s/snow/dry/g
For the search and ...
1
vote
2answers
37 views
Copy line in file if we know the pattern [closed]
We have a file which has 10 lines, and I know that some line has text QWERTY.
How can I manipulate the file so that it copies that line and paste it.
The output is similar to that yy and p command ...
0
votes
1answer
53 views
Help in manipulating a test file
I have a file look like
aaa/bbb/ccc.bbb
aaa/bbb/ddd.bbb
aaa/eee/fff.eee
aaa/eee/ggg.eee
I would like to skip lines ending similarly (where ending is defined by the extension, e.g. .bbb & .bbb) ...
0
votes
3answers
52 views
Removing line-breaks, whitespaces and tabs from every file
Say I have a folder with 10K text files. I would like to remove every space, TAB and linebreak from each file.
How can I do this efficiently?
1
vote
2answers
67 views
AWK script to compare content of 2 files
I need experts help in resolving my following problem ... I have 2 files ..
file1:
abc|123|check
def|456|map
ijk|789|globe
lmn|101112|equator
file2:
check
map
equator
globe
AWK function ...
2
votes
3answers
91 views
add space 5 times at the beginning of each line in a text file
I am looking for a command line or bash script that would add space 5 times before the beginning of each line in a file.
for example:
abc
after adding spaces 5 times
abc
0
votes
2answers
44 views
Using sed, add line X lines above match
I know that to prepend/append something, I just have to do:
sed -i '/pattern/a new thing!'
However, I'd like to match a line, and prepend something two lines higher.
I'd like to limit tools only ...
0
votes
2answers
41 views
Order the output using sed [duplicate]
I need your help using sed. I have this output after executing diff:
< #R1#Number = Gauge32: 258 Name = STRING: "TATA"
---
> #R1#Number = Gauge32: 280 Name = STRING: "TATA"
I need to ...
0
votes
3answers
35 views
Loop for snmpwalk in bash
I have an issue in my output using snmpwalk so i'm trying in this script:
var=`snmpwalk -v2c -c private ${ip} .1.1.1.6`
echo "${hostname}=>${var}"
to have this output:
R1=>Gauge32: 240
...
0
votes
3answers
70 views
Remove lines in a shell script
I have this file:
# more file.txt
2c2
lns-ld-wall-01-t2 old:261,260
4c4
Prive_ORANGE old:258,259
I need to remove all lines like this one 2c2 and this one 4c4. How can I do in a shell script?
5
votes
3answers
70 views
How to use sed to manipulate continuously streaming output?
I am putting together a presentation for a non-technical audience. I have a program running in bash that outputs a continuous stream of values, a few of which are important. I would like to highlight ...
-4
votes
2answers
33 views
How to extract the ten digit no 0470420692 from the description given below using unix command? [on hold]
Please give a one-line summary of your problem : 0470420692 verworpen gesprekken
OMS Order : 111111111
CDB Id : 666666666
MSISDN : 0470420692
SIM Card # : 2000000000000
Type of request : Order verder ...
0
votes
1answer
20 views
How can I match and substitute this multiline pattern in a bash script?
I have a file jail.local and it has sections like this
[apache]
enabled = true
logpath = /var/log/apache/error.log
[ssh]
enabled = false
port = ssh
I want to
match on the string [ssh] and
...
1
vote
2answers
34 views
Using Awk to Search Bro Log for IPs/Keywords Found in Online Text File
I am examining Bro logs and would like to find a way to use AWK to grab IPs from a tracker link and search the Bro logs for those IPs automatically, printing the results of each match into ...
3
votes
3answers
73 views
Shell script to move files if certain percentage of lines contain a certain string
I am working on a shell script to move files if certain percentage of the lines contain a certain string.
I've got ~2000 files in a directory that each contain a single column of data. The number of ...
2
votes
1answer
60 views
Command for killing specific PID provided by previous command
Sometimes I need to kill a process (the reasons why are not important). And I know I can find that process with the following command: lsof -i :8080, being my candidate the last process in the output ...
2
votes
1answer
55 views
Filter a log file by month, date and time in bash
I'd like to fetch the data between Apr 24 10:00:00.000000 and Apr 25 24:00:00.999999 in the following log but I am not sure how to get this work:
files/file1:Apr 22 02:47:00.663117 somedata
...
1
vote
1answer
29 views
How to filter a basic csv by columns
How do I filter by third column of the following file
list.txt [[ list of people ]]
billy baxter - @baxter - [email protected]
james woods - @woods - [email protected]
I currently have
cat ...
1
vote
1answer
56 views
Awk Command vs Grep
I had a question regarding the awk command. I want to take a bunch of files and find a single line from each and extract them into a comma seperated text file such that I can import it into excel for ...
1
vote
1answer
29 views
Read data from a file in a csh script
Is there a way to read data from a csv file into a Cshell script?
I have a small script, which calls another script recursively with a set of parameters. I want to read this set of parameters from a ...
1
vote
3answers
89 views
Using sed to consolidate diff output
I have this output as a result of comparing files with diff:
< IF-Name :STRING: "lns-wall-01-t2" Index:Gge32: 260
---
> IF-Name :STRING: "lns-wall-01-t2" Index:Gge32: 25
I need ...
6
votes
4answers
136 views
Extracting subset of lines of a file based on regex for first & last line
I have a large text file and I only want to look at some of the lines. The first line I want matches a regex R, and when the line matches the regex S, I don't care about that line, or any following ...
2
votes
2answers
160 views
Extract group name from `id` and store it in a variable
$ id usera
uid=830(usera) gid=799(groupa) groups=799(groupa)
I need to extract the group name from the output of id and store it in a variable. In this case it's groupname=groupa
1
vote
1answer
36 views
Merging rows with common values
I am trying to combine pairwise matches in a text file so that i can find all matches in a group: e.g.
my file contains two tab separated columns like this:
Simon John
Simon Paul
Steve Simon
Graham ...
4
votes
2answers
65 views
How to find out which process keeps tunnel interface(tun) up?
If I create a tun interface with ip tuntap add mode tun command and force it administratively up with ip link set dev tun1 up command, then the interface itself is always "physically" down:
...
1
vote
3answers
76 views
How to copy lines from multiple files into one new file and keep file name?
I have 81 files in .fasta format that contain (up to) 53 items. Such as:
/User/MyData/Sample_1.fasta
/User/MyData/Sample_2.fasta
....
/User/MyData/Sample_81.fasta
Each .fasta file contains a name ...
1
vote
1answer
38 views
Reinterpret terminal output
I have a program that executes some commands in a terminal (for example apt-get update) and stores the output.
How can I reinterpret the output, so I can remove all information that is not needed ?
...
-2
votes
0answers
43 views
Longest/shortest Username in linux
I'm trying to configure a script that displays all the users on the computer, the groups they belong to, and the shortest/longest usernames.
I have the first part set up with the users and groups but ...
0
votes
1answer
81 views
need help merging two Unix files
I have 2 pipe delimited files say file1 and file2.
Where file1 has say 33 columns and file2 may have 34/35/36 columns. So let's not consider how mant columns we have.
What I want to do is compare the ...
0
votes
1answer
60 views
make a table output
if it possible to do a snmpwalk in a perl script and put the output in a table to make a sort of association like for each hostname i have in the same line if index and desc i have the script in bash ...
