Manipulation of text by programs, scripts, etc.
1
vote
2answers
36 views
I need a text editor to edit/visualize >10GB text files
So as to vizualize, and hopefully edit, a very large text file, more than 10GBytes (a backup dump of whole database), I have tried using vim... it didn't behave very well.
I have also tried cat to at ...
3
votes
1answer
30 views
How to edit files in-place while performing operation on the values matching the searched pattern?
Given a list files containing random text and many datetimes in ISO format in it (e.g "2012-07-02T10:47:24+02:00"), how can I find all these datetimes and add 2 hours to each one ?
Ideally a solution ...
1
vote
3answers
43 views
Get separate used memory info from free -m comand
As a output of free -m command I got following.
total used free shared buffers cached
Mem: 2496 2260 236 0 5 438
-/+ ...
1
vote
3answers
49 views
How to get data count between two strings with multiple occurences in the same file
I have the data in my file as
START-OF-FIELDS
ID
NAME
DEPT
END-OF-FIELDS
START-OF-DATA
1|joy|cs
2|sam|ec
END-OF-DATA
START-OF-FIELDS
ID
NAME
DOB
DEPT
ADDRESS
END-OF-FIELDS
START-OF-DATA
...
2
votes
1answer
32 views
Best way to remove lines from a file where matching text (not whole line) exists in another file
I have a file of email addresses (file 1)
and another file (file 2) of lines of data that contain some of these email addresses
I want to compare files and remove all lines in file 2 that have a ...
0
votes
1answer
46 views
How can I find and filter a specific column in a .csv file? [duplicate]
I have .csv files with the following structure:
cat,dog,mouse,pig,bird,cow,...
21,34,54,566,78,1,...
23,10,12,569,56,4,...
32,20,13,123,56,3,...
34,30,44,322,66,2,...
I want to filter the column ...
4
votes
4answers
166 views
Count lines matching pattern and matching previous line
I want to count the number of times that a role is successfully deleted. The problem is that one line of the log file will show that the script is about to delete the role:
Prepare to remove role X
...
1
vote
2answers
53 views
Match ids and print original file
I have two files
Original: (5000 entries)
Chr Position
chr1 879108
chr1 881918
chr1 896874
…
and a file with allele frequencies (2000 entries)
Chr Position MAF
chr1 881918 0.007
...
0
votes
1answer
38 views
Help with understanding a regular expression
I have this regular expression \\..\\{3\\}$
I want to understand how this expression works to match a string. My thought is that it matches any 8 characters at the end of the line. Is that how this ...
3
votes
2answers
57 views
Concatenate block of lines with following block of lines
I am trying to process some text exported from lecture slides using pdf2text. The bullet points of some slides show up like this:
title for the list
-
-
-
a bullet point text
another bullet point ...
2
votes
2answers
77 views
Extract Data between comma
I have an output file which contains different numbers seperated by commas. But the issue is I do not know the number of commas.
I would in turn like this data to be grep on another file. For ...
2
votes
3answers
54 views
Linux command line that will fail if any standard out is produced
Is there a simple utility which I can pipe output to on Linux that will:
Return a success code if there is no output on standard out (and / or standard error).
Return a failure code if output is ...
5
votes
3answers
71 views
Filtering redundant paths from a list of paths
I have a list of relative paths such as this:
dir1
dir2
dir2/dir3
dir2/file1
dir3/file2
dir3/dir4
dir3/dir4/file3
In the example above, the specifier dir2/file1 (for example) is redundant, because ...
2
votes
2answers
96 views
How can I replace text after a specific word using sed?
I have a file named .ignore. In need to replace the projdir. For example:
ignore \..*
ignore README
projdir Snake
I need to replace Snake with, for example, "PacMan". I read the man page, but I ...
2
votes
6answers
126 views
Is there a faster way to remove a line (given a line number) from a file?
A related question is here.
I often have to edit a large file by removing a few lines from the middle of it. I know which lines I wish to remove and I typically do the following:
sed ...
2
votes
1answer
37 views
extracting and copying parts of tex files into new tex files
I have a bunch of tex files in different folders on my system. The goal is to create new tex files (using a template, that has the title page, packages, new commands,...) that combines all the ...
2
votes
2answers
54 views
Extract a column from whitespace-separated fields that are not aligned
How can we extract a column from output in which whitespace is not in a regular format?
For example, when we type ps -ef, it shows all the process, process id's.... etc. How can we get only the last ...
5
votes
2answers
152 views
Find all files with a Python Shebang
I'm trying to run a PEP8 check against a large source tree. The tree is comprised of a mix of files in various languages. The idea is to check all the Python scripts without having to explicitly list ...
0
votes
1answer
26 views
Select line not selecting whitespace
Is there a way to select (or paste) text without trailing/ending whitespace?
eg. To copy the whole line I press y, to paste it somewhere else I use p,
but it pastes the whole text with the trailing ...
41
votes
5answers
1k views
Why is printf better than echo?
I have heard that printf is better than echo and I can recall only one instance from my experience where I had to use printf because echo didn't work for feeding some text into some program on RHEL ...
3
votes
3answers
89 views
How can I replace text in the last non-blank line?
I want to find the last line of text in a file, and delete the comma at the end of it. I asked about this already, but, after I got an answer I realized my question was not specific enough.
This sed ...
2
votes
4answers
87 views
How do I correct both too many and too few space characters?
I have files that are edited by people other than me. They contain lines that look like this:
<E> This is text </E>
I use some Bash shell scripts do a bunch of text replacement ...
0
votes
1answer
74 views
How to delete a line containing pattern which is first found after a specific pattern
I am writting a shell script to delete an entry from a specific group.
eg: file name is dest.xml
<domain id="1" group_name="group1">
<node id="ABC">
<node id="PQR">
...
1
vote
2answers
59 views
Print fields from file only if line starts with a specific word
I have the following file:
…
LINK=dummy 172.17.100 => 10.218.11 [PATH=/etc]
FILE=hosts 172.17.100 => 10.218.11 [PATH=/etc]
FILE=network 172.17.100 => 10.218.11 ...
0
votes
2answers
54 views
Align columns in ASCII file
I have a text file that looks like his:
#c1 c2 c3 c4 c5 c6 c7 c8 c9 c10 c11 c12 c13
4599 995,274 2523,658 ...
11
votes
4answers
240 views
How to do a continous 'wc -l' with gnu texttools?
I know of course that
cat logfile.txt | wc -l
120
will tell me the number of lines in a file.
Whereas
tail -f logfile.txt
will show me the new lines that another program writes to logfile.txt.
...
1
vote
3answers
102 views
Getting a list of users by grepping /etc/passwd
For a homework assignment, I need to use grep to parse /etc/passwd and get a list of all users (their usernames only, nothing else). It should use a pipe, and the results should end up in the allusers ...
0
votes
0answers
11 views
plugin for gedit that can auto format Java code
I haven't been able to find a gedit plugin that allows one to auto format code, like in Eclipse:
Cntrl+Shift+F by default
There's only auto-indentation
Running Manjaro Linux x64 gedit 3.6.2.2
4
votes
2answers
100 views
Why doesn't this xargs command work?
I wanted to delete all .sh extensions so did this:
ls *.sh | xargs -I {} mv {} `basename {} .sh`
However it doesn't work, it behaves like basename returns unchanged file name.
Why is it behaving ...
3
votes
2answers
51 views
Comparing two columns with two columns
I have a file that I need to compare the values in two columns with another two columns. For examples:
Item A B C D
1 201 3101 3101 201
2 3101 201 202 3101
3 3101 201 ...
2
votes
2answers
52 views
using grep to show entries that match a pattern and are present at least X times
I have a file that have entries like this among other lines
Feb 16 17:30:18 ns1 dovecot: pop3-login: Disconnected (auth failed, 1 attempts in 17 secs): user=<[email protected]>, ...
0
votes
2answers
80 views
How to find and replace a particular string in a specific line
I developed a code which find and replaces a specific string in all xmls under all subdirectories. But i dont want all occurances to be replaced. In entire xml, code should find the tag xyz and ...
3
votes
3answers
37 views
How to separate numerical values from identifiers
I'm currently writing a shell script that seperate values from their identifiers (retrieved from grep).
For example, if I grep a certain file I will retrieve the following information:
value1 = 1
...
2
votes
5answers
122 views
How can I split a text file into multiple text files using Perl?
I have a file ABC_TabDelim.txt that contains the following:
00:00:00:00 00:00:05:00 01SC_001.jpg
00:00:14:29 00:00:19:29 01SC_002.jpg
00:01:07:20 00:01:12:20 01SC_003.jpg
00:00:00:00 00:00:03:25 ...
5
votes
2answers
101 views
How to cut (select) a field from text line counting from the end?
I know how to select a field from a line using the cut command. For instance, given the following data:
a,b,c,d,e
f,g,h,i,j
k,l,m,n,o
This command:
cut -d, -f2 # returns the second field of the ...
4
votes
4answers
200 views
Text between two tags
I wanna retrieve whatever is between these two tags – <tr> </tr> – from an html doc.
Now I don't have any specific html requirements that would warrant for an html parser. I just plain ...
1
vote
1answer
136 views
How to remove trailing whitespace at the end of the line in given files (more than one)?
I'm sooo fed up with useless spaces in source files that I've even configured vim to show them.
The problem is that I'm very often (if not always) have to deal with useless spaces with my mates ...
1
vote
1answer
71 views
How to match 2 large file and print the difference in shell script
I have 2 files.
File1 has 400k numerical records.
e.g:
1
2
3
4
5
6
.. and so on
File 2 also has 420k numerical records.
e.g:
1
2
3
4
6
.. and so on
Both these file are in unsorted manner. I want ...
2
votes
2answers
106 views
System calls, AWK, and Bringing in external inputs
awk '{ TEMPVAR="/usr/bin"; printf("%s", system("ls $TEMPVAR")); }' empty
In this example I'm trying to bring in the variable TEMPVAR into the system call. How would I do this?
What I'm aiming to ...
3
votes
2answers
85 views
AWK : Converting Row Formated Data into Columnar
I have a dataset as such:
Name: Jim Bean
Vice: Dice
ID: AFDSDFDSFDSFASFA
LoginTime: 12343314
Name: Bob Dylon
Vice: Trumpets
ID: AFD232SFDSFASFA
LoginTime: 12343314
...
1
vote
1answer
32 views
Replaced matched text in Vim with pattern
Here's a minimal example of what I'm trying to do. Start with:
"this")
"that")
[hundreds of lines like this]
End with:
"this",&this;)
"that",&that;)
[hundreds of lines like this]
I have gotten this ...
1
vote
1answer
138 views
Reading the contents of the file and splitting using ksh
We're using a ksh script for installing one product.
I've another config file, I'd need to read this configuration file from my main script
Content of the Configuration file:
...
4
votes
1answer
63 views
Why is uniq ignoring Unicode and lines with a single letter?
I'm trying to combine both the American and British dictionaries into one large dictionaries, and I'm trying to remove all the duplicates from the superset, but it seems that uniq is not outputting ...
2
votes
1answer
81 views
How can I use awk to (sometimes) remove the first character of a column?
I run a command and pipe it through awk as below :
mycommand | awk '{print $1,tolower($3),$4}
This gives me the following output:
fred.o: t .abcdefg
fred.o: u .rstuvwxy
fred.o: t .defghi
...
3
votes
6answers
122 views
how to use patch and diff to merge two files
I have read about diff and patch but I can't figure out how to apply what I need.
I guess its pretty simple, so to show my problem take these two files:
a.xml
<resources>
<color ...
4
votes
3answers
223 views
How to remove all white spaces just between brackets [] using bash? [duplicate]
Replace text between brackets
Input
testing on Linux [Remove white space] testing on Linux
Output
testing on Linux [Removewhitespace] testing on Linux
So, how can we just remove all the ...
2
votes
2answers
27 views
sort a file based on length of the column/row
I need to sort a file based on the number of chars in the first column.
I have no idea on how to go about this. (On Linux, so sed/awk/sort is available).
An example:
.abs is bla bla 12
.abc is bla ...
5
votes
1answer
63 views
array[@] output all messed up?
I've got this code:
Unix+=("Stock List")
while read line; do
result=$(wget -O - -o /dev/null "http://finance.yahoo.com/d/quotes.csv?s=$line&f=sl1&e=.csv" | tr ',' ' ' | tr '"' ' ')
...
4
votes
1answer
84 views
Modify an incoming mail from text/plain to text/html
I'm using mutt to process my mail. There is one sender which declares the wrong content type. The mail arrives with
Content-Type: multipart/alternative;
boundary=--…
and has two parts. They are ...
7
votes
1answer
116 views
Vim: Making XML text “pretty”
Is there an easy way within vi to make your XML text "pretty"?
For example:
I want this..
<person>
<name>Nick</name>
<age>26</age>
...
