Comma-separated values: files containing data arrange in a table, with commas to separate columns
2
votes
3answers
109 views
Replacing the values in one file with the values in another file in bash
I am having a csv file naming List.csv in following format:
Location,IP Address,Host Name,Domain,Domain Name, User Name,Manufacturer,Model,System Type, Serial Number, Operating System,RAM ...
3
votes
1answer
28 views
update line based on previous line
I have a issue since I'm not familiar with awk. I have csv file generated from sar -d output converted to csv style:
12:33:41 unix,restarts
12:35:00,lofi4096,0,0.0,0,0,0.0,0.0
...
1
vote
2answers
105 views
Find the maximum length of characters in a field
Can you tell me how to find the maximum length of characters in a given field?
For instance, with this input file:
s.no,name
1,ATM
35,money
63,back
The maximum length in field 1 is 2. So, I want ...
2
votes
1answer
86 views
solaris awk problems with date and if statements
I have daily files that come in via FTP with incorrect dates in the first column of the file. I have figured out how to deduct one day to derive the correct date and print this to a new file. The main ...
1
vote
5answers
68 views
Replicate a column in a one-line CSV file
I have a file that consists of a single line:
a,x,b,c,d,e
I want to convert this into
a,x,b,x,c,x,d,x,e,x
Which is the easiest way to achieve this? A solution based on Python will be most ...
0
votes
1answer
75 views
Removal of extra spaces but not the newline from csv files
I have to compare 2 files by using comm command like the below
comm -23 csv1 csv2 > csv3
I seem to face the problem that both the csv are generated from different sources (csv1 is extracted ...
6
votes
3answers
184 views
adding an empty first line inside csv file
I input data into a csv by 'echo'-ing some values into another csv.
I tried echo "" and in the next line echo "values" but i get intermittent empty lines..
I need empty line only at the start....
2
votes
2answers
130 views
Create an ASCII art table from tabular data
Given perhaps comma- or tab-delimited input, I'd like to present a series of appropriately padded columns to stdout, so I can easily scan columnar information which would otherwise present rather ...
0
votes
4answers
134 views
Removing certain fields from a CSV file
I have a large CSV file (~15GB) which a client has sent over as an export of a table. The format is a follows:
field, ... , field, field which sometimes contains commas,
another field which sometimes ...
2
votes
3answers
56 views
Extract data from a file and place in different files based on1 column value
We will generate a csv file with below values
yp1234,577,1,3
yp5678,577,3,5
yp9012,132,8,9
I need to extract data and create files based on second column. If it's 577 then the whole line has to be ...
4
votes
2answers
51 views
awk pairwise differences
I run into this problem a lot with data, if I have some integers, I frequently want to look at differences between adjacent integers, I usually solve this in Ruby or Python, but I think such a thing ...
2
votes
3answers
57 views
I need data from a csv file which is not present in the second csv file
For example: I have a CSV file (first file ) with below data (vertically):
1
2
3
4
5
6
Second CSV with below data (vertically):
5
6
7
8
I want my resultant file (action needs to be performed ...
2
votes
3answers
116 views
Select lines from text file which have ids listed in another file
I use a lot of grep awk sort in my unix shell to work with medium-sized (around 10M-100M lines) tab-separated column text files. In this respect unix shell is my spreadsheet.
But I have one huge ...
1
vote
1answer
60 views
removing data from txt file
I have a comma-delimited giant file that was exported from a database. The file has this format
Date,n1,n2,n3,n4,n5,n6,color,type,reference
10-Jan-2000,344,13,21,2,55,3,"rose garden" ,3444,32
...
1
vote
2answers
113 views
Unable to do head or tail for a file
I have a .CSV file which upon passing the file test_file.csv command gives the output as:
test_file.csv: ISO-8859 English text, with CR line terminators
When I am using cat, head or tail command on ...
1
vote
2answers
72 views
Renaming folders based on a dictionary in form of a CSV file?
I will describe the problem quite verbose, as I have learned that only talking about a specific sub-problem I have identified leads to missed oppurtunities ... So I will put a tl;dr up front: There is ...
4
votes
5answers
1k views
extract the 4th column from a csv file using unix command
I have a csv file formatted as below.
"col1","col2","col3","col4"
"1","text1","<p>big
html
text</p>
","4th column"
"2","text2","<p>big2
html2
text2</p>
...
2
votes
2answers
782 views
Convert csv to HTML table
I have a Medical.csv file with rows of following format,
field: 'participation.type', displayName: 'program_type', type: 'String',path:'participation'
field: 'participation.program', ...
5
votes
5answers
737 views
Parsing a delimited text file in bash as command arguments
I have a text file split up like so:
field1,field2,field3
xield1,xield2,xield3
dield1,dield2,dield3
gield1,gield2,gield3
Each of these columns will be a parameter to a program, and I would like ...
2
votes
2answers
120 views
Print some fields of /etc/passwd
I have a question concerning the usage of wild cards in Linux. Here I have some lines from a long list of lines in the /etc/passwd -file.
root:x:0:0:root:/root:/bin/bash
...
0
votes
0answers
88 views
Error while Converting Excel to CSV using xls2csv
I am using xls2csv to convert a xls file into a regular .csv file. However, while using the command as
xls2csv -c\, fromfile.xls > tofile.csv
.I am getting the following error. Do not know what ...
5
votes
3answers
128 views
Robustly segmenting a csv file containing noisy data
Say I have a csv file with these columns:
timestamp,cpu,memory
An example of the shape of the actual data would be this chart:
Having the data visualized in this way, it's easy for the human eye ...
3
votes
4answers
221 views
How to Prefix a column values with an apostrophe ( ' )?
I have a CSV file with multiple columns and 1000's of records, I need to prefix all the values of one of the columns (lets say 2nd column) with an apostrophe ' except in the first line or header line, ...
5
votes
3answers
1k views
Filter a .CSV file based on the 5th column values of a file and print those records into a new file
I have a .CSV file with the below format:
"column 1","column 2","column 3","column 4","column 5","column 6","column 7","column 8","column 9","column 10
"12310","42324564756","a simple string with a , ...
2
votes
4answers
320 views
Format a Date field of a .CSV file with multiple commas in a string field
I have a .CSV file (file.csv) whose data are all enclosed in double quotes. Sample format of the file is as below:
column1,column2,column3,column4,column5,column6, column7, Column8, Column9, Column10
...
2
votes
1answer
101 views
How to extract single value from single line tuples?
I have a dump file of a database in the following form
('value1','value2',value3','value4'),('value1','value2',value3','value4'),...
I want first to have each tuple in a single line and next i ...
1
vote
1answer
164 views
How to use awk or sed to convert csv diffs into more readable format
Can anyone give me the example of how to use awk or sed (not sure which one, as I haven't use either of these much, as I have mostly been using grep and cut to work with csv data) to convert ...
6
votes
4answers
518 views
uniq a csv file ignoring a column, awk maybe?
Given this file (annotations are not part of file, but form part of explanation)...
x,a,001,b,c,d,y
x,a,002,b,c,e,yy
x,bb,003,b,d,e,y
x,c,004,b,d,e,y
x,c,005,b,d,e,y # nb - dupe of row 4
...
2
votes
2answers
411 views
How to remove an apostrophe ( ' ) from couple of columns of a .CSV file?
I have a .CSV file with 7 fields, and the 3rd and 4th columns of the file has a number starting with an apostrophe ( ' ). Please see the example below.
col0,col1,col2,col3,col4,col5,col6,
...
1
vote
2answers
70 views
Given a CSV file, how do I delete the content between the 2nd and 3rd tabs of each row? [duplicate]
I am using a mac and I have a CSV file delimited by tabs. I want to just remove all the content between the 2nd and 3rd tabs or replace it with something like "XXXX". Is there a command for this?
1
vote
1answer
177 views
how to convert multiple txt to CSV with field data separated by blank lines
I have some data in multiple text files where fields are separated by blank lines. There are only 4 fields but in the second field there are more subfields, could be three or more.
The first field is ...
0
votes
2answers
158 views
Extract given column from comma-separated values with inner commas and quotation marks [duplicate]
I have a file in which the fields are separated by comma.
Some example input:
col1,"1,2",col3
col1,"1,2,3",col3
col1,"1 2,3",col3
col1,"1 "2,3"",col3
Now, I have to fetch the second column, so ...
1
vote
5answers
314 views
delete lines in a csv file older than 7 days
I have a csv that I need to remove all lines that are older than 7 days. This is the format of the csv
Person ID VIP CS SS LT FTLT PS Modified Datestamp
T001028 1 1 1 0 0 0 ...
-1
votes
1answer
380 views
Selecting rows in a CSV file based on column value
I have a CSV file with 4 columns: Itemname,Value,Description and component which is quite huge.
I have to generate a template from the above CSV file that displays only the rows of the specified ...
2
votes
1answer
177 views
Parse csv using awk [duplicate]
Possible Duplicate:
Is there a robust command line tool for processing csv files?
I am trying to parse the second line in a file (which contains commas within double quotes)
"100 Watt ...
3
votes
0answers
229 views
Catdoc xls2csv converting only one sheet
I have Catdoc's xls2csv installed on Debian/Squeeze. It used to work beautifully when I copied files to a folder like /var/www/xyz, converting all sheets into .csv format, separated by ^L.
But now, ...
2
votes
2answers
69 views
How can I show the number of positions lines have moved in diff output?
top1
1,facebook.com
2,google.com
3,youtube.com
4,yahoo.com
5,wikipedia.org
6,baidu.com
7,live.com
8,amazon.com
9,qq.com
10,twitter.com
top2
1,facebook.com
2,google.com
3,youtube.com
4,yahoo.com
...
2
votes
2answers
954 views
Data pivot with awk
I'm trying to pivot a file using awk. This is an example of my input File:
VarName;TimeString;VarValue;Validity;Time_ms
A;23.11.201215:03:53;1;1;41236627696,7593
...
3
votes
2answers
2k views
Is there a command line utility to transpose a csv-file?
Given a file like so
First, Last, Age
Cory, Klein, 27
John Jacob, Smith, 30
Is there a command line utility to transpose the contents so the output appears like so
First, Cory, John Jacob
Last, ...
1
vote
4answers
905 views
How to rename files with sed and csv
I'm new with Linux, sed, and awk but I don't mind challenging myself on new ideas. That being said, I understand the purpose and how to use rename and sed for a common event such as adding a $date or ...
4
votes
4answers
906 views
A unix command to truncate each line of a file
I have a CSV file and I want to truncate it from the third semicolon. For example, if I have this file:
1;foo;bar;baz;x;y;z
2;foo;bar;baz;x;y;z
3;foo;bar;baz;x;y;z
I want to get the following ...
2
votes
3answers
3k views
Extracting column from comma separated text
I have a long comma-separated delimited file with 20K lines. Here's a sample:
"","id","number1","number2","number3","number4","number5","number6","number7"
...
8
votes
5answers
6k views
Remove comma between the quotes only in a comma delimited file
I have a input file delimited with commas (,). There are some fields enclosed in double quotes that are having a comma in them. Here is the sample row
123,"ABC, DEV 23",345,534.202,NAME
I need to ...
2
votes
1answer
106 views
Read only new parts of a file?
I have a large CSV file, that grows by some entries every day (created by an external software which writes always to this one file, no splitting possible). For performance reasons and further ...
2
votes
1answer
439 views
Regarding generating intersection and union of two csv files
I have two csv files, there are some overlap columns between these two files. Assume one file is called as A.csv and another is called as B.csv. The intersection of A and B is called as C.
I would ...
3
votes
3answers
513 views
Using CSV line as command parameters
I have a CSV file like:
Name,Age,Address
Daniel Dvorkin,28,Some Address St. 1234
(... N ...)
Foo Bar,90,Other Address Av. 3210
And I have a command that take this parameters:
./mycommand ...
2
votes
3answers
514 views
How can I turn ugly output into pretty, useful information?
How can I turn this ugly output into pretty, useful data?
The output:
/* ---------- TA#box#AbC_p ---------- */
insert_job: TA#box#AbC_p job_type: a
#owner: bob
permission: gx
date_conditions: 1
...
2
votes
1answer
2k views
Swap two columns in a CSV using SED
I have a CSV file that contains 10 different fields (, is the deliminator). Example data:
student-id,last,first,hwk1,hwk2,hwk3,exam1,hwk4,hwk5,exam2
pts-avail,,,100,150,100,200,150,100,300
...
5
votes
3answers
4k views
Search for values in 2 columns from CSV
I have a csv files with say 20 odd columns.
I need to get the 14th and 15th column values where value in 14th column is say "VALUE1" and value in 15th column is "VALUE2".
My condition gets satisfied ...
2
votes
6answers
3k views
How to swap columns in such a file?
I have a text file, each line is stored like this :
"Video or movie" "parent" "Media or entertainment" "1" "1" "1" "0" "0"
I want to swap the columns 3 with 2, i.e.
"Video or movie" ...
