Comma-separated values: files containing data arrange in a table, with commas to separate columns
0
votes
0answers
20 views
combining files of different lengths into csv?
How do I combine multiple files ( of different length ) into one / a csv?
I've used paste and column, but it merges the latter (which has the most) left to fill the empty rows.
I have 10 files, with ...
-2
votes
0answers
18 views
using mysqlimport on a large amount of csv files in a directory via bash scripting [on hold]
I have a directory with approximately 75 csv files in it that I need to mysqlimport into a MySQL database. The MySQL database does not have the tables created.
The files:
* The data is all numbers
...
1
vote
0answers
50 views
How to create CSV file without unnecessary spaces
I use the xls2csv binary in order to translate XLS doc to CSV in my linux red-hat machine,
example: ( from man page )
xls2csv -x "1252spreadsheet.xls" -b WINDOWS-1252 -c "ut8csvfile.csv" -a UTF-8
...
2
votes
2answers
77 views
CSV file + set CSV parameters and values in bash script
I have the following CSV file:
Alabama,Alaska,Arizona,Arkansas,California,Colorado,Connecticut,Delaware,Florida,Georgia,Hawaii,Idaho,Illinois,Indiana,Iowa
1000,"1 0 0 1",1002,1002,1003,1004,1005,"1 0 ...
1
vote
2answers
60 views
merge csv files, escape comma inside quotes
I have 3 csv files I want to join by first column (id column)
Each file has the same 3 columns
Row example :
id | timestamp | Name
3792318, 2014-07-15 00:00:00, "A, B"
When I join ...
1
vote
2answers
48 views
Merge csv files with conditions
I have 2 CSV files for each date (csv_2014_4_15 and csv_2014_4_16) , with basic structure and couple of unique columns as below.
id,name,created_at,updated_at,other columns
12, joe, 2013-1-1 18:30, ...
2
votes
1answer
65 views
merge csv files by first column
I have 3 csv files like this.
csv 1:
1,aaaa,bbb,2014-04-01
2,qwe,rty,2014-04-03
3,zxc,cvb,2014-04-05
csv 2:
2,j,k,2014-04-01
3,a,s,2014-04-04
5,g,h,2014-04-08
csv 3:
2,a,s,d,f,g,2014-04-01
...
2
votes
3answers
55 views
Concatenate filename inside of CSV file for each line for multiple CSV files
I have lots of CSV files and I need the filename of each specific CSV file in every line of each file.
Original file content of abc123.csv:
ColVal_1;ColVal_2;ColVal_3
ColVal_4;ColVal_5;ColVal_6
...
0
votes
3answers
48 views
how to export all numbers that are unique in few text files into another file?
I have a.txt,b.txt,c.txt. Each has different numbers as below:
a.txt:
12
14
111
1
15
2
b.txt
12
18
22
23
1
2
c.txt
12
14
15
16
17
1200
The output should contains all the numbers from ...
0
votes
2answers
27 views
Trim last chars from a line
I have a csv file and the lines have different length. But all of them ends with a number surrounded by a pair of braces. For example,
........{5}
........{16}
I'd like to remove the number and the ...
1
vote
2answers
71 views
Shell script to Capture the file name and size in csv file and add header on that file
I have a number of zip files in a certain folder. I want to capture file name and the size of the files in a CSV format in day wise and put it in crontab so I can report on a daily basis, and then ...
-3
votes
2answers
114 views
Shell Script to capture file name and file size in CSV format [closed]
I have number of zip files in a certain folder.
I want to capture file name and the size of the file in a csv format containing file names and sizes, one file per day. The file names do contain the ...
2
votes
3answers
146 views
replace column of a csv file with column from another file
I have two sample files like this:
$ cat file1
abc,sachin
cat,kumar
$ cat file2
xyz
pressure
$ cat file3
xyz,sachin
pressure, kumar
I want the first column of file1 to be replaced with file2.
...
6
votes
3answers
381 views
Remove only the commas present within the double quotes
In a text file, I want to remove , (commas) and also the " (quotes) (only if the double quotes contains numbers seperated by commas).
56,72,"12,34,54",x,y,"foo,a,b,bar"
Expected ouput
...
11
votes
8answers
727 views
How do I get the character count of words in a particular column?
I have a CSV file like this:
abd,123,egypt,78
cde,456,england,45
How can I get the character count of only the 3rd column words?
I can't figure out how to get wc to do this.
2
votes
3answers
801 views
Replacing the values in one file with the values in another file in bash
I am having a csv file naming List.csv in following format:
Location,IP Address,Host Name,Domain,Domain Name, User Name,Manufacturer,Model,System Type, Serial Number, Operating System,RAM ...
3
votes
1answer
35 views
update line based on previous line
I have a issue since I'm not familiar with awk. I have csv file generated from sar -d output converted to csv style:
12:33:41 unix,restarts
12:35:00,lofi4096,0,0.0,0,0,0.0,0.0
...
1
vote
2answers
400 views
Find the maximum length of characters in a field
Can you tell me how to find the maximum length of characters in a given field?
For instance, with this input file:
s.no,name
1,ATM
35,money
63,back
The maximum length in field 1 is 2. So, I want ...
2
votes
1answer
192 views
solaris awk problems with date and if statements
I have daily files that come in via FTP with incorrect dates in the first column of the file. I have figured out how to deduct one day to derive the correct date and print this to a new file. The main ...
1
vote
5answers
73 views
Replicate a column in a one-line CSV file
I have a file that consists of a single line:
a,x,b,c,d,e
I want to convert this into
a,x,b,x,c,x,d,x,e,x
Which is the easiest way to achieve this? A solution based on Python will be most ...
0
votes
1answer
346 views
Removal of extra spaces but not the newline from csv files
I have to compare 2 files by using comm command like the below
comm -23 csv1 csv2 > csv3
I seem to face the problem that both the csv are generated from different sources (csv1 is extracted ...
6
votes
3answers
276 views
adding an empty first line inside csv file
I input data into a csv by 'echo'-ing some values into another csv.
I tried echo "" and in the next line echo "values" but i get intermittent empty lines..
I need empty line only at the start....
2
votes
2answers
160 views
Create an ASCII art table from tabular data
Given perhaps comma- or tab-delimited input, I'd like to present a series of appropriately padded columns to stdout, so I can easily scan columnar information which would otherwise present rather ...
0
votes
4answers
470 views
Removing certain fields from a CSV file
I have a large CSV file (~15GB) which a client has sent over as an export of a table. The format is a follows:
field, ... , field, field which sometimes contains commas,
another field which sometimes ...
2
votes
3answers
129 views
Extract data from a file and place in different files based on1 column value
We will generate a csv file with below values
yp1234,577,1,3
yp5678,577,3,5
yp9012,132,8,9
I need to extract data and create files based on second column. If it's 577 then the whole line has to be ...
4
votes
2answers
64 views
awk pairwise differences
I run into this problem a lot with data, if I have some integers, I frequently want to look at differences between adjacent integers, I usually solve this in Ruby or Python, but I think such a thing ...
2
votes
3answers
170 views
I need data from a csv file which is not present in the second csv file
For example: I have a CSV file (first file ) with below data (vertically):
1
2
3
4
5
6
Second CSV with below data (vertically):
5
6
7
8
I want my resultant file (action needs to be performed ...
2
votes
3answers
530 views
Select lines from text file which have ids listed in another file
I use a lot of grep awk sort in my unix shell to work with medium-sized (around 10M-100M lines) tab-separated column text files. In this respect unix shell is my spreadsheet.
But I have one huge ...
1
vote
1answer
75 views
removing data from txt file
I have a comma-delimited giant file that was exported from a database. The file has this format
Date,n1,n2,n3,n4,n5,n6,color,type,reference
10-Jan-2000,344,13,21,2,55,3,"rose garden" ,3444,32
...
1
vote
2answers
154 views
Unable to do head or tail for a file
I have a .CSV file which upon passing the file test_file.csv command gives the output as:
test_file.csv: ISO-8859 English text, with CR line terminators
When I am using cat, head or tail command on ...
1
vote
2answers
111 views
Renaming folders based on a dictionary in form of a CSV file?
I will describe the problem quite verbose, as I have learned that only talking about a specific sub-problem I have identified leads to missed oppurtunities ... So I will put a tl;dr up front: There is ...
4
votes
5answers
4k views
extract the 4th column from a csv file using unix command
I have a csv file formatted as below.
"col1","col2","col3","col4"
"1","text1","<p>big
html
text</p>
","4th column"
"2","text2","<p>big2
html2
text2</p>
...
2
votes
2answers
2k views
Convert csv to HTML table
I have a Medical.csv file with rows of following format,
field: 'participation.type', displayName: 'program_type', type: 'String',path:'participation'
field: 'participation.program', ...
5
votes
5answers
3k views
Parsing a delimited text file in bash as command arguments
I have a text file split up like so:
field1,field2,field3
xield1,xield2,xield3
dield1,dield2,dield3
gield1,gield2,gield3
Each of these columns will be a parameter to a program, and I would like ...
2
votes
2answers
188 views
Print some fields of /etc/passwd
I have a question concerning the usage of wild cards in Linux. Here I have some lines from a long list of lines in the /etc/passwd -file.
root:x:0:0:root:/root:/bin/bash
...
0
votes
0answers
100 views
Error while Converting Excel to CSV using xls2csv
I am using xls2csv to convert a xls file into a regular .csv file. However, while using the command as
xls2csv -c\, fromfile.xls > tofile.csv
.I am getting the following error. Do not know what ...
5
votes
3answers
138 views
Robustly segmenting a csv file containing noisy data
Say I have a csv file with these columns:
timestamp,cpu,memory
An example of the shape of the actual data would be this chart:
Having the data visualized in this way, it's easy for the human eye ...
3
votes
4answers
325 views
How to Prefix a column values with an apostrophe ( ' )?
I have a CSV file with multiple columns and 1000's of records, I need to prefix all the values of one of the columns (lets say 2nd column) with an apostrophe ' except in the first line or header line, ...
4
votes
3answers
3k views
Filter a .CSV file based on the 5th column values of a file and print those records into a new file
I have a .CSV file with the below format:
"column 1","column 2","column 3","column 4","column 5","column 6","column 7","column 8","column 9","column 10
"12310","42324564756","a simple string with a , ...
2
votes
4answers
459 views
Format a Date field of a .CSV file with multiple commas in a string field
I have a .CSV file (file.csv) whose data are all enclosed in double quotes. Sample format of the file is as below:
column1,column2,column3,column4,column5,column6, column7, Column8, Column9, Column10
...
2
votes
1answer
131 views
How to extract single value from single line tuples?
I have a dump file of a database in the following form
('value1','value2',value3','value4'),('value1','value2',value3','value4'),...
I want first to have each tuple in a single line and next i ...
1
vote
1answer
196 views
How to use awk or sed to convert csv diffs into more readable format
Can anyone give me the example of how to use awk or sed (not sure which one, as I haven't use either of these much, as I have mostly been using grep and cut to work with csv data) to convert ...
6
votes
4answers
670 views
uniq a csv file ignoring a column, awk maybe?
Given this file (annotations are not part of file, but form part of explanation)...
x,a,001,b,c,d,y
x,a,002,b,c,e,yy
x,bb,003,b,d,e,y
x,c,004,b,d,e,y
x,c,005,b,d,e,y # nb - dupe of row 4
...
2
votes
2answers
706 views
How to remove an apostrophe ( ' ) from couple of columns of a .CSV file?
I have a .CSV file with 7 fields, and the 3rd and 4th columns of the file has a number starting with an apostrophe ( ' ). Please see the example below.
col0,col1,col2,col3,col4,col5,col6,
...
1
vote
2answers
79 views
Given a CSV file, how do I delete the content between the 2nd and 3rd tabs of each row? [duplicate]
I am using a mac and I have a CSV file delimited by tabs. I want to just remove all the content between the 2nd and 3rd tabs or replace it with something like "XXXX". Is there a command for this?
2
votes
2answers
318 views
how to convert multiple txt to CSV with field data separated by blank lines
I have some data in multiple text files where fields are separated by blank lines. There are only 4 fields but in the second field there are more subfields, could be three or more.
The first field is ...
0
votes
2answers
167 views
Extract given column from comma-separated values with inner commas and quotation marks [duplicate]
I have a file in which the fields are separated by comma.
Some example input:
col1,"1,2",col3
col1,"1,2,3",col3
col1,"1 2,3",col3
col1,"1 "2,3"",col3
Now, I have to fetch the second column, so ...
1
vote
5answers
379 views
delete lines in a csv file older than 7 days
I have a csv that I need to remove all lines that are older than 7 days. This is the format of the csv
Person ID VIP CS SS LT FTLT PS Modified Datestamp
T001028 1 1 1 0 0 0 ...
-1
votes
1answer
645 views
Selecting rows in a CSV file based on column value
I have a CSV file with 4 columns: Itemname,Value,Description and component which is quite huge.
I have to generate a template from the above CSV file that displays only the rows of the specified ...
2
votes
1answer
189 views
Parse csv using awk [duplicate]
Possible Duplicate:
Is there a robust command line tool for processing csv files?
I am trying to parse the second line in a file (which contains commas within double quotes)
"100 Watt ...
