Files containing data arrange in a table, often with commas (hence Comma Separated Values), to separate columns. Rows are separated by newlines (but not all newlines are row separators as fields can be quoted to contain the separator newlines. Use this tag for full-fledged CSV data not the simpler ...
4
votes
4answers
115 views
Compare two files with four columns
I have two csv files with some log information.
I need to compare line to line the field1 (file1) with field2 (file2) with a range of 1 minute (-00:00:01 < time < 00:00:01) and the field5 ...
0
votes
2answers
43 views
Delete customers domain extension only (in .csv file)
I have a list of my customers emails and want to remove some that end in .br for example. I would normally do the following command:
sed -i '/.br/d' customers.csv
But that would also delete a ...
0
votes
0answers
36 views
Code for creating charts from csv data in shell - pure scripting
I am running mysql queries and put the data in csv file and mailing the same using shell script.[This is an automated script] I want to create a chart out of the csv data and put it in the mail body ...
0
votes
1answer
53 views
Creating charts from the data extracted from csv file and mail the chart as content in the mail body [closed]
I running mysql queries and put the data in csv file and mailing the same using shell script.[This is an automated script]
I want to create a chart out of the csv data and put it in the mail body and ...
1
vote
1answer
30 views
How to filter a basic csv by columns
How do I filter by third column of the following file
list.txt [[ list of people ]]
billy baxter - @baxter - [email protected]
james woods - @woods - [email protected]
I currently have
cat ...
0
votes
1answer
84 views
need help merging two Unix files
I have 2 pipe delimited files say file1 and file2.
Where file1 has say 33 columns and file2 may have 34/35/36 columns. So let's not consider how mant columns we have.
What I want to do is compare the ...
0
votes
0answers
26 views
How to create a Kate highlight syntax to CSV files
I'm working with CSV files and I'm using vim because of the rainbow csv plugin that makes the editing easier:
http://www.vim.org/scripts/script.php?script_id=4598
But I prefer using Kate and as I ...
0
votes
2answers
44 views
convert xml to csv
i need to convert a xml to csv using script. I found a solution with xmlstarlet but it is not available so i am returning to point 0. Here is my xml.
<root>
<record>
...
0
votes
1answer
20 views
Appending to a CSV file after obtaining datum from a database
I can [finally] extract data from the MySQL DB
my_database_query="SELECT * FROM my_database.database"
mysql -uUSER -p'PASSWORD' << EndOfFile
$my_database_query
EndOfFile
How can I append this ...
2
votes
1answer
48 views
Using fixed strings in regular expressions?
I am writing a bash script which accepts a list of CSV files as arguments and outputs e-mail addresses only found in the first file. To accomplish this, for each record in the first CSV file I look ...
1
vote
1answer
44 views
Switch Columns in .csv files so that they are all the same
I have 20 excel files having 6x6 rows and columns, where the first row and column are string headers. Each of these files has 4 same columns and row headers except 1 different. I want to know how I ...
0
votes
0answers
17 views
Any application scans for wifi connected devices with the RSSI and saves as csv?
I'm working on a project at which people are to be identified in a closed area through their wifi connected devices. I accordingly need to find an application, code, or command that helps me scan for ...
1
vote
1answer
39 views
Convert Tab Separated csv using unoconv
I am attempting to convert a tab separated csv into an xls file using unoconv. However, the tabs are not being recognized as the field separator. Can someone please tell me how this can be done.
I've ...
2
votes
1answer
28 views
Match and delete lines with ~100 strings
I have a list of roughly 100 entries to be deleted from a csv-delimited file. They are already in another text file called 'tbd.txt;
My first thought is to write a bash for loop around 'sed -i' but ...
0
votes
2answers
76 views
I want to print the value from csv file dynamically
I want to print the values from properties file dynamically I am using below code but getting wrong output.
testProperty.properties
edmcws,edmbws,edmwt
Code
...
2
votes
6answers
147 views
Scripting to split a single CSV row into multiple
I need a script that can take a CSV file with that has a column of semicolon-delimited attributes, and break it into multiple rows to normalize that multi-valued attribute? Here is the scenario:
...
0
votes
0answers
13 views
how to keep the spaces when inserting line to parameter [duplicate]
When I want to see for example the line number 2 , from the CSV file I get the following line
example1
LINE_NUM=2
# sed -n "${LINE_NUM}p" file.csv
cwdfwefwe,"aaaaa ","\dsw\dawd\aw ...
-3
votes
1answer
35 views
Empty columns which were seen before
I have a large CSV file and want to empty certain columns if they were seen before.
So I have (to illustrate my problem):
Category | Subcategory
---------+------------
foo | bar
foo | bar
...
0
votes
1answer
104 views
csv file with multiple columns to zenity
How could I output the contents of a .csv file to a Zenity list. The file has multiple rows and columns and the rows are no fixed numbers. I would also like it so that when the user double clicks, on ...
-4
votes
1answer
212 views
Merge two csvfiles by column header
I have two csv files:
success.csv
"ID","Legacy ID","Field1", "Field2"
1,1111,google,news
2,2222,yahoo,news
error.csv
"Legacy ID","Field1","Field2","Message"
3333,aol,news,"Failed to upload ...
1
vote
2answers
45 views
change non-zero padded hour and minute into 24hr time
I have data that I need to change before running it through the pandas library in python.
It is currently in a format that stores midnight values as 2400 and that should be changed to 0000. The ...
0
votes
0answers
16 views
Gzip Mutliple files using single command [duplicate]
Want to Gzip my all the .csv files into a gzip file.
I am using this command.
gzip *.csv;
Problem File contains '//' in its name that's why command is giving error.
Error : - '//File.csv' is not ...
2
votes
2answers
786 views
Sorting a csv file [duplicate]
I need to sort a CSV file, but the the header row (1st row) keeps getting sorted.
This is what I'm using:
cat data1.csv | sort -t"|" -k 1 -o data1.csv
Here's a sample line:
...
0
votes
1answer
236 views
Is there a standalone tool which will write reports from CSV data files
I download CSV files (e.g. my bank account transactions) and would like to build reports (to print) from the data.
I have searched and found several report writer libraries, but all would force me to ...
-1
votes
1answer
69 views
insert comma after the last letter
I use a txt file, and I need to convert it in csv file
Saint Petersburg 0 10 0.1 - N
Moscow - 9 0 - N
Novgorod 0 7 1 30 Y
In ...
0
votes
1answer
514 views
export from MYSQL to CSV
How do I export from MYSQL to CSV in mariadb on Centos 7?
I am using:
Server version 5.5.37-MariaDB
Protocol version 10
Connection Localhost via UNIX socket
UNIX socket ...
2
votes
1answer
406 views
splitting a CSV and keeping the header without intermediate files [closed]
I am trying to split a dozen 100MB+ csv files into managable smaller files for a curl post.
I have managed to do it but with a lot of temporary files and IO. It's taking an eternity.
I am hoping ...
1
vote
1answer
1k views
Confused about how to convert XML to CSV using xmlstarlet on OS X?
I asked a question earlier and it was closed as being a duplicate: How to parse XML to CSV with a shell script?. The answer in the duplicate that was linked suggested using a tool called XMLStarlet, ...
6
votes
4answers
3k views
Using jq to extract values and format in CSV
I have the below json file:
{
"data": [
{
"displayName": "First Name",
"rank": 1,
"value": "VALUE"
},
{
"displayName": "Last Name",
"rank": 2,
...
0
votes
1answer
1k views
Save file as Tab delimited text and spreadsheet
I am using the following command line in Linux to save the content of my input file (txt file consisted of columns) as a spreadsheet:
less input_file > out_put.csv
my output file is
The ...
1
vote
1answer
434 views
How can I convert this excel file so that it is not only one row?
I've got an Excel file, shown in the picture below, and available for download here. What I need is to extract the variables under Item (Column B) and the values in column G. As a start, I tried ...
0
votes
1answer
105 views
Cut a specific part of a column from a csv file [closed]
I have to process a csv containing about 15 columns one of which contains a particular timestamp that includes milliseconds. When I go to insert this csv into a DB, I get an error because of this ...
9
votes
2answers
325 views
Reindexing a large CSV file
I went through the answers in this helpful thread, but my problem seems to be different enough that I can't think of good answer (at least with sed).
I have a large CSV file (200+ GB) with rows that ...
-1
votes
3answers
378 views
How can I select columns based on a control file with invalid column numbers?
I have to generate the file dynamically from the source file based on the below control file.
control_file.txt
1,3,5,-1,8,-1,4
The control file contains the position of columns which I require ...
2
votes
6answers
2k views
Handling comma in string values in a CSV file [duplicate]
I have a comma-separated file which has numeric and string columns. String columns are quoted and can have comma in between the quotes. How do I identify the columns with FS =","?
Sample records
...
-2
votes
2answers
651 views
Extract certain column from CSV files [closed]
I have a directory with a group of CSV files. All files have the same column headers. I need to extract values in a certain column from all files.
The common part in files names is ...
4
votes
4answers
510 views
Convert key=value blocks to CSV
I am trying to transpose a content of a file into another. I am new to shell scripting any help would be really appreciated
Input file Test.txt:
HLRSN = 3
IMSI = 404212109727229
KIVALUE = ...
0
votes
3answers
128 views
Remove all capital letters from a particular column in a CSV file
I am having a CSV file containing 3000+ lines in the following format:
R1C1, ABC123abc, R1C3
R2C1, DEFG456defg, R2C3
R3C1, HIJKL789hijkl, R3C3
I need to remove all the capital letters from the ...
3
votes
1answer
1k views
How to export data in the terminal to a file (e.g. csv file)
I am a complete beginner in using Linux and I would like to know how to export a bunch of data in the command line to a csv or spreadsheet file (or even text would work). Right now I am simply using ...
1
vote
0answers
823 views
How to create CSV file without unnecessary spaces
I use the xls2csv binary in order to translate XLS doc to CSV in my linux red-hat machine,
example: ( from man page )
xls2csv -x "1252spreadsheet.xls" -b WINDOWS-1252 -c "ut8csvfile.csv" -a UTF-8
...
2
votes
2answers
886 views
CSV file + set CSV parameters and values in bash script
I have the following CSV file:
Alabama,Alaska,Arizona,Arkansas,California,Colorado,Connecticut,Delaware,Florida,Georgia,Hawaii,Idaho,Illinois,Indiana,Iowa
1000,"1 0 0 1",1002,1002,1003,1004,1005,"1 0 ...
1
vote
2answers
359 views
merge csv files, escape comma inside quotes
I have 3 csv files I want to join by first column (id column)
Each file has the same 3 columns
Row example :
id | timestamp | Name
3792318, 2014-07-15 00:00:00, "A, B"
When I join ...
1
vote
2answers
325 views
Merge csv files with conditions
I have 2 CSV files for each date (csv_2014_4_15 and csv_2014_4_16) , with basic structure and couple of unique columns as below.
id,name,created_at,updated_at,other columns
12, joe, 2013-1-1 18:30, ...
3
votes
1answer
1k views
merge csv files by first column
I have 3 csv files like this.
csv 1:
1,aaaa,bbb,2014-04-01
2,qwe,rty,2014-04-03
3,zxc,cvb,2014-04-05
csv 2:
2,j,k,2014-04-01
3,a,s,2014-04-04
5,g,h,2014-04-08
csv 3:
2,a,s,d,f,g,2014-04-01
...
3
votes
4answers
996 views
Concatenate filename inside of CSV file for each line for multiple CSV files
I have lots of CSV files and I need the filename of each specific CSV file in every line of each file.
Original file content of abc123.csv:
ColVal_1;ColVal_2;ColVal_3
ColVal_4;ColVal_5;ColVal_6
...
0
votes
3answers
54 views
how to export all numbers that are unique in few text files into another file?
I have a.txt,b.txt,c.txt. Each has different numbers as below:
a.txt:
12
14
111
1
15
2
b.txt
12
18
22
23
1
2
c.txt
12
14
15
16
17
1200
The output should contains all the numbers from ...
0
votes
2answers
55 views
Trim last chars from a line
I have a csv file and the lines have different length. But all of them ends with a number surrounded by a pair of braces. For example,
........{5}
........{16}
I'd like to remove the number and the ...
1
vote
2answers
1k views
Shell script to Capture the file name and size in csv file and add header on that file
I have a number of zip files in a certain folder. I want to capture file name and the size of the files in a CSV format in day wise and put it in crontab so I can report on a daily basis, and then ...
-3
votes
2answers
326 views
Shell Script to capture file name and file size in CSV format [closed]
I have number of zip files in a certain folder.
I want to capture file name and the size of the file in a csv format containing file names and sizes, one file per day. The file names do contain the ...
2
votes
3answers
1k views
replace column of a csv file with column from another file
I have two sample files like this:
$ cat file1
abc,sachin
cat,kumar
$ cat file2
xyz
pressure
$ cat file3
xyz,sachin
pressure, kumar
I want the first column of file1 to be replaced with file2.
...
