All Questions
35
questions
-3votes
0answers
20views
How can i create excel pivot table through python pandas [closed]
I have one excel(data.xls) which has some user details and billing minutes, need python script for pivot table and pivot table should be in same excel sheet as Sheet number 2.
I want pivot table of ...
- 33
1vote
2answers
83views
How to store .txt file data in different columns of a CSV file using BASH?
I have a .txt file with the following data structure:
Scan Times:
33.3 seconds
77.4 seconds
33.3 seconds
77.4 seconds
Check Times:
110.30 seconds
72.99 seconds
72.16 seconds
110.30 seconds
...
0votes
1answer
146views
Reading in a CSV file using while IFS= but not treating the comma as the separator if the field is in quotations
I have a large CSV file where columns are separated by commas in each case. However, about 5% of the time the first column contains a comma within itself and is denoted this way by the fact that these ...
- 41
-1votes
3answers
78views
Store terminal output in HTML file
I have a CSV file that I'm counting the occurrences of column 4:
Input:
2,Serveis,14,"Restaurants, bars i hotels (Inclos hostals, pensions i fondes)",Restaurants,1
2,Serveis,11,"...
- 145
2votes
2answers
135views
awk extract columns that meet condition
I'm writing a script to prepare a csv file that takes columns number 5, 6, 7, 8, 10 and 13; takes the rows that on column 44 are equal to 7 and also meet that the rows that on column 3 are equal to 1, ...
- 145
7votes
2answers
1kviews
awk script to prepare csv file
I'm stuck creating an awk script that prepares a csv file before analysis. I need to create an output file with columns 1-2, 10, 13-15, 19-21. Also I need to replace the numbers on column 2 to the ...
- 145
0votes
1answer
95views
Filter a CSV file to remove if Field 1 matches and the date/time in field 3 is less than 5 mins from the first field-1 match
Using a bash shell script on Ubuntu on a Raspberry Pi, I am trying to remove lines from a (comma separated) CSV list where { Field 1 matches AND Field 3 is less than 5 mins (300 secs) from the first ...
- 3
1vote
4answers
2kviews
Bash script/awk to input and output a CSV file
I'm having some trouble creating a bash/awk/sed script that would take a comma-separated CSV file of three columns (firstname, lastname, date of birth), and outputs another CSV file that has the same ...
- 25
0votes
2answers
78views
Unable to separate valid and invalid file in unix script topic
I have three files named as a.csv, b.csv, and c.csv.
a.csv has content
1234567,11111111111111111111111111111111111111111111111111
b.csv has content like
...
-1votes
4answers
93views
Compare two files, if the columns are same then replace next column with some value
How do I replace a column value in a file when compared with another file?
I have two files test1.csv and test2.csv; I need to replace the empdep column in test1.csv if it has the value as "sign*". ...
- 25
1vote
2answers
64views
Remove duplicates csv based on first value keeping the longest line between duplicates
I have many different csv files in a folder (megadrive.txt, snes.txt) like this:
Aerial Assault (USA);Aerial Assault (USA);Sega Master System;;;;;;;;;0;;;;;
Aerial Assault (USA);Aerial Assault (USA);...
- 19
-2votes
4answers
261views
shell: map columns of a TSV file using a separately defined mapping
Suppose I have the following tab-separated file data.tsv:
1 a 3
2 b 2
3 a 10
2 c 2
2 a 8
Now, I want to exchange the values in column 1 - say, turn them from IDs to names - by using the following ...
- 125
0votes
0answers
121views
Json to Csv with JQ [duplicate]
I want to convert a json file to csv file.
I have a script:
cat convert.sh
INPUTFILE="test.json"
OUTPUTFILE="outputfile-formatted.csv"
jq -r '(.[0] | keys_unsorted) as $keys | $keys, map([.[ $keys[]...
- 1
-1votes
1answer
267views
Check if a two columns are completely numeric in a csv using awk
I have a csv with columns A & B as below which has numeric or alphanumeric data.I need to write column C such that if,
A or B is blank -> write "Error" in C
A or B not numeric -> write "Error"...
3votes
2answers
338views
Parsing csv file to filter rows basing on a matching set of characters from a column value
Consider the following csv file:
A,3300
B,8440
B,8443
B,8444
C,304
C,404
M,5502
M,5511
The real csv file is really big (around 60,000 rows) but I only included a small version for ...
- 31
3votes
3answers
135views
Text Manipulation - Transpose column into rows depending on the value
I want to write a code to be able to manipulate text on my CSV file which contains the following:
71w - Rus,51200
71w - Phi,307200
71w - Ukr,307200
71w - Ukr,51200
71w - Mic,102400
71w - Mic,51200
...
0votes
1answer
466views
How to write a csvcut script to cut column by header with multiple files?
Since csvcut (from csvkit) does not take more than a single file at a time, I need to write a script to process multiple files using it.
The first parameter should be the delimiter, the second ...
- 75
1vote
2answers
349views
Script to combine separate csv files
I have two separate files containing information on clients that I would like to merge into one. The first file contains global information, the second is for options.
Basically, every client has an ...
- 159
1vote
1answer
64views
How to extract data from csv file with modification to data
For eg i have a file called test.csv containing
emp1,dep1
emp2,dep2
emp3,dep1
emp4,dep3
emp5,dep2
expected output is
emp1 works in dep1
emp2 works in dep2
.
.
emp5 works in dep2
- 11
0votes
2answers
697views
Split into file with first column Name
I have this CSV file:
"Product ID";"Product Name";"Price";"Description";
"11;"Example";"200";"Descripcion here...";
"21;"Example2";"300";"Some here...";
I want to split it line by line and save them ...
- 25
0votes
1answer
259views
.sh script for counting the number of names in a category in a .csv
I would like to count the number of students in a .csv file depending on the category
Category 1 is the name, Category 2 is the country, Category 3 is the city
The .csv file is displayed as such :
...
- 1
-1votes
1answer
103views
Data Comparison in 2 csv files columnwise, Input from 3rd file
I have 3 files:
In file1 I've data like this
A,B,C,D,E,F,G
1,2,3,4,5,6,7
7,6,5,4,3,2,1
In file2 I've data like this
A,B,C,D,E,F,G
1,2,3,4,5,5,7
7,6,6,4,3,2,1
In file3 I've only one field i.e A:
...
- 29
1vote
1answer
604views
Paste multiple large csv files with different header order
I have multiple large csv files in a folder and I am trying to rbind(concatenate) them into one csv. But, while doing this I want to make sure all column values go in an appropriate column after ...
- 111
1vote
1answer
1kviews
BASH | CSV formatted report
I need to create a report of all files renamed with the ff fields in the CSV formatted report (Output.rpt).I need an output that displays
a. Column1 = Original Filename
b. Column2 = Original ...
- 135
2votes
1answer
3kviews
Export netstat data from terminal to csv file
having a bit of an issue, i am looking to create a script which runs the following command every 60 seconds
netstat - tn
which should give an output like
root@guest:~# netstat -tn
Active Internet ...
- 21
3votes
2answers
55views
create text reading data from a file
I have a csv file which has a list of 600 numbers in the following format
442030580306
442030580307
I need it to output the file in one long file like the below from reading the numbers above
...
- 55
1vote
1answer
66views
repeated value in .csv file
everybody, I've got this script that I've used without any problem until now, it uses two files in order to create a .csv, those two files came from another script and it contains information about ...
-1votes
2answers
44kviews
generate a csv file
I have a script that generates a .txt file.
That file have tab-delimited columns.
The number of columns vary depending on the input file.
How can I convert that .txt file to csv format? I want to ...
- 21
1vote
4answers
820views
Can I use variable in a .csv file in the for loop?
I am trying to replace these numbers "A00002 X53307 BB145968 CAA42669 V00181 AH002406 HQ844023" in the following for loop with a new list of numbers. But my new list is a .CSV file and there are ...
- 35
0votes
1answer
440views
Parsing a delimited in ksh as command argument
My system generates a flat file with 7 arguments :
Field1,Field2,Field3,Field4,Field5,Field6,Field7
Field1,Field2,Field3,Field4,Field5,Field6,Field7
Field1,Field2,Field3,Field4,Field5,Field6,Field7
...
- 1
2votes
6answers
3kviews
Scripting to split a single CSV row into multiple
I need a script that can take a CSV file with that has a column of semicolon-delimited attributes, and break it into multiple rows to normalize that multi-valued attribute? Here is the scenario:
...
- 23
-5votes
1answer
2kviews
Merge two csvfiles by column header
I have two csv files:
success.csv
"ID","Legacy ID","Field1", "Field2"
1,1111,google,news
2,2222,yahoo,news
error.csv
"Legacy ID","Field1","Field2","Message"
3333,aol,news,"Failed to upload data" ...
- 9
10votes
5answers
75kviews
Parsing a delimited text file in bash as command arguments
I have a text file split up like so:
field1,field2,field3
xield1,xield2,xield3
dield1,dield2,dield3
gield1,gield2,gield3
Each of these columns will be a parameter to a program, and I would like ...
- 101
4votes
3answers
35kviews
How to iterate through a comma-separated list and execute a command for each entry
I have been give a command which outputs a comma-separated list for autosys jobs in the variable $all_jobs:
box=box-of-jobs;all_jobs=$(jobscout -box $box | egrep "^\w+" | tr '\n' ',' | sed s/.$//);
...
- 53
4votes
1answer
1kviews
An efficient filter for .sql files with just "INSERT INTO"
Several times I've found myself working on data that has been exported to .sql files that I want to process rather than import into a database.
I'm only talking about "pure data" sql files. Ones that ...
- 326