All Questions
Tagged with csv performance
120 questions
3
votes
3
answers
133
views
Increase time efficiency when writing arrays to CSV file
I have the following code to amend two rows of "test_base.csv" with the entries of the arrays "a_temp" and "b_temp," saving the result into "result.csv." .csv ...
- 33
8
votes
5
answers
837
views
adding data to a CSV file for it to be read
I'm making a program that lets you enter a name and house that adds it to the CSV file for it to be read and print out "Tre is in house Dragon", etc. The code works; I'm just wondering if ...
- 121
3
votes
1
answer
120
views
CSV Data Plotting Program using CsvHelper in C#
This is a follow-up question for CSV Data Plotting Program in C#. I am trying to use CsvHelper to build an application for plotting CSV Data in this post. In the ...
- 7,224
2
votes
2
answers
278
views
CSV Data Plotting Program in C#
I am trying to implement a CSV data plotting program in C#. In the plotting part, I use ScottPlot.NET to perform plotting operation. There are five column in the given CSV file, therefore, I use ...
- 7,224
1
vote
1
answer
73
views
Insert huge data from csv file in .Net framework 4.7.2
I need to insert about 500k - 1 million data lines from a CSV file. The time of getting the reports list and insert to database by using SqlBulkCopy is very good (...
- 11
1
vote
1
answer
163
views
Process comma separated input and check format
I have to read console input and store data in the vector of structs. In case of any data format violation I have to print "Malformed Input" and return <...
- 459
7
votes
4
answers
1k
views
Calculate average values for each day of the week for each Meter
I have a program with these two methods. One method to import a set of data from a CSV within the given time period and store them in a dictionary. Here the data in CSV file is stored in following ...
6
votes
2
answers
1k
views
Read a large CSV order book
I am trying to read entries of my CSV file into an entries vector whose data type is object OrderBookEntry, which is an object that stores the tokenized variables ...
0
votes
1
answer
160
views
Compare hostnames and leases between CSV and DHCP info
I have 3 nested loops and they do the following:
Get scopes from DHCP
Get hostnames from csv file
Compares hostnames and leases, matches are added to a hash table
There are 5 DHCP servers, 100's of ...
- 11
4
votes
2
answers
445
views
Read a csv file and insert into DB table
I've found some resources about PHP performance optimization here and there but I don't know how to start. I send 130k CSV rows in approximatively 7min. My script read CSV file and send ...
4
votes
1
answer
286
views
Reorder the Columns in a CSV File in Descending Order
I wrote a script to reorder the columns in a CSV file in descending order and then write to another CSV file. My script needs to be able to handle several tens of millions of records, and I would like ...
3
votes
1
answer
103
views
Calculating the total daily amount of confirmed cases of Coronavirus
I'm writing a small program to plot new COVID-19 infections. As of right now, I have it so the program reads the given data file, pulls out the daily cases and dates for each country, and adds ...
- 31
3
votes
3
answers
2k
views
Efficiently convert 60 GB JSON file to a csv file
Description
Simply take a JSON file as input and convert the data in it into a CSV file. I won't describe the functionality in too much detail since I have reasonable docstrings for that. As you can ...
- 9,316
9
votes
3
answers
619
views
Performance - Read large amount of XMLs and load into single csv
I am dealing with a large amount of XML files which I obtained from here https://clinicaltrials.gov/ct2/resources/download#DownloadAllData. The download yields around 300.000 XML files of similar ...
10
votes
2
answers
2k
views
Are there ways to speed up this string fuzzy matching in Golang?
I have a piece of python code doing fuzzy matching which works very well and is pretty fast. For reference, it uses the following files:
https://raw.githubusercontent.com/datasets/s-and-p-500-...
- 243