A comma-separated values (CSV) file stores tabular data (numbers and text) as plain text, with rows and columns delimited by line terminators and commas, respectively. You may also use this tag for variants where fields are separated by tabs or semicolons.
2
votes
2answers
33 views
Updating tables with bulk of data through CSV
My work is pretty simple. I have to migrate a bulk of data into the specified tables, so that the data does show up at the front-end. The data is provided through the Excel sheet and I have to ...
4
votes
1answer
53 views
Fastest Way to Write Multiple CSV's
I'm writing a program to process log files. Currently, I have a script that will extract the desired information from each log and put it into a list - I have about nine different lists all ...
4
votes
1answer
60 views
Cutting strings into smaller ones based on specific criteria
So, I've got this largish (for me) script, and I want to see if anybody could tell me if there are any ways to improve it, both in terms of speed, amount of code and the quality of the code. I still ...
2
votes
0answers
22 views
Knn console application; problematic implementation
I'm working on a small command line application in c# where data is read from a CSV file along with a query and KNN is calculated for the query based on the data.
My problem isn't the algorithm at ...
2
votes
0answers
16 views
Generating two .csv files from named parameters
I wrote Clojure code which takes named params and has to generate 2 .csv files as output.
Please review it.
(ns my_cli_clojure.core
(:gen-class :main true))
(require '[clojure.data.csv :as csv]
...
6
votes
1answer
180 views
Speed up CSV reading code (vector of doubles)
I am trying to read a single-columned CSV of doubles into Java with a string header. It is 11 megabytes and takes over 15 minutes to read, which is clearly unacceptable. In R this CSV would take about ...
5
votes
1answer
61 views
Calculate query coverage from BLAST output
I have a BLAST output file and want to calculate query coverage, appending the query lengths as an additional column to the output. Let's say I have
2 7 15
f=open('file.txt', 'r')
...
1
vote
1answer
53 views
Grouping a CSV by column
I've started writing a small CSV parser class as I needed a method to group a CSV file by a given column.
class CSVParser {
private $FileName;
private $FileHandle;
private $HasHeaderRow ...
2
votes
0answers
106 views
Reading columns and rows in a .csv file
I have some data in a .csv file, which looks roughly like this:
[fragment1, peptide1, gene1, replicate1, replicate2, replicate3]
[fragment1, peptide2, gene1, replicate1, replicate2, replicate3]
...
3
votes
2answers
118 views
Reading in a file and performing string manipulation
In a question I answered I posted the following code:
#include <iostream>
#include <fstream>
#include <algorithm>
#include <sstream>
int main()
{
fstream ...
6
votes
1answer
106 views
Creating a pythonic snippet to read and clean .csv files
I am trying to import a .csv file, clean the contents, and make sure all the errors are eliminated.
IO error
file format error
data error
I am using the snippet below. Could someone help me clean ...
3
votes
1answer
61 views
How to make this csv-reading PHP code more efficient?
I have the following code that reads a .csv file that than outputs the result in the form of a HTML table:
$fhdrop = fopen("ac.csv", "r");
while (!feof($fhdrop) ) {
$at[] = fgetcsv($fhdrop, 1024);
...
3
votes
2answers
92 views
Perl Code Improvements - reading CSV file
I am very new to Perl, but quite pleased with my short introduction to it!
So, scenario is I have a number of text files (they are reports from a financial system) stored in a directory. Each report ...
1
vote
1answer
116 views
get city and state using zip code, scrape in Python
I'm looking for any and all feedback on quality, style and efficacy of the code. If there's a simple way to get the header put in, I'd love to hear that.
The code is used to take zip codes from a csv ...
5
votes
1answer
72 views
Improve my copying of a CSV file
I need to merge some large csv files but I feel that the way I am doing it is not optmized. It gets a number of csvfiles and creates one
public static void MergeFiles(IEnumerable<string> ...
1
vote
3answers
70 views
CSV file cleanup
I'm fairly new to Python and am going to be parsing hundreds of thousands of names eventually. What I wanted to do was take individual CSV files, and create new CSV files with only the info I wanted. ...
3
votes
1answer
55 views
Using a .csv file to insert data into the Django model
I have a script that uses a .csv file into insert data to the Django model. Now, I wonder if there's a better way of doing this.
with open('some/path/to/file.csv') as f:
reader = csv.reader(f, ...
16
votes
4answers
1k views
Regex to parse semicolon-delimited fields is too slow
I have a file with just 3500 lines like these:
filecontent= "13P397;Fotostuff;t;IBM;IBM lalala 123|IBM lalala 1234;28.000 things;;IBMlalala123|IBMlalala1234"
Then I want to grab every line from the ...
4
votes
1answer
250 views
Review: Converting latitude and longitude coordinates from CSV using web service
I am working on converting a mailing list that has longitude and latitude coordinates within the CSV file. This script I came up with does what I need, but this is my first real-world use of python. I ...
2
votes
1answer
132 views
Code correctness and refinement for quantile normalization
The below code is still far from feature complete, but am looking to have some of the sections critiqued to learn better idioms or adjustments (e.g. - yet to be implemented: handling of csv files with ...
0
votes
1answer
302 views
How can I iterate cell value of csv file using Python? [closed]
import csv
ifile=csv.reader(open('C:\Users\BKA4ABT\Desktop\Test_Specification\RDBI.csv','rb'),delimiter=';')
for line in ifile:
row = line
if [True for cell in row if cell=='']:
...
2
votes
2answers
276 views
Python: Open Multiple Files
I am trying to open two CSV files, one for reading and one for writing. The script should continue only if both files were successfully opened. My code seems to accomplish that but it seems like it ...
1
vote
2answers
264 views
Writing defaultdict to CSV file
I have a defaultdict being written to file like this:
writer = csv.writer(open(locationToSaveFile, 'wb+'))
for k,v in dict.iteritems():
writer.writerow([k ,v])
I then have this horribly ...
1
vote
4answers
121 views
How can I shorten and remove repetition from this Python script?
I've got a CSV that contains users and permissions in the below format, where users can have as little as one or as many as eight different permissions:
...
6
votes
1answer
314 views
Intern with no mentor struggling with refactoring and OOP
A little background
I'm an intern at a large engineering company, and I'm also the only CS major in the entire building. The people on my team don't have technical backgrounds, but hired me to start ...
2
votes
2answers
132 views
Python : Loop in a csv file with IF cases
I am new to Python and can't quite make my if-cases any shorter. Any ideas how to do that?
import csv
fileheader = csv.reader(open("test.csv"), delimiter=",")
# Defines the header of the file ...
1
vote
1answer
151 views
Ruby CSV reading performance problems
I (ruby beginner) have a performance issue with the following ruby code. The idea is to aggregate a CSV into a smaller, better-to-handle file and calculating the average of some values. Basically the ...
3
votes
1answer
110 views
Is there a better or more elegant way to code this CSV cell-parser (using CSVReader)?
I'm currently doing my first internship and had to create an application that will check through the first row of a CSV for valid or invalid input. Is there a more elegant way to code or to refactor ...
1
vote
2answers
233 views
Custom template iterator on CSV reading class
I made myself this code to read CSV files as I want to delve into machine learning. I do think that my code design is poor but I can't quite see why. The syntax of it does not feel right. I want to be ...
2
votes
3answers
328 views
Design Strategy of CSV Parser
I wanted to review my design strategy for CSV parser.
I have 4 CSV files, which have different layouts as shown below.
Each row of a CSV file will be mapped to a class. For example, if CSV_FILE_A has ...
2
votes
2answers
358 views
refactoring golang code
There are not many available resources for refactoring go code. I'm a novice gopher who would appreciate any feedback on a small program which reads a csv file, string converts and parses a few rows ...
5
votes
2answers
353 views
Critiques on a trivially easy to use Python CSV class
I have been working on a project where I needed to analyze multiple, large datasets contained inside many CSV files at the same time. I am not a programmer but an engineer so I did a lot of searching ...
1
vote
0answers
165 views
Correctly import CSV data, even when possibly malformed
I've created a CSV parser that tries to build a string table out of a CSV file. The goal is to handle CSV files as well as Excel.
Input CSV file:
First field of first row,"This field is multiline
...
2
votes
3answers
795 views
More efficient way to retrieve first occurrence of every unique value from a csv column?
A large .csv file I was given has a large table of flight data. A function I wrote to help parse it iterates over the column of Flight ID's, and then returns a dictionary containing the index and ...
3
votes
1answer
99 views
Python: Improve csv script efficiency
I'm tasked with getting emails from a .csv file and using them to submit a form. I am using the csv and mechanize Python libraries to achieve this. This is my code:
import re
import mechanize
...
5
votes
5answers
3k views
Faster way of reading csv to grid
I have following in Windows Forms .NET 3.5
It works fine for csv with records less than 10,000 but is slower for records above 30,000.
Input csv file can can any records between 1 - 1,00,000 records ...
1
vote
2answers
386 views
Updating a chart from Yahoo Finance CSV files
I want to load some CSV columns or contents in my Ajax code. Please help me or improve my code. This will be live API code for me.
Data will be downloaded from Yahoo Finance in CSV format on an ...
13
votes
8answers
17k views
Java function to read a CSV file
I have a java function that reads a csv and returns its content as a Vector<Vector<String>>.
The function seems to work as I need it, but my gut feeling says it could be better (never ...
-1
votes
1answer
677 views
CSV parsing in Perl
I am looking for a Perl (5.8.8) script for CSV parsing that would follow CVS standards.
(See Wikipedia or RFC-4180 for details)
Sure, code should be fast enough, but more important is that it should ...
4
votes
4answers
2k views
searching a value from one csv file in another csv file - Python
I am writing a script that takes one csv file searches a value in another csv file then writes an output depending on the result it finds.
I have been using python's csv. Distreader and writer, I ...
4
votes
4answers
690 views
File parsing code in C++
I'm processing about 3.5gb/15minutes, and I was wondering if I could get this down to a better time...
Note that this is for a very specific CSV file, so I am not concerned about generality - only ...
1
vote
1answer
3k views
Fast import of numbers stored in a csv file in C++
I would like to efficiently import real numbers stored in csv format into an array. Is there a away to modify my code so it is faster? Also, is there a way to scan the file and compute the number of ...
3
votes
3answers
667 views
73 Lines of Mayhem - Parse, Sort and Save to CSV in PHP CLI
Inside of a folder named txt I have 138 text files (totaling 349MB) full of email addresses. I have no idea (yet) how many addresses there are. They are separated from one another by line breaks. I ...
8
votes
3answers
1k views
Python: getting lists of values from a CSV - Is there a cleaner way to do this?
So, I have a function that takes a column title, and a response.body from a urllib GET (I already know the body contains text/csv), and iterates through the data to build a list of values to be ...
13
votes
4answers
565 views
Is there a better way to build my CSV output than String.Format()?
I'm generating CSV strings for various 3rd party utilities and this section of code gets repeated in many classes. Is there a better way to generate this string?
public override string CsvString()
{
...
