A comma-separated values (CSV) file stores tabular data (numbers and text) as plain text, with rows and columns delimited by line terminators and commas, respectively. You may also use this tag for variants where fields are separated by tabs or semicolons.
3
votes
2answers
34 views
Advanced CSV-to-SQLite converter
I just wrote an advanced CSV parser that translates CSV to SQL(SQLite Compatible) statements, which are then inserted into the database. It translates variable names in the CSV to values defined in ...
2
votes
1answer
24 views
Create an array of categories from comma-seperated text input
This code works just fine, but could obviously be more elegantly written.
The basic idea is to create an array of categories from a text input. The code splits up the text input by commas to create ...
4
votes
1answer
81 views
Generating frequency tables based on CSV dataset, in R
As a follow-up to this question concerning Python I present you with the exact same thing - for R.
Initially I started with R. However, it takes 44 minutes to run when unleashed on all my data. I ...
4
votes
2answers
59 views
Dataset and frequency list generation by looping over files
I have been told that it would be wise to split up my code into semantically useful blocks. I tried the following but I need feedback.
What I already did:
As you can see I create two .csv files, so ...
2
votes
1answer
29 views
CSV parsing program that creates distinct header rows with transaction rows underneath
My code reads in the data using DictReader, then creates a header row that contains my composite key (PEOPLE_ID, DON_DATE), and then adds various values that are ...
1
vote
0answers
43 views
Creating CSV to email [closed]
While this isn't a problem, I wanted to hear if someone could please check the format string I use when I create an CSV file? It does work, but it looks kind of "weird" and was wondering if this was ...
4
votes
3answers
58 views
Writing huge 2D vectors to text file too slow
I have 10 large (34 million cell) 2D gridded vectors storing doubles. When written they are over 200 MB in size. I use an ofstream object to write them to a text ...
3
votes
1answer
75 views
Stripping whitespace in a CSV file
I am interested in removing leading/trailing whitespace in a .csv file, and I was wondering if there's a better way to execute this:
...
4
votes
1answer
74 views
Updating a .csv file
I have a CSV file, call it csv_file. It has the following content:
Username, Password
name1, pass1
name2, pass2
...
I also have a dictionary, call it ...
6
votes
3answers
159 views
Manipulating a .csv file to look for two common values to create a key, then summing up values
My code reads a .csv file, looks for a couple values to create its key, and then wraps up the data based on a few business rules I have. I'm trying to learn "the right way", so I'd really appreciate ...
3
votes
2answers
147 views
Python script to create Android/iOS strings from a .csv
My script takes a CSV file of strings and then creates an XML file for Android and a .strings file to be used for iOS. It separates each locale meaning that a file will be created for each language.
...
6
votes
3answers
88 views
3
votes
1answer
135 views
Reading, writing and filtering a CSV file
I have a CSV with 5+ million rows and I want to filter it:
...
3
votes
1answer
53 views
Web link parser that hunts for a specific string
This handles a parsing system to look at every users profile page and determine their job classification based on the class of a div element. I noticed that the class was always consistent on specific ...
1
vote
1answer
110 views
CSV File Reader Project
I'm trying to focus on learning so I can get an entry level position and the best way for me to learn right now is to have someone review and provide areas of improvement. This project isn't entirely ...
18
votes
5answers
1k views
Inventorizing the universe
I recently started re-factoring a project I hadn't touched in many months.
The original purpose was to be an inventory management system for a game I played, Ogame. In this game you play inside a ...
4
votes
2answers
282 views
Converting CSV files to Excel workbooks
My converter converts all CSV files in the subfolders of folders 1, 2 & 3 into Excel workbooks. As of now, I am converting using codes for each folders. I previously tried to combine those into ...
3
votes
1answer
108 views
Converting a CSV file to a social network graph for Pajek
The code will ask for a directory and will then read the CSV file and then generate a file in that directory describing a social network for Pajek.
Does anyone know a way to load this CSV file to a ...
2
votes
0answers
78 views
Filter CSV file and display it using Highcharts
I created a function which reads CSV file and displays those values in a graphical form using Highcharts.
Although it works, I have to read the CSV file twice because first:
I need to get all the ...
7
votes
3answers
148 views
Extracting data from CSV file on the internet
This is my code I made that extracts data from a .csv file stored on the web. My sheet called 'New Data' stores this data for 2 days. I store 2 days worth of data so that when I run it next time, it ...
4
votes
1answer
33 views
Consolidating a raw csv of two ids to grouped set of ids in json
The file I'm processing here is an outputted file from a Java library called duke.
Its goal was go through all purchases made on a site and group them into a single customer. The output is a csv ...
4
votes
3answers
156 views
Rating tennis players in a database, taking days to run
I have this project in data analysis for creating a ranking of tennis players. Currently, it takes more than 6 days to run on my computer.
Can you review the code and see where's the problem?
...
2
votes
2answers
50 views
Python script to read CSVs and regroup it into the data types I need
This code does the following:
Read CSV file and determine sub_type based on series_desc.
Turn row into dictionaries ...
0
votes
2answers
47 views
Comparing two columns in two different rows
I want to go through each line of the a .csv file and compare to see if the first field of line 1 is the same as first field of next line and so on. If it finds a match then I would like to ignore ...
5
votes
1answer
181 views
Extracting emails from a file and writing them to another file
The code below works fine for me. Is there any way that I can improve this code more? Note: delimiter is single whitespace only for email address.
getmail.py
...
5
votes
1answer
87 views
Import Excel, unpivot data, output tab-delimited for SQL Import
I have a client GL file (Excel) that needs to be regularly imported to SQL Server (as tab-delimited txt).
Sample (simplified):
...
2
votes
1answer
63 views
Read and output CSV content OO PHP
Okay, I am wanting to expand my armoury within PHP, so I've been researching OO PHP. With my researched knowledge I went on to create a quick script that reads a CSV file and outputs the results.
CSV ...
4
votes
2answers
93 views
CSV import-export in Python
I have finished a really short and small import / export script using Python 2.7, and now I would like to structure it using classes and methods where possible.
Could anybody give me some advice on ...
3
votes
3answers
553 views
Parsing multiple line records using Java 8 Streams
There was a suggestion in this StackOverflow question to move this question here. I added some more information and restructured the question a bit.
I'm trying to parse the following file which ...
1
vote
1answer
60 views
Finite state machine for CSV data
I want to read a file containing comma-separated values, so have written a finite state machine:
...
2
votes
1answer
77 views
Denormalizing an OrderedDict to CSV
Cross post from stackoverflow, as suggested.
I have an ordereddict, and I need to store it in CSV file.
Input:
...
7
votes
1answer
111 views
Creating a Wrapper for CSV Data
I am trying to complete a Ruby coding exercise. The specifics of the excercise can be found on a the level_up_exercises GitHub repo. This repo also contains the CSV file used in the exercise.
I ...
1
vote
1answer
231 views
Split excel file with multiple sheets, manipulate the data and create final out file
I have an excel file with 20+ separate sheets containing tables of data. My script iterates through each sheet, manipulates the data into the format I want it and then saves it to a final output file. ...
3
votes
1answer
84 views
Parsing large CSV in Perl
I am getting Out of Memory errors when I try to parse through a large CSV file (2.5Gb). My computer has 32Gb of Memory but Perl uses all of it up. The CSV has 2 ...
1
vote
0answers
148 views
Using BeautifulSoup to scrape various tables and combine in a .csv file
A page contains a table of links, each link contains a table relevant to the link (a subject). Create a list of these links to pass to the function called ...
1
vote
0answers
40 views
Ruby Script to modify and split CSVs
I have written the following script that is pointed at a CSV through it's file name and then splits the file into "drops" (for mailings) and does a couple of operations on it. As of now it works, but ...
4
votes
1answer
170 views
Efficiently filter a large (100gb+) csv file (v3)
I'm in the process of filtering some very(!) large files (100gb+): I can't download files with a lower granularity. This is a followup from this question.
The problem is as follows: I need to filter ...
4
votes
2answers
142 views
Efficiently filter a large (100gb+) csv file (v2)
EDIT: This question is followed up by this question.
I'm in the process of filtering some very(!) large files (100gb+): I can't download files with a lower granularity. This is a followup from this ...
2
votes
1answer
84 views
Populate a .txt document with a CSV spreadsheet using multiple templates, outputting documents for each data row
I'm looking for feedback on functionality, PEP8, advice, etc as a learning tool. I'm not sure how to tell if this program should be broken into separate functions or combined as one.
While hand ...
3
votes
1answer
774 views
Create a struct to store student data and perform statistical analysis on data
I had to create a program that had a struct that held data inside of it. I had to read an input file and take the contents and read it into an array of structs. Then, I had to solve for some ...
3
votes
3answers
134 views
Merging three CSV files with a primary key
This is my first time coding in my life. I don't really know much about coding. I googled whatever I need and combined them all together. It works great, but want to know if there is any improvement ...
2
votes
1answer
46 views
Tweeting statistics on shared links
I tend to be rather prolific in my sharing of things I find online. I'm interested in tracking my own habits to make them more effective. Therefore, I submit the above to be critiqued by the lot of ...
1
vote
1answer
73 views
Reading two csv files as dictionaries with differing key definitions
My script reads two .csv files and generates one dictionary per .csv file.
...
4
votes
1answer
130 views
Printing lines in a file, filtering by the first column
I am setting up a project which involves a lot of data analysis, each data point represented by a line in a large text file. In order to test for certain types of data, I added a tag to each line (It ...
1
vote
1answer
89 views
Reduce columns to produce product keycodes
I have written the following code that achieves my desired goal of converting a "raw" CSV file into a keycoded list. The idea is that a person will get the first product they ask for, and English ...
3
votes
3answers
135 views
Parsing a single CSV line into a list of strings
I've written this method to replace an older method that was much simpler, but used the regex split method and couldn't tell if a comma was in quotes/brackets/etc. and didn't read double quotes as ...
2
votes
1answer
114 views
Efficiently filter a large (100gb+) csv file (v1)
EDIT: This question was followed up by this question, with improved code.
I have a set of pretty large files (>2 GB, over 30m rows) containing intraday data in the following format:
...
13
votes
0answers
120 views
Utility that decodes and logs UDP packets
I have written the following utility, as my first non-tutorial program in Go.
The purpose of the utility is
to connect to a torque/force sensor (aka load-cell) via UDP;
to send an initialization ...
0
votes
2answers
727 views
Parsing a big text file, extract data & store it in a CSV file
I have a big log file (say 1-3 Gb) which I need to parse, extract data & save it in a CSV file.
Text File Data
...
3
votes
4answers
259 views
Splitting an array at commas
I was given a file of size 46KB full of names (around ~5000), in the format as shown below:
"MARY","PATRICIA","LINDA","BARBARA","ELIZABETH"..."DARRELL","ALONSO"
...
