A comma-separated values (CSV) file stores tabular data (numbers and text) as plain text, with rows and columns delimited by line terminators and commas, respectively. You may also use this tag for variants where fields are separated by tabs or semicolons.
-3
votes
0answers
21 views
Converting txt file into csv in python for Analytics [on hold]
I just started to learn data analytics and recommendation system. For the first part of learning, I followed a web-page to build a food recommendation system. The first step is to change the dataset ...
2
votes
1answer
36 views
Multiple files data processing in Clojure
I have made a small script which grabs data into a map from 80 CSV files and calculates some statistics like average, standard deviation etc. It's also adding some additional data to map from filename....
3
votes
3answers
46 views
Converting a text file to a CSV file
I'm attempting to learn more about Java and have created a method that takes a text file with stdout (space separated file) and converts it to a CSV file.
I was ...
0
votes
1answer
75 views
Read a CSV file and do natural language processing on the data
I am studying the techniques of data mining and data processing. I'm doing this through data I've collected and stored in a csv file. The problem is that this filed was very large, to the point of ...
3
votes
1answer
34 views
ECG Bash selection tool
I made the following bash script for extracting a group of ECG signals from ECG files. I would like to know if there is any mistakes and/or weaknesses. I have experienced difficulties in integrating ...
0
votes
0answers
23 views
Generating Django fixtures from .csv
I've written this script to create a Django .json fixture starting from a .csv.
The .csv must have 'model' and 'pk' fields. What do you think?
...
3
votes
1answer
54 views
Gather information about computers from multiple CSV files
I created a script to import several CSV files from various sources and one CSV file with a list of systems in it. the script searches each CSV file to see if the system exist in the file and if it ...
5
votes
1answer
75 views
Calculating a table of deBroglie wavelengths for various electron energies
Here is the formula for the deBroglie wavelength of an electron versus its kinetic energy:
$$ \lambda(E_k) = h\left/\sqrt{\frac{(E_k+m_eC^2)^2-m_e^2C^4}{C^2}}\right.$$
and here is simple script that ...
3
votes
1answer
85 views
Perl script to analyze CSV files
I made a Perl script to analyze CSV files. However, it is running a little slow. Is there anything that I can change to make it more efficient?
...
3
votes
2answers
50 views
2
votes
1answer
70 views
Arduino weather station logger
I made a program that uses the Adafruit pressure sensor, and the DHT22 temperature and humidity sensor to log pressure, temperature and humidity every 30 minutes. I left it outside my house overnight ...
3
votes
0answers
47 views
Fetching certain eventIDs from servers on network.
The PowerShell script below queries the Security event log on one or more servers for events with id 4663. When trying to retrieve all audit events for event id 4663 with the following code the ...
4
votes
2answers
57 views
Generate SQL UPDATE from Excel CSV file
I must write updates in a database from a flat file (CSV). I want to do that in the shell, with tools such as AWK.
...
6
votes
2answers
100 views
~600k external API requests for uploading part numbers to an inventory
I have a CSV containing ~600k part numbers to be uploaded to my website's inventory. However, this CSV only contains limited information. We're missing pricing and other related information. To get ...
3
votes
1answer
76 views
Extracting specific rows and columns from a CSV file
I have written a function to selectively extract data from a file.
I want to be able to extract only from a certain line and only given rows.
Would convert this function into a generator improve the ...
1
vote
1answer
53 views
Streaming learning OCaml
I wrote a simple online logistic regression, calibrated using gradient descent to compare the speed of an OCaml implementation vs the same Python script, executed with Pypy. It turned out that the ...
3
votes
0answers
108 views
Pandas calculation speed of stock beta on many dataframes
I have many (4000+) CSVs of stock data (Date, Open, High, Low, Close) which I import into individual Pandas dataframes to perform analysis. I am new to Python and want to calculate a rolling 12month ...
8
votes
2answers
987 views
Reading a repetitive file with repetitive code
I made this code for take some data from a .txt file. The text file is a large list of data from temperature with a format that is repetitive and have some lines at the beginning with the station ...
3
votes
1answer
151 views
PowerShell script to read line by line large CSV files
I am managing large CSV files (files ranging from 750 Mb to 10+ Gb), parsing their data into PSObjects, then processing each of those objects based on what is required.
I wrote the following script ...
1
vote
2answers
35 views
Generate Orders from Matrix and save as CSV
I had a task to create a VBA macros which will take a matrix data set with products in rows and persons as a columns (on intersection we have an ordered qty for each material for each person) and ...
3
votes
1answer
56 views
Imported data from CSV successfully, but slow
Currently I am using the gem activerecord-import to seed more than 55,000 records into my database, and it works fine. However, it takes too much time. I am ...
3
votes
2answers
51 views
CSV to Object converter with JavaDoc comments
This is a simple class but I'm looking to check I have the basics down to scratch.
Is my documentation thorough enough? (I'm very new to writing JavaDoc) Have I created too many variables, and does ...
1
vote
2answers
72 views
Comparing adjacent time stamps
I have this code which compares two adjacent time stamps. If they are less than 15 min apart, it adds a new time stamp (row) in between with the new time stamp equal to the previous time + 15 min and ...
3
votes
1answer
276 views
Basic C++ CSV class to read and write to .csv files
I've been working through the fifth edition of Lajoie's C++ Primer and decided to write a CSV reader/writer as my first project. Its main features are:
Able to explicitly call for:
Individual/...
6
votes
3answers
85 views
CSV concatenator
I have the following code (running in LINQPad) which takes tens of thousands of CSV files and concatenates them to create a single file.
Each CSV file has two lines: a header line and a data line. I ...
3
votes
1answer
99 views
CSV file parser and compare
This may seem like a lot of stuff? I just need help with 2 small parts the code works, however I have provided the rest of the info in case some one can help. USING PYTHON 3.4
Code below is ...
6
votes
2answers
72 views
Showing virus scan results from an API and a CSV file
I am utilizing an API module for interacting with virustotal.com in order to get AntiVirus results based of SHA256 hashes. The code I have is working but I feel like it can be improved greatly. I ...
5
votes
1answer
57 views
Powershell Interaction with Excel (Highlighting Cells)
I'm using Powershell to compare two, large csv files and output the rows which don't match. Once the two output files are made (one for the differences on each), I bring them up on Excel. Now I want ...
1
vote
2answers
89 views
PHP script that combines data from two CSV files to produce two new files
I've spent some time looking at various posts on Stack Overflow regarding PHP CSV file manipulation and have devised a script that combines data from two CSV files to create two new files.
The aim of ...
2
votes
3answers
130 views
Wrapping a CSV file for access and testing
I have a simple class that represents and accesses the data from a CSV file (using the CsvHelper library). I've tried restructuring the code to allow for better unit testing, but I'm sure the ...
5
votes
1answer
35 views
Improving CSV filtering with Python using regex
Consider a .csv file that contains a set of video names like so:
...
6
votes
1answer
595 views
Python search for array in large text file
I asked similar question at: http://stackoverflow.com/questions/38410982/superfast-regexmatch-in-large-text-file
...
2
votes
0answers
49 views
Building nested object structures from CSV input
I'm reading big csv files and mashing the lines into hierarchical objects for analysis and display. The csv might look like this:
...
7
votes
2answers
152 views
Load and analyze a list of people from a file
I recently completed a programming task in Java for a job that was javascript heavy but the hiring company wanted some Java knowledge. I've been using Java since the turn of the year. Other than ...
6
votes
1answer
72 views
Calculate where the patient results reside within the normal distribution
My background is medical, but I required a program that calculated whether my patients are "normal" or 1-2 SD values away from the mean. The program works just fine, but I'm interested how I can do ...
8
votes
1answer
70 views
Extract data from a DBMS, create a CSV dump and transfer it to an FTP server
First of all, my code does work as expected at least to some extent. The intention of the program is to extract data from a DBMS in order to create a CSV dump and transfer it to an FTP server.
My two ...
3
votes
2answers
63 views
Optimize a simple and quick python script for transposing a .csv file
I need to transpose the following file output1.csv, which is is a result from a quantum chemistry calculation into a single colum efficiently:
...
1
vote
0answers
30 views
Export database values as csv with a helper class
Should I be using getter and setter functions in my class nodeStruct?
I am currently using them where there is need to manipulate the input value (e.g for ...
1
vote
0answers
26 views
Grouping logs and providing counts
The code is supposed to group start and end time logs and provide log counts and unique ID counts. The grouping will be variable 1 hour, 6 hours, 12 hours, 24 hours, etc.
What is the better way to ...
5
votes
2answers
55 views
Sieve of Eratosthenes with a Swing UI
I'm fairly new to Java. I want to divide my code into 3 classes. First class to contain my GUI, second to compute the algorithm and third class to write results in my file. I have the code working but ...
8
votes
2answers
69 views
Reusable VBA macro to prevent Excel from destroying the data (+ backup capacity)
My goal is to prepare the Excel file (see the download link at the end) I would share with my non-technical colleagues, so that they enter some data into it and then I was able to automatically ...
2
votes
1answer
27 views
An interesting use of the reduce function to convert tab-delimited data into JSON
I am learning functional programming in JavaScript. This is some code from a video tutorial by funfunfunction. I am particularly concerned about the way the existence of the customer key is checked in ...
3
votes
1answer
514 views
Converting embedded JSON into flat table
I am new Python user, who decided to use Python to create simple application that allows for converting json files into flat table and saving the output in cvs format. I was wondering if you could ...
3
votes
4answers
312 views
Finding the average of each row in a CSV file
I am getting back into python after a year of not doing it at all. I just wrote a bit of code to create a file that looks like this:
1,2
1,1,1,1
-1,0,1
42,17
...
3
votes
1answer
61 views
Using Ruby to record tasks
I started writing this program to get a better understanding of different aspects of Ruby. I tried to follow the ruby style guide here.
What it's supposed to do is take the exercises/tasks you give ...
5
votes
3answers
481 views
Parse a CSV file and return an object or array
I recently published a JavaScript module for NPM which is also available on Github by the name of rawiki-parse-csv. The code takes raw CSV data and returns an array. The method has one option to ...
2
votes
1answer
186 views
csvshuf: a tool to shuffle CSV columns written in Python
Edit: After reading Gareth's answer I have pushed an updated version of the code to github.
I needed to shuffle cells from specific columns in several CSV files. One of the requirements was to be ...
6
votes
2answers
159 views
Fast CSV parser
I have written a CSV parser to read in 12609000 lines of CSV data. Each line has 16 fields of both string and double type data. The following code takes about 40 seconds to parse the data. I would ...
5
votes
2answers
215 views
Writing a hiveserver2 query result to CSV with Python using pyhs2
I am trying to create a small application that will help me extract data from Hadoop via hiveserver2 by simply writing it to CSV. At the moment, it takes about 40 seconds to pull 10,000 records from a ...
3
votes
1answer
65 views
Extracting coordinates file that are inside a bounding box
I'm scanning a CSV file to see which values fit inside of a given box. The CSV file has X and Y coordinates, and the arguments to the function are the perimeters of a box (i.e, top, bottom, left side, ...
