A comma-separated values (CSV) file stores tabular data (numbers and text) as plain text, with rows and columns delimited by line terminators and commas, respectively. You may also use this tag for variants where fields are separated by tabs or semicolons.
-2
votes
1answer
14 views
PHP: Efficient way to search key value in csv file using php [on hold]
So i have a list of customers and i need to look up their address in a csv file. If i loop through the csv file, it takes way to much time (its a big csv file). So is there a way to search the key ...
4
votes
1answer
64 views
Calculating a table of deBroglie wavelengths for various electron energies
Here is the formula for the deBroglie wavelength of an electron versus its kinetic energy:
$$ \lambda(E_k) = h\left/\sqrt{\frac{(E_k+m_eC^2)^2-m_e^2C^4}{C^2}}\right.$$
and here is simple script that ...
3
votes
0answers
18 views
Perl script to analyze CSV files
I made a Perl script to analyze CSV files. However, it is running a little slow. Is there anything that I can change to make it more efficient?
...
0
votes
0answers
7 views
Powershell - User group membership from text file [on hold]
How can I improve below ?
As it is working however I only require the group names.
Instead I am getting response like 'CN#=Group,OU=Local server,OU=DH' which is very confusing to a Powershell noob,...
3
votes
2answers
34 views
2
votes
1answer
59 views
Arduino weather station logger
I made a program that uses the Adafruit pressure sensor, and the DHT22 temperature and humidity sensor to log pressure, temperature and humidity every 30 minutes. I left it outside my house overnight ...
3
votes
0answers
38 views
Fetching certain eventIDs from servers on network.
The PowerShell script below queries the Security event log on one or more servers for events with id 4663. When trying to retrieve all audit events for event id 4663 with the following code the ...
4
votes
2answers
50 views
Generate SQL UPDATE from Excel CSV file
I must write updates in a database from a flat file (CSV). I want to do that in the shell, with tools such as AWK.
...
5
votes
1answer
53 views
~600k external API requests
I have a CSV containing ~600k partnumbers to be uploaded to my website's inventory. However, this CSV only contains limited information. We're missing pricing and other related information. To get ...
3
votes
1answer
54 views
Extracting specific rows and columns from a CSV file
I have written a function to selectively extract data from a file.
I want to be able to extract only from a certain line and only given rows.
Would convert this function into a generator improve the ...
1
vote
1answer
52 views
Streaming learning OCaml
I wrote a simple online logistic regression, calibrated using gradient descent to compare the speed of an OCaml implementation vs the same Python script, executed with Pypy. It turned out that the ...
3
votes
0answers
55 views
Pandas calculation speed of stock beta on many dataframes
I have many (4000+) CSVs of stock data (Date, Open, High, Low, Close) which I import into individual Pandas dataframes to perform analysis. I am new to Python and want to calculate a rolling 12month ...
8
votes
2answers
983 views
Reading a repetitive file with repetitive code
I made this code for take some data from a .txt file. The text file is a large list of data from temperature with a format that is repetitive and have some lines at the beginning with the station ...
3
votes
1answer
82 views
PowerShell script to read line by line large CSV files
I am managing large CSV files (files ranging from 750 Mb to 10+ Gb), parsing their data into PSObjects, then processing each of those objects based on what is required.
I wrote the following script ...
1
vote
2answers
33 views
Generate Orders from Matrix and save as CSV
I had a task to create a VBA macros which will take a matrix data set with products in rows and persons as a columns (on intersection we have an ordered qty for each material for each person) and ...
3
votes
1answer
51 views
Imported data from CSV successfully, but slow
Currently I am using the gem activerecord-import to seed more than 55,000 records into my database, and it works fine. However, it takes too much time. I am ...
3
votes
2answers
44 views
CSV to Object converter with JavaDoc comments
This is a simple class but I'm looking to check I have the basics down to scratch.
Is my documentation thorough enough? (I'm very new to writing JavaDoc) Have I created too many variables, and does ...
1
vote
2answers
67 views
Comparing adjacent time stamps
I have this code which compares two adjacent time stamps. If they are less than 15 min apart, it adds a new time stamp (row) in between with the new time stamp equal to the previous time + 15 min and ...
3
votes
1answer
126 views
Basic C++ CSV class to read and write to .csv files
I've been working through the fifth edition of Lajoie's C++ Primer and decided to write a CSV reader/writer as my first project. Its main features are:
Able to explicitly call for:
Individual/...
6
votes
3answers
82 views
CSV concatenator
I have the following code (running in LINQPad) which takes tens of thousands of CSV files and concatenates them to create a single file.
Each CSV file has two lines: a header line and a data line. I ...
3
votes
0answers
78 views
CSV file parser and compare
This may seem like a lot of stuff? I just need help with 2 small parts the code works, however I have provided the rest of the info in case some one can help. USING PYTHON 3.4
Code below is ...
6
votes
2answers
66 views
Showing virus scan results from an API and a CSV file
I am utilizing an API module for interacting with virustotal.com in order to get AntiVirus results based of SHA256 hashes. The code I have is working but I feel like it can be improved greatly. I ...
5
votes
1answer
53 views
Powershell Interaction with Excel (Highlighting Cells)
I'm using Powershell to compare two, large csv files and output the rows which don't match. Once the two output files are made (one for the differences on each), I bring them up on Excel. Now I want ...
1
vote
2answers
67 views
PHP script that combines data from two CSV files to produce two new files
I've spent some time looking at various posts on Stack Overflow regarding PHP CSV file manipulation and have devised a script that combines data from two CSV files to create two new files.
The aim of ...
2
votes
3answers
111 views
Wrapping a CSV file for access and testing
I have a simple class that represents and accesses the data from a CSV file (using the CsvHelper library). I've tried restructuring the code to allow for better unit testing, but I'm sure the ...
5
votes
1answer
34 views
Improving CSV filtering with Python using regex
Consider a .csv file that contains a set of video names like so:
...
6
votes
1answer
555 views
Python search for array in large text file
I asked similar question at: http://stackoverflow.com/questions/38410982/superfast-regexmatch-in-large-text-file
...
2
votes
0answers
48 views
Building nested object structures from CSV input
I'm reading big csv files and mashing the lines into hierarchical objects for analysis and display. The csv might look like this:
...
7
votes
2answers
151 views
Load and analyze a list of people from a file
I recently completed a programming task in Java for a job that was javascript heavy but the hiring company wanted some Java knowledge. I've been using Java since the turn of the year. Other than ...
6
votes
1answer
70 views
Calculate where the patient results reside within the normal distribution
My background is medical, but I required a program that calculated whether my patients are "normal" or 1-2 SD values away from the mean. The program works just fine, but I'm interested how I can do ...
8
votes
1answer
67 views
Extract data from a DBMS, create a CSV dump and transfer it to an FTP server
First of all, my code does work as expected at least to some extent. The intention of the program is to extract data from a DBMS in order to create a CSV dump and transfer it to an FTP server.
My two ...
3
votes
2answers
61 views
Optimize a simple and quick python script for transposing a .csv file
I need to transpose the following file output1.csv, which is is a result from a quantum chemistry calculation into a single colum efficiently:
...
1
vote
0answers
27 views
Export database values as csv with a helper class
Should I be using getter and setter functions in my class nodeStruct?
I am currently using them where there is need to manipulate the input value (e.g for ...
1
vote
0answers
26 views
Grouping logs and providing counts
The code is supposed to group start and end time logs and provide log counts and unique ID counts. The grouping will be variable 1 hour, 6 hours, 12 hours, 24 hours, etc.
What is the better way to ...
5
votes
2answers
54 views
Sieve of Eratosthenes with a Swing UI
I'm fairly new to Java. I want to divide my code into 3 classes. First class to contain my GUI, second to compute the algorithm and third class to write results in my file. I have the code working but ...
8
votes
2answers
59 views
Reusable VBA macro to prevent Excel from destroying the data (+ backup capacity)
My goal is to prepare the Excel file (see the download link at the end) I would share with my non-technical colleagues, so that they enter some data into it and then I was able to automatically ...
2
votes
1answer
26 views
An interesting use of the reduce function to convert tab-delimited data into JSON
I am learning functional programming in JavaScript. This is some code from a video tutorial by funfunfunction. I am particularly concerned about the way the existence of the customer key is checked in ...
3
votes
1answer
212 views
Converting embedded JSON into flat table
I am new Python user, who decided to use Python to create simple application that allows for converting json files into flat table and saving the output in cvs format. I was wondering if you could ...
3
votes
4answers
249 views
Finding the average of each row in a CSV file
I am getting back into python after a year of not doing it at all. I just wrote a bit of code to create a file that looks like this:
1,2
1,1,1,1
-1,0,1
42,17
...
3
votes
1answer
59 views
Using Ruby to record tasks
I started writing this program to get a better understanding of different aspects of Ruby. I tried to follow the ruby style guide here.
What it's supposed to do is take the exercises/tasks you give ...
5
votes
3answers
292 views
Parse a CSV file and return an object or array
I recently published a JavaScript module for NPM which is also available on Github by the name of rawiki-parse-csv. The code takes raw CSV data and returns an array. The method has one option to ...
2
votes
1answer
149 views
csvshuf: a tool to shuffle CSV columns written in Python
Edit: After reading Gareth's answer I have pushed an updated version of the code to github.
I needed to shuffle cells from specific columns in several CSV files. One of the requirements was to be ...
6
votes
2answers
138 views
Fast CSV parser
I have written a CSV parser to read in 12609000 lines of CSV data. Each line has 16 fields of both string and double type data. The following code takes about 40 seconds to parse the data. I would ...
5
votes
2answers
168 views
Writing a hiveserver2 query result to CSV with Python using pyhs2
I am trying to create a small application that will help me extract data from Hadoop via hiveserver2 by simply writing it to CSV. At the moment, it takes about 40 seconds to pull 10,000 records from a ...
3
votes
1answer
54 views
Extracting coordinates file that are inside a bounding box
I'm scanning a CSV file to see which values fit inside of a given box. The CSV file has X and Y coordinates, and the arguments to the function are the perimeters of a box (i.e, top, bottom, left side, ...
1
vote
2answers
95 views
Reading CSV files and inserting in a SQL database in VB.Net
This program reads all files in a folder and subfolders line by line, converts them in a supported character set, replaces all unsupported characters and inserts them in a SQL database. My idea was to ...
3
votes
2answers
47 views
Combining values corresponding to the same key
What the following code does is:
The key field is column0 (where sometimes there could be single key, and sometimes keys separated by comma).
The rest of the columns are also either single or comma ...
1
vote
1answer
223 views
Reading and processing a big CSV file
I am trying to write a script to read a CSV file containing 1 million domain names, look up these domain names and store the results in another CSV file. I am trying the following code, but the number ...
4
votes
1answer
45 views
Saves Scores of an Arithmetic Quiz
This code is part of a larger code that generates an Arithmetic Quiz, although the code I have show below just saves the score that the user got on the test and their name. When saving the code it ...
-1
votes
1answer
66 views
Displaying membership information
The code is about displaying membership information of all clients from a database. This is for homework and I'm just curious if more can be done to improve this. Feel free to alter it, but please ...
