Tagged Questions
-1
votes
0answers
23 views
Create a function using text data from another file
I have a support file that has a predefined function called data_of_a_date.
It takes a date string in the form 'DD-MM-YY' and returns the text of a csv file, each line represents a minute of the day.
...
-1
votes
2answers
17 views
python write items prior to first match
i am looking to fine tune this script i wrote.
i remove all items in list1 contained in list0.
however i ran into an issue, and i would like to find the first item in list1 that matches list0 and ...
0
votes
0answers
31 views
reading foreign characters in python
I'm trying to read a csv file containing foreign characters (french accents at the moment but will be russian as well in the future). Is there a way to read these csvs without removing/replacing the ...
-2
votes
3answers
32 views
How to add a header to an existing csv file?
I know this is a very basic question.
I have a CSV file, which contains data already. This file is generated automatically not using opening with Dictreader or open object.
Goal
I want to open an ...
-1
votes
0answers
21 views
trying to format data into table (new to python) [on hold]
I am using the following code to access an external FTP site but would like to format the data into a table to eventually stick into SQL - im very new to python so any help is appreciated ...Initially ...
-1
votes
0answers
17 views
Compare directories to CSV and move matching directories [on hold]
I have a CSV file which contains a list of client names. For each client they have their own directory on our file server (Z:\Clientele). A large number of these directories need to be moved to an ...
-2
votes
1answer
47 views
if row[0] in row[1] print row
I have a csv file that has 2 columns. I am simply trying to figure if each row[0] value is in some row[1] and if so, to print row.
Items in csv file:
COL1, COL2
1-A, 1-A
1-B, 2-A
2-A, 1-B
...
0
votes
0answers
55 views
how to open a 200GB csv file [on hold]
I'm planning to run some statistical analysis on a data set I recently downloaded from a website, but I don't know how to open it since it's around 200GB stored in a single .csv file, my past ...
1
vote
1answer
37 views
Python pandas: type error in groupby
I read in a csv file as the following:
import pandas as pd
out = "M:/transitions.csv"
transitions = pd.read_csv(out)
transitions = transitions.groupby('unique_pid')
Here is what my dataframe ...
1
vote
1answer
45 views
Read file in text mode but also count raw bytes in Python?
I'd like to read a file in text mode line-wise, but at the same time I'd like to insert an intermediate step which works on bytes data and basically counts the bytes read so far.
Is there a good way ...
-3
votes
1answer
29 views
Python record level comparision 2 large delimited files
I have 2 large delimited files.
Help needed in :
a) I need to get the row count based on the key column for both the files
b) finding the duplicates based on key column in both the files
c) Get ...
0
votes
3answers
23 views
How to export the result of print into a csv file in Python?
How to export the result of following code into a csv file in python?
print(['%s %0.2f'%(node,centrality[node]) for node in centrality])
The result looks like this:
['faef 0.37', 'efef 0.60', ...
1
vote
2answers
28 views
How to import csv file as numpy.array in python?
say i have a csv file.csv in this format:
dfaefew,432,1
vzcxvvz,300,1
ewrwefd,432,0
how to import the second column as a numpy.array and the third column as another one like this:
second = ...
0
votes
1answer
35 views
Python: write.csv adding extra carriage return
I am writing an Excl to CSV converter using python.
I'm running in Linux and my Python version is:
Python 2.7.1 (r271:86832, Dec 4 2012, 17:16:32)
[GCC 4.1.2 20080704 (Red Hat 4.1.2-51)] on linux2
...
0
votes
4answers
39 views
Python - Calculate average for every column in a csv file
I'm new in Python and I'm trying to get the average of every (column or row) of a csv file for then select the values that are higher than the double of the average of its column (o row). My file have ...
-2
votes
1answer
46 views
Python CSV - use dictionary for each cell
i hope my question makes sense. i am looking for a way to read a csv file, and map a dictionary to each cell. i can make it work without csv, but i am having a hard time making it work when reading a ...
1
vote
1answer
24 views
CSV translate data within rows
i hope someone can point me in the right direction.
what would be an efficient way to translate data within a row[x]?
for example i want to convert the following: street,avenue,road,court to ...
-1
votes
1answer
19 views
Python: Setting tuples from csv file
So essentially what I have is a csv file which is loaded in via some function, lets call it
get_csv
So when I have this data, I want to create a new function to format the data sent from the server ...
0
votes
0answers
51 views
csv multiprocessing load to dictionary or list
I'm moving from MATLAB to python my algorithms and I have stuck in parallel processing
I need to process a very large amount of csv's (1 to 1M) with a large number of rows (10k to 10M) with 5 ...
4
votes
3answers
42 views
Reorganize a CSV so Dates are not column Headers
I am trying to reorganize an excel table (or csv) so that dates are no longer column headers. I'm using a limited knowledge of python to attempt to do this but for lack of knowing where to start I ...
0
votes
1answer
33 views
Index Error: Index out of bounds when using numpy in python
I have a code that works fine when I have small CSV's of data but errors out when I try to run large CSV's through it. In essence this code is supposed to place 3 CSV's worth of data into 3 separate ...
0
votes
1answer
22 views
Python: When outputting a list to a csv, each row element has strange spacing added
I have finally got my python program outputting what I want into each cell of row 1 of a csv, however weird tabbing or spacing occurs after each item
Here is the segment of code in question:
def ...
0
votes
1answer
32 views
Copy (from) csv with heades in postgres with python
I'm trying to fill a table from CSV files in a python script.
The SQL statement, which follows, runs without error:
COPY registro
FROM '/home/pablo/Escritorio/puntos/20140227.csv'
DELIMITER ','
CSV ...
0
votes
1answer
21 views
Split records based on a space or a tab using csvreader
I have a record file an example record looks like this:
122792 2014-07-26 1672
The first two records are separated by space the second two by a tab space. How can I make the csvreader read all ...
-1
votes
2answers
52 views
Remove lines and print the average in csv with python
I'm working on a python script that generates a csv file reading different columns from all the csv files I have in a folder. For now I generate the file and sort the columns.
The code that generates ...
0
votes
0answers
33 views
Constructed file path to csv not recognized. But the file is there
I want to load 134 CSVs of baby names from the social security into pandas.
First, I create a list of file names:
In [48]:
file_name[0:5]
Out[48]:
['yob1800.txt', 'yob1801.txt', 'yob1802.txt', ...
1
vote
2answers
48 views
Reading CSV's with Pandas and dealing with comments
This is a example of the data files I am trying to read in using Pandas. All the files have different number of comment lines but all start the data section with the BEGIN and end with END and maybe a ...
0
votes
2answers
21 views
Double quotes keep coming in csv when merging file in python, how to remove?
I have many files with URL links and I want to merge them all into one big file. The links in the inidividual files do not have double quotes around them. The merged file somehow added double quote to ...
0
votes
2answers
41 views
Python to extract unique CSV rows
I'm trying to get the first occurrences of each row of a CSV in Python. However, I'm facing an issue. My CSV files looks like this:
1,2,3,a,7,5,y,0
1,2,3,a,3,5,y,8
1,2,3,a,5,3,y,7
1,2,3,d,7,5,n,0
...
0
votes
1answer
28 views
Cutplace : Conditional check for empty fields
I'm trying to create a configuration with the tool 'cutplace' (http://roskakori.github.io/cutplace) to check a csv file for empty fields based on a particular field not being empty.
So the rule is ...
0
votes
4answers
36 views
Two dimensional dictionary, list or array from CSV file in Python
I am very new to python...
I am trying to read a regression coefficient matrix into python from a csv file of the format below:
0.10 0.15 0.20 0.25 0.30 0.35
a1 -0.0011 0.0008 0.0019 0.0034 0.0067 ...
0
votes
2answers
20 views
In Python how can I create and organized dictionary through the process of reading a CSV file
I am attempting to read a CSV file into Python and create and organized dictionary from the data retrieved. The CSV has a similar format to the one shown below.
Time A B C D
0 1 2 4 ...
0
votes
1answer
15 views
Python count numbers of null valors in my imported csv file
I've exported a raster (in ascii) file to csv with hundres of rows and cols. Null valors are strored with the value -999.
I've created a script to count the numbers of -999 in every row and col in ...
0
votes
1answer
13 views
delimiter for string or file: using csv.reader() and not csv.Sniffer()
Background: streaming log files from Amazon's S3. From zipped files, after a few steps, I get a file-like object. For gzipped files, I decompress a chunk in the stream, which is a string, and then use ...
1
vote
1answer
27 views
reading multiple csv file into a big pandas data frame by appending the columns of different size
so i am creating some data frame in a loop and save them as a csv file. The data frame have the same columns but different length. i would like to be able to concatenate these data frames into a ...
-1
votes
1answer
23 views
writing a nested dictionary to csv file
Novice question: I'm trying to write a function to write a nested dictionary to a csv file, so I pulled some code from Stack Overflow (Convert Nested Dictionary to CSV Table), but I keep getting the ...
0
votes
1answer
24 views
Python CSV module handling comma within quote inside a field
I am using Python's csv module to parse data from a CSV file in my application. While testing the application, my colleague entered a piece of sample text copy-pasted from random website.
The sample ...
1
vote
0answers
13 views
Django import-export importing data with object relationships
I've been playing with django import-export because it seems to be the obvious choice for anything import/export related in django and it's fantastic. Trouble is I can't find much in the way of docs ...
0
votes
1answer
26 views
Python Pandas - Split different entries into separate rows allowing only certain combinations
I have searched along the web for an answer, but, while similar questions have been asked, they were not the same and I have not been able use them.
I have a pandas dataframe in which several columns ...
0
votes
3answers
26 views
write each line as it's processed using for-loop in python, but only writing the first line
I have some code that I am trying to optimize to be more efficient. One part of that is to process my file and after each line is processed, immediately write it to csv. This is ideal as then I am ...
0
votes
1answer
23 views
Python throwing IndexError: list index out of range, with CSV
I have had an error in a Python program when trying to read a CSV file. What is most perplexing is the error was not there initially, and has "come and gone" once before, but has now returned. The ...
-1
votes
0answers
38 views
Score Strings by position from .csv [closed]
So, I have a CSV file that contains sub-strings in different sizes from a main string.
I want to split the sub-strings in characters and score them based on the frequency that each character appears.
...
0
votes
1answer
44 views
How to separate a UTF-8 file into separate rows (comma delimited) using python?
I am trying to convert a UTF-16 file to a UTF-8 file (as I am using the python csv module which apparently does not handle UTF-16 files). I then want to delimit this UTF-8 file so I can import it into ...
-2
votes
1answer
36 views
How to calculate difference between rows of csv file with python
I am new to Python. I would like to do the difference between two rows of a csv file when they have the same id. This csv dataset is built from an sql table export which has more than 3 millions rows. ...
0
votes
2answers
39 views
Sort csv columns with Python
I have a CSV file that looks like:
Timestamp;A;O;P;M;
22:05;5;7;2;1
22:10;2;3,4;7
22:15;7;4;3;2
With a python script I would like to order the header (except the timestamp. it has to be the first ...
1
vote
2answers
40 views
How to compile multiple csv files while checking for duplicate row titles and co-adding new data
I have searched quite thoroughly attempting to find an answer to my question with no luck.
To explain my question slightly better, I am tasked with merging multiple .csv files while also doing some ...
-3
votes
1answer
42 views
export from dictionary to csv file [closed]
I have the following data as dictionary in python:
data=dict(
User1=dict(drdgbd=4, ffff=4.5),
User2=dict(ffff=3.5, drdgbd=3.5, jhuer=4),
User3=dict(drdgbd=2.5, df34=5)
)
Now I want to export ...
3
votes
1answer
37 views
Simple CSV lexer
I want to color CSV files with pygments by column like here:
See that same column is colored with the same color.
Currently pygments doesn't include CSV parser because CSV is said to be obscure ...
-2
votes
1answer
53 views
Find the difference of the two csv files
I have two csv files named x.csv and y.csv. x.csv has only one row - Column A:0, Column B:1, Column C:2, Column D:3. In y.csv, only one row - Column A:2, Column B:3, Column C:4. I need to find the ...
-1
votes
1answer
28 views
Need help filtering out incomplete entries in a .csv [Python]
I have a set of data points, which look like this:
ID 70014 1940 1 1 26.8 1 Y
ID 70014 1940 1 2 29.8 1 Y
ID 70014 1940 1 3 34.3 1 Y
ID 70014 1940 ...
