current community

your communities

more stack exchange communities

company blog
Stack Exchange Inbox Reputation and Badges
tour help
up vote 1 down vote favorite

I would like to load a csv file as a numpy array. Each row contains string fields with spaces. I tried with both loadtxt() and genfromtxt() methods available in numpy. By default both methods consider space as a delimiter and separates each word in the string as a separate column. Is there anyway to load this sort of data using loadtxt() or genfromtxt() or will I have to write my own code for it?

Sample row from my file:

826##25733##Emanuele Buratti## ##Mammalian cell expression

Here ## is the delimiter and space denotes missing values.

share|improve this question
    
Did you try to include the delimiter keyword in the call to np.loadtxt()? i.e. np.loadtxt(delimiter="##") – reptilicus May 21 '13 at 4:00
    
I made use of this np.genfromtxt(trainFile, dtype = str, delimiter = '##', comments = '//', missing_values = ' ') loadtxt follows similar syntax. – viper May 21 '13 at 4:05
up vote 4 down vote accepted

I think your problem is that the default comments character # is conflicting with your delimiter. I was able to load your data like this:

>>> import numpy as np
>>> np.loadtxt('/tmp/sample.txt', dtype=str, delimiter='##', comments=None)
array(['826', '25733', 'Emanuele Buratti', ' ', 'Mammalian cell expression'], 
      dtype='|S25')

You can see that the dtype has been automatically set to whatever the maximum length string was. You can use dtype=object if that is troublesome. As an aside, since your data is not numeric, I would probably recommend using csv module rather than numpy for this job.

share|improve this answer
    
+1 for CSV, what I would have suggested as well. – Burhan Khalid May 21 '13 at 4:12
up vote 1 down vote

Here is the csv equivalent, as wim suggested:

import csv

with open('somefile.txt') as f:
    reader = csv.reader(f, delimiter='##')
    rows = list(reader)

As @wim pointed out the comments, this doesn't really work since the delimiter must be one character. So if you change the above so that delimiter='#', you get this as the result:

[['826', '', '25733', '', 'Emanuele Buratti', '', ' ', '', 'Mammalian cell expression']]
share|improve this answer
    
Burhan, I think this will not work without pre-processing because csv delimiter must be a 1-character string. – wim May 21 '13 at 4:49
    
Source: docs.python.org/2/library/csv.html#csv.Dialect.delimiter – wim May 21 '13 at 4:51
    
Right you are @wim. Thanks; but with using a single # as the delimiter, you still get the desired results, except you have extra spaces in the list. – Burhan Khalid May 21 '13 at 4:51

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.