how to convert regular numpy array to record array?

Question

I read in a sequence of numbers with

np.array(f.read().split(),dtype=np.float64)

Then I convert this to a 2-D array using np.reshape().

After this, how do to convert arr to a record array? I've tried (something like) the following:

filename = 'unstructured-file.txt'
nfields = 3
names = ('r','g','b')
with open(filename,'r') as f:
    arr = np.array(f.read().split(),dtype=np.float64)
    arr = arr.reshape(-1,nfields)
    out = np.array(arr,dtype=zip(names,['float64']*length(names))

but says TypeError: expected a readable buffer object

Any suggestions?

Edit: The main thing I want to do is to name my columns.

Instead of

out = np.array(arr,dtype=zip(names,['float64']*length(names))

If I use this,

out = np.core.records.fromrecords(arr.reshape(-1,nfields),names=','.join(names))

I can use out['r'] and so on, but out.dtype.names is None`. What is going on?

Edit2

The unstructured file looks like

 Some text
 More text
       100  1.000000E-01        46
 -1.891701E+04  1.702921E+02 -2.323660E+04  4.547841E+03 -2.778444E+04
  0.000000E+00  0.000000E+00  0.000000E+00  0.000000E+00 -2.149862E+04
  1.753467E+02  3.410277E+03 -1.034898E+05  2.778692E+04  0.000000E+00
  0.000000E+00  0.000000E+00  0.000000E+00  1.492281E+04  0.000000E+00
  0.000000E+00  0.000000E+00  9.000000E+01  9.000000E+01  9.000000E+01
  0.000000E+00 -4.774939E-01  0.000000E+00  0.000000E+00  0.000000E+00
 -2.243495E-01  3.513048E-01 -2.678782E-01  3.513048E-01 -7.155493E-01
  5.690034E-01 -2.678782E-01  5.690034E-01 -4.783123E-01  2.461974E+01
  0.000000E+00  0.000000E+00  0.000000E+00  2.461974E+01  0.000000E+00
  0.000000E+00  0.000000E+00  2.461974E+01
       200  2.000000E-01        46
 -1.891815E+04  1.421984E+02 -2.424678E+04  5.199451E+03 -2.944623E+04
  0.000000E+00  0.000000E+00  0.000000E+00  0.000000E+00 -2.174561E+04
  1.274613E+02 -6.004790E+01 -1.139308E+05  2.944807E+04  0.000000E+00
  0.000000E+00  0.000000E+00  0.000000E+00  1.445855E+04  0.000000E+00
  0.000000E+00  0.000000E+00  9.000000E+01  9.000000E+01  9.000000E+01
  0.000000E+00  7.785923E-01  0.000000E+00  0.000000E+00  0.000000E+00
  8.123304E-01  3.023486E-01 -5.891595E-01  3.023486E-01 -8.560144E-02
 -3.830618E-01 -5.891595E-01 -3.830618E-01  1.608437E+00  2.436174E+01
  0.000000E+00  0.000000E+00  0.000000E+00  2.436174E+01  0.000000E+00
  0.000000E+00  0.000000E+00  2.436174E+01

unutbu · Accepted Answer · 2011-10-11 11:31:55Z

5

To convert a plain numpy array to a structured array, use view:

import numpy as np

filename = 'unstructured-file.txt'
nfields = 3
names = ('r','g','b')
with open(filename,'r') as f:
    arr = np.array(f.read().split(),dtype=np.float64)
    arr = arr.reshape(-1,nfields)
    out = arr.view(dtype=zip(names,['float64']*len(names))).copy()

edited Oct 11, 2011 at 11:31

answered Oct 11, 2011 at 10:56

unutbu

885k197 gold badges1.8k silver badges1.7k bronze badges

Thanks, but I meant for the 'unstructured-file.txt' designation to indicate that it's not in a table and so does not work for this.

hatmatrix
– hatmatrix

10/11/2011 11:03:25
Commented Oct 11, 2011 at 11:03
Can you give an example of what unstructured-file.txt looks like?

unutbu
– unutbu

10/11/2011 11:35:29
Commented Oct 11, 2011 at 11:35
brilliant -- what happens if you don't use the .copy() method? Seems to still work?

hatmatrix
– hatmatrix

10/11/2011 11:54:10
Commented Oct 11, 2011 at 11:54
2

out=arr.view(...) makes out a view of arr. So modifying out would also modify arr. They share the same underlying data. I added copy() so that out would be an independent array. Both are useful; it just depends on what you want to do.

unutbu
– unutbu

10/11/2011 11:58:06
Commented Oct 11, 2011 at 11:58

Add a comment |

Collectives™ on Stack Overflow

how to convert regular numpy array to record array?

1 Answer 1

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Your Answer

Sign up or log in

Post as a guest

Linked

Related