Take the 2-minute tour ×
Stack Overflow is a question and answer site for professional and enthusiast programmers. It's 100% free, no registration required.

I am reading a stream of data from an A-D converter via a socket from Python; the data come in as raw bytes. I want to format these bytes as int32 and place them into an ndarray. The read process looks something like this:

def datarecv():
    global msgbuf
    binlen = BURSTLEN + 4
    while len(msgbuf) < binlen:
        msgbuf = msgbuf + socket.recv(4096)
    reply = msgbuf[0:binlen]
    msgbuf = msgbuf[binlen:]
    # each recv comes with a 4 byte header that I throw away...
    return reply[4:]

The following is used successfully to write the received data to a file:

with open(filename, "wb') as f:
    bytesremaining = framesize
    for i in range(lines):
        f.write(datarecv()[0:min(linesize, bytesremaining)])
        bytesremaining -= linesize

I can then read back the file with something like this:

>>> data = numpy.fromfile(filename, dtype='int32')
>>> type(data)
<type 'numpy.ndarray'>

So my data variable is the format I'm looking for, I.E.

>>> data[1:10]
array([4214234234, 2342342342, 2342342342, 34534535, 345345353, 5675757, 
      2142423424, 35334535, 35353535, 4754745754], dtype=int32)

** BUT ** I want to omit the intermediate step of writing to a file. After I read in the raw stream of data I want to make it an ndarray so that I can manipulate the data. I can change the line from

        f.write(datarecv()[0:min(linesize, bytesremaining)])

to

        bigbuf = bigbuf + datarecv()[0:min(linesize, bytesremaining)]

and then I end up with a big string. It's a string of raw bytes (not ASCII) which I have to convert to 32 bit integers. I'm hung up on this last step. I hope this makes sense what I'm asking. Thanks.

share|improve this question
    
store it as a StringIO.StringIO and use genfromtext –  Padraic Cunningham 3 hours ago

1 Answer 1

Instead of writing the data to a file, assemble the data into a single string, and convert it to an array with numpy.fromstring

share

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.