up vote 0 down vote favorite

I've run into a situation where I have binary data (created using os.urandom(BLOCK_SIZE)) that I need to store in a Postgres database using Django Models.

I've read several references to how Django doesn't yet support Bytea (VARBINARY) field types yet. I found a reference here but don't know where to get that library (django_postgresql).

I'm currently using Python 2.7 and Django 1.4 on Ubuntu 12.04 - if that helps.

I've read some suggestions as a way around this is to convert my data to hex before storing it.

import binascii
key = binascii.hexlify(value)

Ok, now what? I tried to save this value to the database but get errors:

invalid byte sequence for encoding "UTF8": 0xd6c2

Ok, which data field type should I use to do this?

key = models.TextField(max_length = 200)?

key = models.CharField(max_length = 200)?

key = models.???(max_length = 200)???

I'd like to stay away from creating my own custom field type. First, I'm pretty new to Django and wouldn't feel comfortable doing this yet. Second, I'm pretty new to databases so I'm not sure where to start on defining something like this.

Any suggestions?

share|improve this question
What is this data being used for and how big is BLOCK_SIZE and is it likely to change? – Jon Clements Jul 5 '12 at 20:28
@ Jon Clements This data represents a key for cryptography. BLOCK_SIZE in this case is 32 Bytes (os.urandom(32)). The BLOCK_SIZE will not change. – Rico Jul 5 '12 at 20:32
Check if type(key) is str or unicode. You may need to key.encode('ascii') if unicode... – Jon Clements Jul 5 '12 at 20:41
feedback

1 Answer

up vote 1 down vote accepted

If you get unicode errors when you save the key value, you're doing it wrong. binascii.hexlify() returns a simple string of hex characters, which will not give you the "invalid byte sequence".

For more efficient storage, use base64 encoding:

import base64
mymodelinstance.key = base64.b64encode(raw_key_value)

And to decode:

raw_key_value = base64.b64decode(mymodelinstance.key)

You can just use a CharField of length BLOCK_SIZE * 2 to hold hex data, or 4 * ( BLOCK_SIZE / 3) for base64.

share|improve this answer
Thank you, this solved my problem. You were right about the error, I was doing it wrong. In short, it was just a syntax error when writing to the database. – Rico Jul 5 '12 at 21:15
feedback

Your Answer

 
or
required, but never shown
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.