up vote 2 down vote favorite

I have following String that I have put together:

v1fColor = '2,4,14,5,0,0,0,0,0,0,0,0,0,0,12,4,0,0,0,0,0,0,0,0,0,0,0,0,0,0,15,6,0,0,0,0,1,0,0,0,0,0,0,0,0,0,20,9,0,0,0,2,2,0,0,0,0,0,0,0,0,0,13,6,0,0,0,1,0,0,0,0,0,0,0,0,0,0,10,8,0,0,0,1,2,0,0,0,0,0,0,0,0,0,17,17,0,0,0,3,6,0,0,0,0,0,0,0,0,0,7,5,0,0,0,2,0,0,0,0,0,0,0,0,0,0,4,3,0,0,0,1,1,0,0,0,0,0,0,0,0,0,6,6,0,0,0,2,3'

I am treating it as a vector: Long story short its a forecolor of an image histogram:

I have the following lambda function to calculate cosine similarity of two images, So I tried to convert this is to numpy.array but I failed:

Here is my lambda function

import numpy as NP
import numpy.linalg as LA
cx = lambda a, b : round(NP.inner(a, b)/(LA.norm(a)*LA.norm(b)), 3)

So I tried the following to convert this string as a numpy array:

v1fColor = NP.array([float(v1fColor)], dtype=NP.uint8)

But I ended up getting following error:

    v1fColor = NP.array([float(v1fColor)], dtype=NP.uint8)
ValueError: invalid literal for float(): 2,4,14,5,0,0,0,0,0,0,0,0,0,0,12,4,0,0,0,0,0,0,0,0,0,0,0,0,0,0,15,6,0,0,0,0,1,0,0,0,0,0,0,0,0,0,20,9,0,0,0,2,2,0,0,0,0,0,0,0,0,0,13,6,0,0,0,1,0,0,0,0,0,0,0,0,0,0,10,8,0,0,0,1,2,0,0,0,0,0,0,0,0,0,17,17,
share|improve this question

4 Answers

up vote 5 down vote

You have to split the string by its commas first:

NP.array(v1fColor.split(","), dtype=NP.uint8)
share|improve this answer
3  
I just learned that numpy will do the string conversions implicitly a few weeks ago. I think that's the coolest thing ever. – mgilson Jul 31 '12 at 19:07
That's a great tip! – David Robinson Jul 31 '12 at 19:14
up vote 5 down vote

You can do this without using python string methods -- try numpy.fromstring:

>>> numpy.fromstring(v1fColor, dtype='uint8', sep=',')
array([ 2,  4, 14,  5,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0, 12,  4,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0, 15,  6,  0,  0,
        0,  0,  1,  0,  0,  0,  0,  0,  0,  0,  0,  0, 20,  9,  0,  0,  0,
        2,  2,  0,  0,  0,  0,  0,  0,  0,  0,  0, 13,  6,  0,  0,  0,  1,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0, 10,  8,  0,  0,  0,  1,  2,
        0,  0,  0,  0,  0,  0,  0,  0,  0, 17, 17,  0,  0,  0,  3,  6,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  7,  5,  0,  0,  0,  2,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  4,  3,  0,  0,  0,  1,  1,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  6,  6,  0,  0,  0,  2,  3], dtype=uint8)
share|improve this answer
(+1) I think this is the way to go here. (I actually just learned about fromstring today through a different question) – mgilson Aug 1 '12 at 17:51
up vote 5 down vote

You can do this:

lst = v1fColor.split(',')  #create a list of strings, splitting on the commas.
v1fColor = NP.array( lst, dtype=NP.uint8 ) #numpy converts the strings.  Nifty!

or more concisely:

v1fColor = NP.array( v1fColor.split(','), dtype=NP.uint8 )

Note that it is a little more customary to do:

import numpy as np

compared to import numpy as NP

EDIT

Just today I learned about the function numpy.fromstring which could also be used to solve this problem:

NP.fromstring( "1,2,3" , sep="," , dtype=NP.uint8 )
share|improve this answer
Sure thanks, I will add that change too.. by the way really tough case, because both answers are great and I am not sure which one to pick... :) – Null-Hypothesis Jul 31 '12 at 19:09
Neither of us would take it personally :) – David Robinson Jul 31 '12 at 19:12
1  
I get that a lot. Pick @mgilson, he knew about the implicit conversion to a float. – David Robinson Jul 31 '12 at 19:15
1  
@DavidRobinson sigh you were supposed to just go with it ... Thanks though. – mgilson Aug 1 '12 at 1:40
1  
@Null-Hypothesis -- what's wrong? looking at the edit history, I haven't really changed anything as far as content is concerned. The version that DavidRobinson posted was something like np.array(map(int,v1fColor.split(','))), but that should be equivalent to what I posted. You can always change the dtype from np.uint8 to int, or np.int32 ... (np.uint8 should be limited to the range 0-255). – mgilson Aug 1 '12 at 17:40
show 8 more comments
up vote 0 down vote accepted

I am writing this answer so if for any future references: I am not sure what is the correct solution in this case but I think What @David Robinson initially publish was the correct answer due to one reason: Cosine Similarity values can not be greater than one and when I use NP.array(v1fColor.split(","), dtype=NP.uint8) option I get strage values which are above 1.0 for cosine similarity between two vectors.

So I wrote a simple sample code to try out:

import numpy as np
import numpy.linalg as LA

def testFunction():
    value1 = '2,3,0,80,125,15,5,0,0,0,0,0,0,0,0,0,0,0,0,0,2,4,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,4,3,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0'
    value2 = '2,137,0,4,96,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0'
    cx = lambda a, b : round(np.inner(a, b)/(LA.norm(a)*LA.norm(b)), 3)
    #v1fColor = np.array(map(int,value1.split(',')))
    #v2fColor =  np.array(map(int,value2.split(',')))
    v1fColor = np.array( value1.split(','), dtype=np.uint8 )
    v2fColor = np.array( value2.split(','), dtype=np.uint8 )
    print v1fColor
    print v2fColor
    cosineValue = cx(v1fColor, v2fColor)
    print cosineValue

if __name__ == '__main__':
    testFunction()

if you run this code you should get the following output: enter image description here

Not lets un commented two lines that and run the code with the David's Initial Solution:

v1fColor = np.array(map(int,value1.split(',')))
v2fColor =  np.array(map(int,value2.split(',')))

Keep in mind as you see above Cosine Similarity Value came up above 1.0 but when we use the map function and use do the int casting we get the following value which is the correct value:

enter image description here

Luckily I was plotting the values that I was initially getting and some of the cosine values came above 1.0 and I took the outputs of these vectors and manually typed it in python console, and send it via my lambda function and got the correct answer so I was very confuse. Then I wrote the test script to see whats going on and glad I caught this issue. I am not a python expert to exactly tell what is going on in two methods to give two different answers. But I leave that to either @David Robinson or @mgilson.

share|improve this answer
1  
quick comment. In the "david" form of solution, do something like np.array(map(int,v1fColor.split(',')),dtype=np.uint8)) and I'm guessing you'll get the same thing you got in my solution. I'm guessing the problem is the data-type (which we were just preserving from your original question). What's probably happening is when you take the inner product of the two arrays, you're getting numbers greater than 255 -- e.g. (2*128) will probably result in 0 – mgilson Aug 1 '12 at 18:08

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.