Take the 2-minute tour ×
Mathematica Stack Exchange is a question and answer site for users of Mathematica. It's 100% free, no registration required.

I'm trying to use Compile[] to improve the performance of an algorithm. The algorithm works on a set of matrices which are not of a similar shape. Unfortunately to use the Compile function, lists passed to it must be of similar shape.
To circumvent this inconvenience, I propose to Flatten the set of matrices before passing the resultant flat vector to the compiled function and then reconstructing the vector into the set of matrices before applying the algorithm. Sounds simple but actually implementing it in Mathematica is proving difficult (for me) - help appreciated.

The general gist of what I want to do is as follows:

(* Generate a set of irregular shaped matricies *)
mSet = {Table[RandomReal[], {i, 3}, {j, 5}], Table[RandomReal[], {i, 5}, {j, 4}], Table[RandomReal[], {i, 4}, {j, 6}]};

(* Create a mapping that describes the structure of the set of matrices *)
mStruct = FunctionToGenerateVectorThatDescribesTheStructure[mSet]

(* myCompiledFunc uses mStruct to reassemble the flattened mSet before doing it's thing *)
myCompiledFunc = Compile[{{setOfMats, _Real, 1}, {matStructure, _Integer, 1}},
    (* Reconstruct the setOfMats *)
    ReconstructedSetOfMats = someReconstructionFunction[setOfMats, matStructure];
    (* do stuff with the reconstructed matricies *)
];


(* Use the compiled function *)
(* Flatten the set of matricies so we can pass it and the structure to a compiled function *)
myCompiledFunc[Flatten[mSet], mStruct];    
share|improve this question
2  
Perhaps you can use the answers in Unflattening a list? –  kguler yesterday
1  
If you attempt to reconstruct the non-tensor object inside Compile it will generate a callback to the main evaluator. –  Simon Woods yesterday
    
Would you please give an example of the type of operation you wish to perform on the reconstructed matrices? –  Mr.Wizard 17 hours ago
    
Hi Mr.Wizard. I purposely left out the detail of what was inside the compiled function as it would no doubt generate a whole separate topic of debate - it basically has a number of nested For loops with a number of sequential matrix operations per iteration of the loop. Each subsequent iteration depends on the result of the previous iteration and each iteration requires modification of the shape of the resultant matrix from the calculations. I felt that adding this detail would have been a distraction from my main question. Happy to add it or start another question if you would like. –  CrustyNoodle 15 hours ago
add comment

2 Answers

This is in case of set of 2D arrays. I hope I've not missed the point.

mSet = {RandomReal[1, {3, 5}], RandomReal[1, {5, 4}], RandomReal[1, {4, 6}]};


reco[flatten_, dims_] := Composition[
  MapThread[Partition, {#, dims[[ ;; , 2]]}] &,
  Take[flatten, #] & /@ # &,
  Transpose,
  {Most[Join[{0}, #]] + 1, #} &,
  Accumulate
  ][
    Times @@@ dims]

 reco[Flatten[mSet], Dimensions /@ mSet] == mSet
True
share|improve this answer
    
Nice style with Composition. I'll have to replace the horrible @-chains I currently write sometimes with this idiom. Actually, also makes me think of x // Reverse@Composition[h, g, f] as a nice idiom closer to F#'s forward-pipe operator, which I like. Anyway, +1 –  billisphere 20 hours ago
    
indeed, Composition is great for code readability :) I'm glad you like it. Tell me if it works at the end, I have not compiled it, I don't compile almost at all so I left here only an approach. :) –  Kuba 20 hours ago
add comment

I'll assume your list of 2D examples, this can easily be extended to arbitrary dimensions, and for that matter to a list with elements of differing depths. On a quick test using

Table[RandomInteger[100, {RandomInteger[{50, 100}], RandomInteger[{50, 100}]}], {2000}];

to generate 2000 randomly sized 2D arrays, over 30X faster than reco:

ranger[list_, lens_] := With[{x = Accumulate@lens},
  Inner[list[[# ;; #2]] &, Most@Prepend[x, 0] + 1, x, List]]

Use example:

test = {RandomReal[10, {2, 3}], RandomReal[20, {5, 5}]};

dims = {{3, 3, 5, 5, 5, 5, 5}, {2, 5}};

target = Flatten[test];

Fold[ranger, target, dims] == test

(* True *)

Note the "dimensions" argument is specified from the "bottom" up, per construction element. This allows, among other things, the targets themselves to be ragged.

Btw- ranger is simply a ragged partitioner I built long ago: given a flat list and a list of lengths, it returns the original list partitioned by the lengths. IIRC, faster than the (undocumented) built-in.

share|improve this answer
1  
Regarding ranger see this Q&A and this comment and link. As I like to say to Leonid in these cases: "Great minds think alike." :-) –  Mr.Wizard 18 hours ago
    
@Mr.Wizard: Ah, that's neat - favorited. There's actually a way using Extract that can be marginally faster, but uses a weird (bug?) "feature" of extract for pulling spans. Thanks for links! –  rasher 17 hours ago
    
Please tell me about this Extract behavior. I hope you will make use of my dynamicPartition function; I put some thought into its construction, e.g. partitioning non-List expressions, and I find it quite useful myself. And please let me know if you can improve it! –  Mr.Wizard 17 hours ago
2  
@Mr.Wizard: Try, e.g., target = Join[Range[100], {{1, 2, 3, Range[20]}}] Rest@Extract[target, {{{}}, {2 ;; 10 ;; 2}, {-1, -1, 1 ;; -1 ;; 2}}] - note the use of Rest and prepended {{}}. This suppresses the error you get normally trying to use spans, and surprisingly allows you to use pretty much and valid part/span spec to get pieces of lists/sublists/etc. returned as a list of results. I've found it faster than say mapping over a list of spans, but don't use it in important code since using "weird" constructs gives me the Willies... –  rasher 17 hours ago
1  
Interesting; in version 7 I just get Extract::argtu: Extract called with 1 argument; 2 or 3 arguments are expected. >> and the expression unevaluated. I suspect they are extending Extract but for some reason the functionality wasn't finalized. MapAt was silently extended to work with Span as well. You should post a self-Q&A about this behavior in my opinion. I bet it will get a lot of votes. –  Mr.Wizard 17 hours ago
show 1 more comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.