How to use a matrix variable for a compiled function



I want to compile the following function which takes a matrix variable.

ClusterFind = 
Compile[{{Inte, _Real}, {Matrix}}, 
Map[If[Last@# == Inte, Take[#, 4]] &, Matrix], CompilationTarget -> "C",
RuntimeAttributes -> {Listable}, Parallelization -> True];

Here Matrix is a $m\times n$ matrix of real,integer or complex numbers. I also would like to know if it is possible to compile without knowing the dimension of the variable matrix beforehand. Can the Matrix in the above example be of PackedArray data type?


Thanks for the suggestions. I found the following compiled version most efficient as far as speed is concerned.

ClusterFind = 
Compile[{{Inte, _Real}, {Matrix, _Complex, 2}}, 
Take[Select[Matrix, (Last@# == Inte) &], All, 4], 
CompilationTarget -> "C"];



Posted 2012-02-02T14:39:43.243

Reputation: 13 888

On my machine, @Oleksandr's version is faster than the one you added in your edit. Is that not so on yours? – acl – 2012-02-03T17:43:39.977

@acl I gave sixteen runs for clustering with sixteen integers from 2 to 16 and I took a complex matrix of dimension {750000, 5}. I tried with my function and that of Oleksandr's. Mine took 3.54 sec. and his took 4.98 sec. Funny that u get opposite results. – PlatoManiac – 2012-02-03T19:09:46.563

Sorry! I meant 15 runs.. – PlatoManiac – 2012-02-03T19:16:14.557

Sorry; I didn't understand from the wording you used in the original question that the matrix had only five columns. I had assumed it would have more, so I extracted the five needed columns into a temporary matrix to avoid copying the larger matrix. As it turns out this step was unnecessary, so yes, your version without the extra operation should indeed be faster. (@acl, if you found mine faster, I guess you also assumed the matrices had more than 5 columns?) – Oleksandr R. – 2012-02-06T23:04:25.977

@OleksandrR yes i tried for randomly chosen sizes – acl – 2012-02-06T23:19:02.417



Not only can Matrix be a PackedArray, it must be a PackedArray. However, it will be packed for you if necessary before the compiled code is called.

The following code is substantially faster than that given by acl above and does not require any post-processing of the output, but is still sub-optimal in terms of requiring CopyTensor calls and using a rather larger working set than one would think necessary. Perhaps these limitations can be lifted, but after a brief survey of possible implementations I didn't find a way better than this (though note that I didn't try anything with Internal`Bag).

clusterFind = Compile[{{inte, _Real, 0}, {matrix, _Complex, 2}},
    Module[{tmp = matrix[[All, {1, 2, 3, 4, -1}]]},
        Select[tmp, Last[#] == inte &][[All, ;; -2]]
    ], RuntimeAttributes -> Listable, Parallelization -> True

An example of the improved timings:

range = {0, 10};
data = RandomInteger[range, {1*^5, 100}];

acl's version:

    cf4[RandomInteger[range], data];

producing: {2.297, Null}

My version:

    clusterFind[RandomInteger[range], data];

which gives {0.047, Null}.

Note that the above timings are not for C-compiled versions of the two functions; compilation to C does not help very much as there is not much computational work to be done in this process anyway and most of the timing consists of copying or extracting parts of tensors. Also, I should mention that RuntimeAttributes -> Listable and Parallelization -> True do not really buy you anything here unless you are operating on a list of matrices.

Oleksandr R.

Posted 2012-02-02T14:39:43.243

Reputation: 22 073

But you should compare to my cf2 (with Select), not cf4. Your version is then still faster, but by 50%. – acl – 2012-02-03T10:06:59.063

1Yes, you're right. Sorry, I only compared against the compiled one, so I didn't notice that this wasn't the best performing of your suggestions. – Oleksandr R. – 2012-02-06T23:06:46.550



ClusterFind = Compile[{{Inte, _Real}, {Matrix, _Complex, 2}},
    If[Last@# == Inte, Take[#, 4]] &, Matrix],
   CompilationTarget -> "C",
   RuntimeAttributes -> {Listable},
   Parallelization -> True

should take care of the matrix. However, If[Last@# == Inte, Take[#, 4]] does not always return the same type (sometimes it returns nothing, right?)


something like

cf2[inte_, matrix_] := 
 Select[matrix, Last[#] == inte &][[All, 1 ;; 4]]


cf3[inte_, matrix_] := 
 matrix[[#, 1 ;; 4]] & /@ Position[matrix[[All, -1]], inte]

might do what you want (although I don't know what form your output should have). I don't know if it is fast enough, though. If not perhaps maybe something compilable to C might be set up.


This works and compiles, but is suboptimal:

cf4 = Compile[{{Inte, _Real}, {Matrix, _Complex, 2}},
    {lst = Most@{1. + I}},
       Last@# == Inte,
       lst = Join[lst, Take[#, 4]]
       ] &,
   CompilationTarget \[Rule] "C"

(I stole the Most[{1.+I}] thing from halirutan). You'll need to use Partition[#,4] on the output (if I am guessing your desired output form correctly).


Posted 2012-02-02T14:39:43.243

Reputation: 19 146

Take is compilable. – rcollyer – 2012-02-02T14:49:30.940

@rcollyer ah yes! hm... – acl – 2012-02-02T14:51:42.837

That list is very nice. :) – rcollyer – 2012-02-02T14:52:11.483

@rcollyer I even have a printout of the sorted list in grid format on my office wall. And, now that I think of it, I routinely use Take in my own compiled code. So, I really have no excuse :) – acl – 2012-02-02T14:53:43.283

@acl You are right it often will return Null – PlatoManiac – 2012-02-02T15:02:38.573