MapIndexed and Compile



I am having trouble compiling a function that uses the MapIndexed function on the second level. The compilation runs fine, but at evaluation time I get an error that I cannot explain right now.

The following scenario shows the problem:

First I define a compiled version and a noncompiled version of a simple funciton using MapIndexed:

MapIndexedCompileTest = Compile[{{list1, _Real, 2}, {list2, _Real, 2}}, 
                          MapIndexed[list1[[#2[[1]], #2[[2]]]] + #1 &, list2, {2}]]

MapIndexedTest[list1_, list2_] := MapIndexed[list1[[#2[[1]], #2[[2]]]] + #1 &, list2, {2}]

MapIndexedTest shows that the function works just fine:

MapIndexedTest[{{1, 2}, {1, 2}}, {{1, 2}, {1, 2}}]

But MapIndexedCompiledTest yields the error:

"CompiledFunction::cflist: Nontensor object generated; proceeding with 
 uncompiled evaluation" 

when evaluated the same way:

MapIndexedCompileTest[{{1, 2}, {1, 2}}, {{1, 2}, {1, 2}}]

Does anybody know what goes wrong?

New intel I just got when reading answers:

The problem seems to be that the index represented by the second argument is not correct when using compiled. To see that I defined the following functions, which should just return the current index inside the Mapping function:

MapIndexedGiveIndexCompiled = Compile[{{p, _Real, 2}}, MapIndexed[#2 & , p, {2}]];
MapIndexedGiveIndex[p_] := MapIndexed[#2 & , p, {2}];

The above functions should obviously return the same results when used on a 2 rank tensor, but they do not, MapIndexedGiveIndexCompiled delivers the wrong result:

MapIndexedGiveIndexCompiled[{{1, 2}, {3, 4}}]


{{{1}, {2}}, {{1}, {2}}}

while MapIndexedGiveIndex returns the expected result:

{{{1, 1}, {1, 2}}, {{2, 1}, {2, 2}}}

This should explain why [[2]] is not working, but overall it looks like a bug to me.


Posted 2013-09-03T12:52:06.827

Reputation: 2 640

Use #2[[-1]] or Last@#2 instead of #2[[2]] is OK. – chyanog – 2013-09-03T13:39:40.457

Although it's possible this is a bug I think it is more likely that it isn't fully supported. There are other functions that only work properly in Compile in a limited fashion. Thanks for bringing this to my attention. – Mr.Wizard – 2013-09-05T15:19:18.613

1Suspect it is a bug in the sense that it didn't compile into something equivalent. Ordinarily, non-compileable code should produce a message to that effect and be handled using a callback to the main evaluator. In this case it seems that Compile thought it could compile your code, but actually it can't. halirutan's answer suggests that this is a problem with the type inference mechanism. @Mr.Wizard – Oleksandr R. – 2013-09-05T21:50:10.737

@Oleksandr On your expertise with Compile I am going to tag this as a bug. – Mr.Wizard – 2013-09-06T00:48:55.880



It seems this has something to do with the element access of the anonymous function argument #2. One possible solution which additionally makes it easier to read is

f = Compile[{{list1, _Real, 2}, {list2, _Real, 2}},
   With[{x = First[#2], y = Last[#2]},
     list1[[x, y]] + #1] &, list2, {2}]]

What I don't understand is, that you could do something even more verbose by defining two functions. The first one does exactly what you do inside the MapIndexed

f1 = Compile[{{v1, _Real, 2}, {v, _Real, 0}, {p, _Integer, 1}},
  Module[{x = p[[1]], y = p[[2]]},
   v1[[x, y]] + v

And you will see that e.g.

ff[{{1, 2}, {3, 4}}, 3, {1, 2}]

runs without complains. But if you incorporate it into the other function

f = With[{f1 = f1},
  Compile[{{list1, _Real, 2}, {list2, _Real, 2}}, MapIndexed[
    f1[list1, #1, #2] &, list2, {2}],
   CompilationOptions -> {"InlineCompiledFunctions" -> True}

you get the error message again. When you change the element access in f1 to First and Last, then it works again. This suggests, that compile cannot properly determine that #2 is indeed a tensor where #2[[2]] is valid but the situation remains weird.


Posted 2013-09-03T12:52:06.827

Reputation: 109 574

Thanks for your hints. Further investigations brought me to the additional information I now included in the question above. The compiled version seems to give the wrong index. Any idea how to fix it, or is it actually a bug? – Wizard – 2013-09-03T17:48:04.987