Why Function is not identical to `&` in autocompilation?



Here is the example, copied from here

square = Function[x, x^2];
square1 = #^2 &;

the timing and unpacking status shows

data = RandomReal[{0, 10}, {10000}];

AbsoluteTiming[Developer`PackedArrayQ[Map[square, data]]]
AbsoluteTiming[Developer`PackedArrayQ[tmp1 = Map[square1, data]]]

{0.000771589, True}
{0.000748647, True}

Now we add external variable into these two definition.

a = 1
square = Function[x, x^2 + a];
square1 = #^2 + a &;

and time it again, you got

{0.0336384, False}
{0.0062035, True}

we can see & is still autocompiled, while Function is not. Why? I think the documentation treats them as identical way of writing. This distinction is oddly subtle, I just found it today. What is bad is that without Function, we can not give parameters names, thus less readability.


Posted 2016-12-03T05:54:45.957

Reputation: 14 483

To be precise, it seems that square = Function[x, x^2 + a]; isn't compared in Map. I think this can be considered as another evidence that the auto-compilation of pure function is still not that stable currently. – xzczd – 2016-12-03T06:27:46.403

1It's not just about autocompilation. square is slower than square1 without compilation. You can use Trace to see that there is a significant difference in the evaluation. Autocompilation can be switched of with SetSystemOptions["CompileOptions" -> {"MapCompileLength" -> \[Infinity]}] and increases the difference in your second case, while it removes the difference in your first case. – Karsten 7. – 2016-12-03T07:22:19.450

Hi, @Karsten7. I think it is might not be the point. The goal to use Function is to make use of autocompilation. When & and Function both been autocompiled, they are of same speed. If speed is not of concern, then I can just use square2[x_] := x^2;, and in case of "MapCompileLength" -> \[Infinity], square2[x_] := x^2 is actually faster than Function. – matheorem – 2016-12-03T07:47:14.457

3square2 = With[{a = a}, Function[x, x^2 + a]] would autocompile properly. Maybe "InlineExternalDefinitions" is handled differently? – Karsten 7. – 2016-12-03T07:50:41.610

@Karsten7. What difference in the evaluation do you see? For me Trace@Map[square1, data] and Trace@Map[square, data] look identical in the sense of the evaluation. – Alexey Popkov – 2016-12-03T07:51:13.023

3SetSystemOptions[ "CompileOptions" -> {"CompileReportFailure" -> True, "InternalCompileMessages" -> True}] can be used to switch on the generation of an error message. – Karsten 7. – 2016-12-03T07:51:14.793

@Karsten7. good advice on using With, thanks! – matheorem – 2016-12-03T07:52:17.390

@Karsten7. So your setting generate a message "Compilation of Function[x,x^2+a]/@Compile`AutoVar12 failed because a was not a form suitable for the compiler", What does it mean? – matheorem – 2016-12-03T08:12:04.843

I wouldn't use Timing. Either AbsoluteTiming or RepeatedTiming. Even if it might not make a difference here, it's unreliable on recent computer systems. See http://mathematica.stackexchange.com/a/14159/4999

– Michael E2 – 2016-12-03T12:59:43.737

@MichaelE2. Thanks for reminding. Normally, I use AbsoluteTiming too. Since the code is copied, unfortunately, I didn't noticed it. I have fixed it now : ) – matheorem – 2016-12-03T13:17:47.430



The solution, as suggested by Karsten7, in a comment is to use With.

data = RandomReal[{0, 10}, {10000}];

a = 1;
With[{a = a}, square = Function[x, x^2 + a]];
square1 = #^2 + a &

Timing[Developer`PackedArrayQ[Map[square, data]]]
Timing[Developer`PackedArrayQ[tmp1 = Map[square1, data]]]

{0.000578, True}

{0.000528, True}


Posted 2016-12-03T05:54:45.957

Reputation: 104 223

1The question asks why, perhaps we could elucidate the underlying reason. – M.R. – 2017-01-10T03:07:27.253