Has compilation of big programs become excessively sluggish in v 10.4?

12

1

Bug introduced in 10.4 and fixed in 11.0


After updating to 10.4, many of my heavily numerical functions defined by Compile takes a huge amount of time to compile to Wolfram Virtual Machine.

As an example, after running the definitions in this question to compute the dilogarithm function, observe how long it takes to compile the following function in Mathematica 10.4, and in previous versions (10.3 and earlier):

f = Compile[{{x, _Real}, {a, _Real}, {b, _Real}, {c, _Real}}, 
   reLi2[((x - b^2 + a^2) (a^2 - c^2))/(x a^2 - (b^2 - a^2) (a^2 - c^2))] - 
   reLi2[-(((b^2 - a^2) (a^2 - c^2))/(x a^2 - (b^2 - a^2) (a^2 - c^2)))] + 
   reLi2[-(((b^2 - a^2) a^2)/(x a^2 - (b^2 - a^2) (a^2 - c^2)))] - 
   reLi2[-((a^2 a^2)/(x a^2 - (b^2 - a^2) (a^2 - c^2)))] + 
   reLi2[((x - a^2 + c^2) (c^2 - b^2))/(x c^2 - (a^2 - c^2) (c^2 - b^2))] - 
   reLi2[-(((a^2 - c^2) (c^2 - b^2))/(x c^2 - (a^2 - c^2) (c^2 - b^2)))] + 
   reLi2[-(((a^2 - c^2) c^2)/(x c^2 - (a^2 - c^2) (c^2 - b^2)))] - 
   reLi2[-((c^2 c^2)/(x c^2 - (a^2 - c^2) (c^2 - b^2)))] - 
   reLi2[((x - (a^2)^2 + (c^2)^2) ((c^2)^2 - (b^2)^2))/(x (c^2)^2 - ((a^2)^2 - (c^2)^2) ((c^2)^2 - (b^2)^2))] - 
   reLi2[-((((a^2)^2 - (c^2)^2) ((c^2)^2 - (b^2)^2))/(x (c^2)^2 - ((a^2)^2 - (c^2)^2) ((c^2)^2 - (b^2)^2)))] + 
   reLi2[-((((a^2)^2 - (c^2)^2) (c^2)^2)/(x (c^2)^2 - ((a^2)^2 - (c^2)^2) ((c^2)^2 - (b^2)^2)))] - 
   reLi2[-(((c^2)^2 (c^2)^2)/(x (c^2)^2 - ((a^2)^2 - (c^2)^2) ((c^2)^2 - (b^2)^2)))], 
 CompilationOptions -> {"InlineExternalDefinitions" -> True}]

My MacOS X 10.10$^†$ computer takes 3 seconds on Mathematica 10.4, and nearly instantaneous on Mathematica 9.0.1. The actual functions used in my code contains over 100 instances of reLi2 in the body of Compile, and it takes over a minute just to compile on version 10.4.

Does anyone else observe this problem? Are there any workarounds? Setting option "ExpressionOptimization"->False helps just a tad, but not nearly enough.

$^†$ update: I also observe the problem in Linux.

QuantumDot

Posted 2016-03-17T15:43:30.660

Reputation: 18 597

1Something spooky: when you put semicolon ; at end of the command to "suppress the output" compilation goes quickly like in earlier versions. This semicolon trick does not work if this code is called from a package file. – QuantumDot – 2016-03-17T18:58:20.277

1It looks like it's the typesetting that takes all the time. Try using SetSystemOptions["TypesetOptions" -> {"IconicElidedForms" -> False}] – Simon Woods – 2016-03-17T21:54:43.037

@SimonWoods Thanks, this seems to solve some of the cases, but not all. In particular, if the body of the Compile is constructed using With for code injection, it still performs much slower than earlier versions. – QuantumDot – 2016-03-18T08:18:25.693

This problem has been reported [CASE:3559742]. – QuantumDot – 2016-03-18T21:01:22.100

Thank you for the bug report, the compilation (not the formatting) issue is under investigation. – ilian – 2016-03-18T22:32:32.237

@ilian It may not really be appropriate to ask this, but do you think you could suggest a workaround? The 90s wait for compiling a function is driving me crazy... thanks – QuantumDot – 2016-05-31T22:09:12.333

@QuantumDot Unfortunately there is no workaround for 10.4.x, however this has been fixed in version 11 (which is not too far away). Let me know if you would be interested in trying out a prerelease build. – ilian – 2016-06-01T00:58:31.670

@ilian Since I'm a package developer, I think it would be nice to get a preview of what's to come with a prerelease build. Thanks for the offer; what's the proper way to request for one? – QuantumDot – 2016-06-04T14:43:50.360

1@QuantumDot I have forwarded your contact details to our prerelease program coordinators and I hope they will be in touch soon with more information. – ilian – 2016-06-04T15:52:48.583

Answers

2

As of version 11, the bug has been fixed. Compile now performs like it did in version 10.3 and earlier.

QuantumDot

Posted 2016-03-17T15:43:30.660

Reputation: 18 597

5

That 3 seconds in 10.4 is an improvement over 5.3 s in 10.3. :)

I can't imagine any scenario where how you construct the function affects its formatting (assuming you're creating the same). I guess the fact that it isn't helped by disabling iconic elided forms suggests the issue is elsewhere. Also, does disabling the outuput (using a semi-colon) still result in a slowdown? That means the slowdown is either in evaluation or in the transmission over MathLink to the kernel. If you have a concrete example, that would be helpful.

The new iconic forms were made interpretable, meaning if you copy an output into an input cell (or hit Cmd/Ctrl-Shift-L), it will parse and evaluate correctly. This means the formatted forms are much larger in size, since they contain complete info for reconstructing the CompiledFunction, and hence notebooks containing them in output will be much larger. So you have a lot of these in output cells, you'll have much bigger notebooks. You might want to disable the icon forms or suppress output to save space (and the time saving the notebook) in addition to the speed up from not formatting them.

Itai Seggev

Posted 2016-03-17T15:43:30.660

Reputation: 12 396

I have reported this problem [CASE:3559742], and have notified technical support that the problem is urgent because compilation time in my package is now over 90 seconds with v10.4, when it was well under 1 second in v10.3 and earlier. While evaluation speeds of Compile called directly within the notebook can be improved with a semicolon or by modifying SystemOptions, nothing seems to be possible to improve compilation times from within a package file. – QuantumDot – 2016-03-18T20:59:36.193

You are confusing 2 issues. An input is first evaluated and then--if displayed--formatted. A Compile in a notebbok and in a package file evaluate in the same amount of time. If you allow it to format, that formatting takes a certain amount of time--but Get doesn't output a result to be formatted. Formatting CompileFunction takes longer in 10.x than 9, which is addressed by the SystemOption or suppressing output. Your particular compile statement exhibits a new in 10.4 slowdown during evaluation itself. This seems to be an unexpected change from a bugfix, but is still under investigation. – Itai Seggev – 2016-03-19T00:49:29.550

Thanks for the clarification. I wait anxiously for a fix/workaround. – QuantumDot – 2016-03-19T08:23:06.377