How can I prevent a compiled function from using uncompiled evaluation?



I write a code that creates a compiled function, and then call that function over and over to generate a list. I run this code on a remote server via a batch job, and will run several instances of it. Sometimes when I make changes to the code, I make a mistake, and inside the compiled function is an undefined variable, such that when the function is called I get the following error messages (repeated several times)

 CompiledFunction::cfse: Compiled expression w should be a machine-size complex number.

 CompiledFunction::cfex: Could not complete external evaluation at instruction 18; proceeding with uncompiled evaluation.

This causes massive memory usage (which puts me on the system administrator's bad side), and the results are garbage if since there was a mistake in the code. Is there any way to force the code to abort and quit the program rather than proceed with uncompiled evaluation?

Jason B.

Posted 2014-07-21T19:42:23.467

Reputation: 58 546

1you can also do fc = Compile[{{z, _Real}}, Sin[z]]; f[x_?NumericQ] := fc[x] – acl – 2014-07-21T20:07:33.530

But that only fixes it if the function argument isn't numeric. My mistake involved an undefined variable inside the expression I feed to Compile. – Jason B. – 2014-07-21T20:15:24.597

OK, I thought you were worried about passing the wrong argument, not actually forgetting to localise your variables. – acl – 2014-07-21T20:16:36.280



Adding this option for Compile

"RuntimeOptions" -> {"RuntimeErrorHandler" ->Function[Throw[$Failed]]}

will cause it to abort evaluation if any error messages come up. To more directly control the memory usage, and stay on the sysadmin's good side, wrap the call to the compiled function with MemoryConstrained, which causes it to abort if the memory goes above a certain threshold.

Jason B.

Posted 2014-07-21T19:42:23.467

Reputation: 58 546


You can add

RuntimeOptions -> {"EvaluateSymbolically" -> False}

to your Compile function.
Consult RuntimeOptions for more details.

Karsten 7.

Posted 2014-07-21T19:42:23.467

Reputation: 26 728

Using version 9,this doesn't seem to change anything. I added that option to Compile but I get the same error message. – Jason B. – 2014-07-21T20:00:13.950

I don't know why that didn't work, but I looked into RuntimeOptions and found this, which seems to work just fine: "RuntimeOptions" -> {"RuntimeErrorHandler" ->Function[Throw[$Failed]]} Thanks for your help – Jason B. – 2014-07-21T20:05:07.187

This doesn't remove the mistake in your code, but it should solve your memory issue. – Karsten 7. – 2014-07-21T20:06:10.993

Totally, I still have to go in and fix the issue, but this stops it from grabbing all the memory in the (inevitable) chance that I make a mistake like this. – Jason B. – 2014-07-21T20:16:13.030

@JasonB, please add that solution as an answer - it is valuable and shouldn't be hidden away in a comment. – Simon Woods – 2014-07-21T20:22:31.517

@JasonB, does using the RuntimeErrorHandler also solve your memory problem? – Karsten 7. – 2014-07-21T20:27:33.427

1@Karsten7., yes it does since it doesn't try to populate the list symbolically. – Jason B. – 2014-07-21T20:31:01.363

3@JasonB Are you aware of SetSystemOptions["CompileOptions" -> "CompileReportExternal"->True]? If not, it might be useful in your case (it warns you at compile time if you reference an external variable). I might have misunderstood the problem you are trying to solve, though. – acl – 2014-07-21T22:57:58.723

@acl, I like that as an option (I would want to add that to my init.m file, right?), but when I run it headless on the server I might not see the error when compiling. I know my problem is the result of sloppy programming (I had to look up what it means to "localize my variables"). – Jason B. – 2014-07-22T00:10:05.983

@JasonB OK, if you only ever run it headless then you won't see it. For what it's worth, what I do is write code on my laptop, make sure it works on small example, then send the whole thing to bigger machines. But if you can't, then the solutions here might be the only way (I do have it in my init.m as you said). – acl – 2014-07-22T00:12:08.043