Compile and uncompilable function bug?

12

1

I think I found a bug, but I still have MMA version 11.0.1 installed. Can somebody please check if this persists in the latest version?

When I compile this function:

cf = Compile[{{n, _Integer}},
  Module[{good},
    good = True;
    If[DuplicateFreeQ[{}] && n == 15, good = False];
    good
  ]
];

The result changes once the condition is met:

cf[14]
cf[15]
cf[14]
(* True *)
(* False *)
(* False *)

Here it is immaterial what list is given to DuplicateFreeQ.

Addendum 1

As @Jason B. and @Michael E2 pointed out, the important thing is that DuplicateFreeQ cannot be compiled. Thus, a more general example is

fun := True; (* use := so fun cannot be compiled *)
With[{cf = Compile[{{n, _Integer}},
    Module[{xyz},
     xyz = True;
     If[fun && n == 15, xyz = False];
     xyz
     ]
    ]},
   {cf[14], cf[15], cf[14]}
 ]
 (* {True, False, False} *)

Now, @Michael E2 wrote that it seems as if

there is a runtime environment in the WVM connected to cf, in which the assignment good = False leaks out in some form and is stored across calls to cf.

Here is some evidence for that, I think:

  1. Quit the kernel.
  2. ?Global`xyz* gives a warning that there is no xyz defined.
  3. Evaluate

With[{cf = Compile[{{n, _Integer}}, Module[{xyz}, xyz = True; If[n == 15, xyz = False]; xyz]]}, {cf[14], cf[15], cf[14]}] (* {True, False, True} *)

as it should be.

  1. ?Global`xyz* shows that a Global`xyz has been defined.

  2. Evaluate the code at the beginning of Addendum 1, giving the wrong result {True, False, False}, and you find that Global`xyz as well as Global`xyz$ are defined, where Global`xyz$ has the attribute Temporary.

JEM_Mosig

Posted 2017-06-29T12:05:10.070

Reputation: 2 559

2Please don't use the bugs tag before the behavior has been confirmed as a bug by other users as well. – MarcoB – 2017-06-29T14:46:22.563

2Can't reproduce the behavior in v9.0.1 – xzczd – 2017-06-29T14:49:43.973

2I can reproduce the behavior in MMA 11.1.1, and indeed it is strange, but I wonder if you are aware that DuplicateFreeQ is not compilable, so it causes a call to the main evaluator within your compiled function. – MarcoB – 2017-06-29T14:59:20.747

3This doesn't seem to have anything to do with DuplicateFreeQ - you could use StringQ[""] instead. I can even get the same result with cf = Compile[{{n, _Integer}}, Module[{good}, good = True; If[And[Evaluate[True], n == 15], good = False]; good]]; – Jason B. – 2017-06-29T16:01:50.907

1I think the symbol xyz is created in evaluating Compile and the xyz$ is created in the MainEvaluate call when you run cf[14] (or cf on anything). The actual variable used in cf[14] has the form xyz$nnn, where nnn is a serial number, different on each call to cf -- these are Remove-d but show up in a Trace. I think they are probably not the source of the leak, if that's what you're thinking. – Michael E2 – 2017-07-01T02:30:53.973

1It seems that we have a consensus that this is a bug, so I've added bugs tag. In versions 8.0 and 9.0 I get run-time type error and function switches to uncompiled evaluation. In versions 10.0 -10.3 and 11.0 I can reproduce described (buggy) behavior, interestingly version 10.4.1 gives correct {True, False, True} result, I'm not sure how to describe this in standard bugs header. Also behavior is correct for CompilationTarget -> "C", so it indeed seems to be related to WVM. – jkuczm – 2017-07-01T12:57:51.137

Can somebody with access to multiple versions please add the customary header? – J. M.'s ennui – 2017-08-06T18:21:25.453

Answers

14

If you declare the type of DuplicateFreeQ, it works consistently (in V11.1.1).

cf = Compile[{{n, _Integer}},
   Module[{good},
    good = True;
    If[DuplicateFreeQ[{}] && n == 15, good = False];
    good],
    {{DuplicateFreeQ[_], True | False}}
   ]; 

One really ought to declare the type of DuplicateFreeQ, but it's as yet unclear to me why omitting the declaration does not lead to an error or warning and exhibits the strange behavior in the OP.


Update: Without the declaration, Compile does not know what to do with the return value of DuplicateFreeQ[], so it has to send off the whole If[DuplicateFreeQ[{}] && n == 15, good = False] to MainEvaluate[]; but since the value of good might be changed, it has to evaluate good in MainEvaluate[], too. The compiled code for this is

{46, Function[{n}, If[DuplicateFreeQ[{}] && n == 15, good = False]],
  {good, 1, 0, 0, Module}, 2, 0, 0, 6, 0, 17}

This translates, according to CompiledFunctionTools`CompilePrint, to

V17B0 = MainEvaluate[
  Function[{n, goodCompile$1},
   Block[{good = goodCompile$1},
    {If[DuplicateFreeQ[{}] && n == 15, good = False], good}
    ]
   ][ I0, B0]
  ]

Note it appears to use Block to localize good. At face value this couldn't be good, since it changes the value of a global variable before calling code in the global context. In fact, it does not use Block in the way the code suggests, because the value of Global`good is not affected. This can be checked in at least of couple of ways, one of which is via an internal Trace. The first time cf[14] is called, it looks like this:

Trace[cf[14], TraceInternal -> True]

Note Module is used, not Block, and contains the initialization expression good$ = True. After c[15] is called, the same trace of c[14] has the initialization good$ = False.

Note that recompiling cf resets the behavior, so that cf[14] will return True until cf[15] is called. So does cf = CompiledFunction @@ cf.

It does raise in my mind whether there is a runtime environment in the WVM connected to cf, in which the assignment good = False leaks out in some form and is stored across calls to cf. That's what appears to be happening. This view inclines me to think it's a bug.

Michael E2

Posted 2017-06-29T12:05:10.070

Reputation: 190 928