The dangers of SaveDefinitions --- should this really happen?

23

6

I have been bitten hard by SaveDefinitions -> True. I'll describe in detail what happened below.

My questions are: Is this a bug? What is the most convenient workaround?


Consider a definition issued like this:

Block[{x, y}, f[x_, y_] = x + y;]

Why didn't I use := instead? Because the expression that stands in place of x+y in my actual problem is computed (symbolically) within the Block so (numerical) evaluations of f are going to be sufficiently fast.

We can check the definition:

?f

f[x_,y_]=x+y

Now let's give x a value ...

x = 1

... and test that f still works as expected:

f[0, 0]

0 (* as expected *)

Let's use f in Manipulate with SaveDefinitions -> True ...

Manipulate[f[a, b], {a, -1, 1}, {b, -1, 1}, SaveDefinitions -> True]

... and check that it works again:

f[0, 0]

1 (* oops!! *)

?f

f[x_, y_] = 1 + y

The definition of f has been rewritten and changed to something else as a side effect of SaveDefinitions.

What is the morale? Probably that SaveDefinitions and Set are not safe to use together.

Note that what happened here is different form the situation when the definition of f is overwritten just because a notebook containing a manipulate with SaveDefinitions has been opened.


My current workaround is to use the following hack to "neutralize" the HoldAll attribute of SetDelayed:

Block[{x, y},
 (f[x_, y_] := #) &[x + y];]

Alternative suggestions are welcome.

Szabolcs

Posted 2013-09-09T19:38:08.287

Reputation: 213 047

3I never use SaveDefinitions. Just never. It's terribly convenient, but for my purposes, it is just not sufficiently predictable. And it can sometimes be incredibly inefficient (e.g., when it stores ridiculous amounts of definitions which were hidden behind a Needs or Get). – John Fultz – 2013-09-12T06:02:29.017

@JohnFultz Isn't it necessary before exporting a Manipulate to a CDF? (I don't have much experience with CDFs but I needed to create some self-contained examples for someone's presentation.) – Szabolcs – 2013-09-12T14:19:03.640

6@Szabolcs All SaveDefinitions does is to auto-construct an Initialization option for you. I'm a control freak. Let me construct my own Initialization option. – John Fultz – 2013-09-13T19:52:26.123

I have the feeling it won't get much better than using Evaluate or your alternative – Rojo – 2013-09-09T19:51:45.413

There's always the possibility to post process the Manipulate – Rojo – 2013-09-09T19:56:33.653

@Rojo yes, maybe it's not the best question ... do you think I should delete? – Szabolcs – 2013-09-09T20:10:41.567

Definately not. I hadn't realised about this, this can be useful. I'll post what I suggested as an answer – Rojo – 2013-09-09T20:12:08.227

2

Didn't I tell you before that SaveDefinitions is dangerous?

– Sjoerd C. de Vries – 2013-09-09T21:11:36.977

1Although this is posed as problem with Manipulate, hasn't this behavior been a problem with Save for a much longer time? I suspect that the dependency tracing mechanism used in Save is just showing up again. – m_goldberg – 2013-09-09T23:21:48.510

@Szabolcs, you could use With to inject the expr or make a function that returns a function. That will be much cleaner and much more modular. – None – 2013-09-10T11:33:20.000

Answers

11

Using InputForm on your Manipulate reveals (like it did in my SaveDefinitions Considered Dangerous post) that it contains the following:

Manipulate[f[a, b], {a, -1, 1}, {b, -1, 1}, Initialization :> {f[x_, y_] = x + y, x = 1}]

So, it actually stores two definitions, one for f and one for x. This actually makes sense, as SaveDefinitions's task is to store any definitions that the Manipulate depends on. When it is checking for this, it finds f, stores its definition and probably then checks whether fitself has any dependencies that need to be taken into account.It checks x and finds a definition in the Global` namespace, so this is stored as well.

Sjoerd C. de Vries

Posted 2013-09-09T19:38:08.287

Reputation: 63 549

4More people should use the InputForm technique. Bravo to you. – John Fultz – 2013-09-12T06:04:03.147

@JohnFultz Thanks! – Sjoerd C. de Vries – 2013-09-12T10:43:54.610

5@kuba ... or just use \[FormalX] and \[FormalY] symbols in your definitions. Can't go wrong with that. – Sjoerd C. de Vries – 2013-09-09T21:50:10.537

Oh, I wasn't aware of this. Thanks :) – Kuba – 2013-09-09T21:52:33.080

13

This seems to be a very easy way to bite yourself in the foot (non-flexible programmers this is not for you). I also have the habit of doing those blocked set-based definitions. It's probably time to change the habit now.

It seems to me that SaveDefinition extracts the definitions of the symbols required by the Manipulate, as you entered them. If you used :=, then it will be stored as :=, if = then =. So, if you made a definition with = that required certain localization, that localization won't be captured by SaveDefinitions.

I am not sure how to avoid that, other than using := in the first place and ensuring what you pass to := is already the final form of the rhs you desire.

So, your own suggestion looks good. The natural alternative is using f[x_, y_]:=Evaluate[...].

Another attempt of an alternative could be to modify your Manipulate after creation, changing the Set-based definitions to SetDelayed (assuming this doesn't bring other unintended sideeffects).

An implementation could be (please @Mr.Wizard, prettify this if you see a nice way. I haven't got the time or brains these days)

postProcessTheManipulate[m_Manipulate] :=
 Replace[m // ToBoxes // ToExpression, (Initialization :> l_List) :> 
   Block[{},
    (Replace[Hold@l, 
        HoldPattern@
          Set[args : PatternSequence[Except[_Symbol], ___]] :> 
         SetDelayed[args], {2}] /. 
       Hold[ll_] :> (Initialization :> ll)) /; True], {1}]

To be used

Manipulate[f[a, b], {a, -1, 1}, {b, -1, 1}, 
  SaveDefinitions -> True] // postProcessTheManipulate

Edit by Mr.Wizard -- As requested here is a terse version, assuming it is safe to replace all Set expressions at level one on the RHS of Initialization:

pptm[m_Manipulate] :=
 Replace[m // ToBoxes // ToExpression,
   init : (Initialization :> _) :>
     RuleCondition @ Replace[init, Verbatim[Set][x__] :> SetDelayed[x], {2}],
   {1}
 ]

Rojo

Posted 2013-09-09T19:38:08.287

Reputation: 40 993

Bite yourself in the foot? Around here we say shoot yourself in the foot, which somehow makes a lot more sense. :^) – Mr.Wizard – 2013-09-09T20:21:37.270

Could you explain the reason for args : PatternSequence[Except[_Symbol], ___] -- at the moment I don't see the need for the complex pattern. – Mr.Wizard – 2013-09-09T20:26:28.663

@Mr.Wizard I was lazy to think whether there was a statistically significant disadvantage to changing ALL = to := so I just tried to prevent the changing in the obvious Set's kingdom: the x=something;-land – Rojo – 2013-09-09T20:28:52.867

3By providing the Manipulate with Method -> {"ExtraVariables" :> {f}} f will get localized and the original un-delayed definition wont be contaminated. ( {f,x} in Mr.Wizards version ) – ssch – 2013-09-09T20:40:51.777

I think it should be safe, so for the sake of shorter code I left it out. Now people have options. – Mr.Wizard – 2013-09-09T20:41:12.363

Thanks @Mr.Wizard! Way clearer code now – Rojo – 2013-09-09T21:02:42.160

@ssch Wow! That's nice. Where did you get that information from? Looking at the source code? – Rolf Mertig – 2013-09-09T21:31:20.100

1@Rolf I used the Spelunking package on Manipulate to see how it created the definitions, ran into it in Manipulate`Dump`MakeManipulateBoxes. Would be even nicer if SaveDefinitions localized things automatically to avoid the problem Sjoerd linked – ssch – 2013-09-09T21:43:00.073

@ssch Cool. Yes, SaveDefinitions, Initialization are not developer-friendly functions (or, not enough documented). Maybe the FrontEnd team thinks that $CellContext` is enough localization (I wished one could turn that off btw.). – Rolf Mertig – 2013-09-09T22:00:22.773

1@ssch That prevents the interaction with f but the Manipulate still leaks the setting for x. After a restart the notebook with that Manipulate panel will have x defined as 1. So you have to add all variables (i.e., x and y) to the list. – Sjoerd C. de Vries – 2013-09-10T08:37:49.320