Replacement inside held expression

93

63

I wish to make a replacement inside a held expression:

f[x_Real] := x^2;
Hold[{2., 3.}] /. n_Real :> f[n]

The desired output is Hold[{4., 9.}], but I get Hold[{f[2.], f[3.]}] instead. What is the best way to make such a replacement without evaluation of the held expression?

Alexey Popkov

Posted 2011-07-09T06:46:19.533

Reputation: 50 220

Answers

85

Generally, you want the Trott-Strzebonski in-place evaluation technique:

f[x_Real]:=x^2;
Hold[{Hold[2.],Hold[3.]}]/.n_Real:>With[{eval = f[n]},eval/;True]

(* Hold[{Hold[4.],Hold[9.]}] *)

It will inject the evaluated r.h.s. into an arbitrarily deep location in the held expression, where the expression was found that matched the rule pattern. This is in contrast with Evaluate, which is only effective on the first level inside Hold (won't work in the example above). Note that you may evaluate some things and not evaluate others:

g[x_] := x^3;
Hold[{Hold[2.], Hold[3.]}] /. n_Real :> With[{eval = f[n]}, g[eval] /; True]

(* Hold[{Hold[g[4.]], Hold[g[9.]]}] *)

The basic idea is to exploit the semantics of rules with local variables shared between the body of With and the condition, but within the context of local rules. The eval variable will be evaluated first (regardless of whether the condition ends up being True - as in this case, or False - thanks to @luyuwuli for pointing out the problem in the original wording for this part) , inside the declaration part of With, while the code inside the Condition , here the body of With (g[eval]), is treated then as normally the r.h.s. of RuleDelayed is. It is important that With is used, since it can inject into unevaluated expressions. Module and Block also have the shared variable semantics, but wouldn't work here: while their declaration part would evaluate, they would not be able to communicate that result to their body that remains unevaluated (more precisely, only the part of the body that is inside Condition will remain unevaluated - see below). The body of With above was not evaluated either, however With injects the evaluated part ( eval here) into it - this is why the g function above remained unevaluated when the rule applied. This can be further illustrated by the following:

Hold[{Hold[2.],Hold[3.]}]/.n_Real:>Module[{eval=f[n]},
   With[{eval = eval},g[eval]/;True]]

(* Hold[{Hold[g[4.]],Hold[g[9.]]}] *)

Note b.t.w. that only the part of code inside With that is inside Condition is considered a part of the "composite rule" and therefore not evaluated. So,

Hold[{Hold[2.],Hold[3.]}]/.n_Real:>Module[{eval = f[n]},
    With[{eval = eval},Print[eval];g[eval]/;True]]

(* print: 4. *)
(* print: 9. *)
(* Hold[{Hold[g[4.]],Hold[g[9.]]}] *)

But

Hold[{Hold[2.],Hold[3.]}]/.n_Real:>Module[{eval = f[n]},
   With[{eval = eval},(Print[eval];g[eval])/;True]]

(* Hold[{Hold[Print[4.];g[4.]],Hold[Print[9.];g[9.]]}] *)

This should further clarify this mechanism.

Leonid Shifrin

Posted 2011-07-09T06:46:19.533

Reputation: 108 027

I'm not sure whether I understand it correctly, but my experiment doesn't agree with this conclusion,"Since the condition is True, it forced the eval variable to be evaluated inside the declaration part of With" e.g. Hold[{Hold[2.], Hold[3.]}] /. n_Real :> With[{eval = (Print["!"]; f[n])}, g[eval] /; False] will always print ! even though the condition is False. – luyuwuli – 2020-08-20T16:13:13.940

@luyuwuli You are right, this has been the wrong choice of words, and that sentence was actually misleading. The part inside declaration is evaluated in any case, since in general it is needed for the condition check. Whether the condition ends up being True or False, decides whether the rule is considered at the end as applicable or not, by the pattern matcher. I have edited the post to address this. Thanks for pointing this out. – Leonid Shifrin – 2020-08-20T21:48:23.460

Glad I can help:) BTW, I always learn a lot after rereading your posts. – luyuwuli – 2020-08-20T23:24:51.917

@luyuwuli Thanks. Sometimes I also learn a lot from rereading my posts :) (I obviously learn a lot from (re)reading posts of others, too, but that's probably less surprising). – Leonid Shifrin – 2020-08-21T08:17:21.337

4The Trott-Strzebonski method looks a bit magical. Could you please explain how it works? – Alexey Popkov – 2011-07-09T08:18:22.960

1@Alexey I updated my post with some explanation. It may not be completely satisfactory, but this is how I think of it currently. – Leonid Shifrin – 2011-07-09T10:54:05.393

Why Condition inside With forces evaluation not only With (with except to the Condition) but also enclosing Module? Without the Condition the r.h.s of the rule stay completely unevaluated. – Alexey Popkov – 2011-07-10T04:03:22.817

3@Alexey For the same reason as when you define a global function as f[x_]:=Module[{var = x^2},With[{var = var},Hold[var]/;var>10]]. What happens is that everything in the enclosing scoping constructs that does not go into Condition gets evaluated - this is needed to compute the result of test in Condition. The semantics of rules with shared local variables is different from the standard rule-substitution semantics, this is what makes all these things possible. The further non-triviality of Trott-Strzebonski technique is that local rules are used, so all expression levels are accessible. – Leonid Shifrin – 2011-07-10T07:45:19.280

@Leonid, can you please add this to http://stackoverflow.com/questions/4198961/what-is-in-your-mathematica-tool-bag ?

– Szabolcs – 2011-08-05T15:07:36.317

@Szabolcs Sure I will. I will collect a few more related useful things and then post it there, within a couple of days. – Leonid Shifrin – 2011-08-05T15:13:04.630

This trick also came up in the question Evaluating only particular Head type in expression?

– Simon – 2011-09-02T02:35:09.687

@Simon Indeed. This was before my time on SO :) – Leonid Shifrin – 2011-09-02T08:29:40.077

3+1 The Trott-Strzebonski technique appears to be the officially sanctioned solution. You may have some academic interest in my response which discusses a technique involving the unofficial, unsupported symbol RuleCondition. – WReach – 2011-10-06T20:40:34.930

@WReach Thanks, very nice indeed. I discussed RuleCondition here before: http://stackoverflow.com/questions/5866016/question-on-condition/5869885#5869885, but it did not occur to me to try using it here. Nice finding!

– Leonid Shifrin – 2011-10-07T08:23:17.110

79

RuleCondition provides an undocumented, but very convenient, way to make replacements in held expressions. For example, if we want to square the odd integers in a held list:

Hold[{1, 2, 3, 4, 5}] /. n_Integer :> RuleCondition[n^2, OddQ[n]]
(* Hold[{1, 2, 9, 4, 25}] *)

RuleCondition differs from Condition in that the replacement expression is evaluated before it is substituted. The second argument of RuleCondition may be omitted, defaulting to True:

Hold[{2., 3.}] /. n_Real :> RuleCondition[n^2]
(* Hold[{4., 9.}] *)

It is very unfortunate that RuleCondition has remained undocumented for so long, given its extreme usefulness. The Trott-Strzebonski trick discussed in @Leonid's answer is one way to achieve the same result using only documented symbols:

Hold[{2., 3.}] /. n_Real :> With[{eval = n^2}, eval /; True]
(* Hold[{4., 9.}] *)

A slightly less verbose technique uses Block:

Hold[{2., 3.}] /. n_Real :> Block[{}, n^2 /; True]
(* Hold[{4., 9.}] *)

Judicious use of Trace reveals that both of these techniques ultimately resolve to RuleCondition. One must make up one's mind whether it is better to use the undocumented RuleCondition or rely upon implementation artifacts in With and Block. I suspect that the behaviour is unlikely to change in all three cases as so much Mathematica code depends upon the existing behaviour.

WReach

Posted 2011-07-09T06:46:19.533

Reputation: 62 787

1

Great analysis! It is unbelievable that WRI had published the Trott-Strzebonski trick but still hide this convenient function! Unbelievable but true...

– Alexey Popkov – 2011-10-06T20:21:37.547

@Alexey I think that Leonid's response better qualifies as the accepted answer as the Trott-Strzebonski trick seems to be as close as one is going to come to an officially sanctioned technique. I discuss RuleCondition mainly out of academic interest. – WReach – 2011-10-06T20:42:46.300

3Well. But RuleCondition is really straightforward and includes no magic and as you mentioned the Trott-Strzebonski trick relies on this function too. Probably we can use RuleCondition without risk. – Alexey Popkov – 2011-10-07T03:47:54.583

29

Although less magical, it can be done by ReplacePart

expr = Hold[{2, 3, 4, 5}]
pos = Position[expr, _Integer]
newparts = Extract[expr, pos] /. n_Integer :> n^2
ReplacePart[expr, Thread[pos -> newparts]]

Federico

Posted 2011-07-09T06:46:19.533

Reputation: 2 483

3

Based on solutions proposed by @Leonid and @WReach I have noticed something peculiar with the use of Block and With for the Trott-Strzebonski solution.

Consider the following code:

f[x_] := x^2;
g[x_] := x^3;

Hold[{Hold[2.], Hold[3.]}] /. n_Real :> With[{eval = (g@*f)[n]},
Print[eval]; eval /; True
]

this will yield:

(* 
64.
729.
Hold[{Hold[64.], Hold[729.]}]
*)

using Block will give the same result as above.

Hold[{Hold[2.], Hold[3.]}] /. n_Real :> Block[{eval = (g@*f)[n]},
Print@eval; eval /; True
]

(* 
64.
729.
Hold[{Hold[64.], Hold[729.]}]
*)

However, things become strange - as @Leonid pointed out - with the use of Composite expression in Condition:

using With prevents the evaluation of Print

Hold[{Hold[2.], Hold[3.]}] /. n_Real :> With[{eval = (g@*f)[n]},
(Print[eval]; eval) /; True
]

(* Hold[{Hold[Print[64.]; 64.], Hold[Print[729.]; 729.]}] *)

However, with the use of Block the Print statements get evaluated

Hold[{Hold[2.], Hold[3.]}] /. n_Real :> Block[{eval = (g@*f)[n]},
(Print@eval; eval) /; True
] 

(* 
64.
729.
Hold[{Hold[64.], Hold[729.]}]
*)

Now one begins to wonder how RuleCondition - as mentioned by @WReach - will behave with Composite expression

Hold[{Hold[2.], Hold[3.]}] /. n_Real :> RuleCondition[Print[(g@*f)[n]]; (g@*f)[n]]

(* 
64.
729.
Hold[{Hold[64.], Hold[729.]}]
*)

It turns out that RuleCondition behaves in a similar way to Block

So perhaps not a million dollar but a $5 question: Which one to use (Block, RuleCondition or With)?

In my opinion both have their own advantages. For instance, With will allow you to evaluate a part of the expression and at the same time inject unevaluated code in the substitution process.

With will enable code injection

Hold[{Hold[2.], Hold[3.]}] /. n_Real :> With[{eval = (g@*f)[n]},
(g[eval]; eval) /; True
]
(* Hold[{Hold[g[64.]; 64.], Hold[g[729.]; 729.]}] *)

In the example above (in spirit with what @Leonid mentioned) we substituted the Reals and also injected code that remained unevaluated i.e. g[64] and g[729]

Block and RuleCondition together with Sow and Reap

However, if the goal is to evaluate some code while making substitution 'Block` can be quite effective:

Reap[Hold[{Hold[2.], Hold[3.]}] /. n_Real :> Block[{eval = (g@*f)[n]},
  (Sow[PrimeQ@eval]; eval) /; True
  ]][[2, 1]]
];
(* {False,False} *)


Reap[
 Hold[{Hold[2.], Hold[3.]}] /. n_Real :> Block[{eval = (g@*f)[n]},
 (Sow[EvenQ[Round@eval]]; eval) /; True]
]
(* {Hold[{Hold[64.], Hold[729.]}], {{True, False}}} *)   

In the example above we could the result whether eval generates a Prime whenever the substitution is made. Likewise, this can be achieved using RuleCondition.

Ali Hashmi

Posted 2011-07-09T06:46:19.533

Reputation: 8 448

1

Also discussed here. Note that With doesn't evaluate its second argument at all (it becomes evaluated after leaving With), while Block evaluates it. This helps to understand the difference you describe.

– Alexey Popkov – 2017-12-09T17:30:48.477

@AlexeyPopkov thanks for pointing me to it ! – Ali Hashmi – 2017-12-09T17:52:06.980