## A smarter nested With?

9

3

I often find myself writing code that looks a bit like this:

f[x_Integer] :=
With[
{
range = Range[2] + x
},
With[
{
a = range[[1]],
b = range[[2]],
c = g[range]
},
h[a,b,c]
]
];


It would be nice if I could avoid Withs and just write

f[x_Integer] :=
Let[
range = Range[2] + x,
{a,b} = range,
c = g[range]
,
h[a,b,c]
];


which would then automatically expand to the above at definition time.

What I'm asking is a bit similar to this question. There are additional requirements however. The new scoping construct (Let in the above) should:

• Group sequential disjoint assignments into single Withs.
• Thread over List assignments.

Of course, it should not evaluate the left-hand-sides and the right-hand-sides of the assignments while expanding to Withs.

Any proposals for such a scoping construct? (I'll post my version soon).

Your example doesn't need any scoping: f[x_Integer] := h[Sequence@@#,g@#]&@(Range[2]+x) – Bob Hanlon – 2014-11-05T14:24:11.170

Why do you want to rewrite using With rather than preserving a higher abstraction such as LetL? – Mr.Wizard – 2014-11-05T14:56:18.917

Also: what is the reason to prefer With over Module? Assignments such as {a,b} = range are simpler with the latter. – Mr.Wizard – 2014-11-05T15:00:13.573

@Mr.Wizard This should be a generalization of LetL, which doesn't do the two points I mentioned (Leonid's answer below does). Module doesn't allow you to do threaded assignments in the first argument, forcing you to write Module[{a,b},{a,b}=Range[2];...], which is duplication I don't like. Also, it'd like to inject into held expressions -- another reason not to go with Module. – Teake Nutma – 2014-11-05T15:07:24.393

@Mr.Wizard I guess, the idea is that we want to avoid user-defined functions in expanded code in definitions, both for efficiency reasons and because we may want to attach conditions. As to With vs Module, With is cleaner (manifestly no side effects), when one knows that variables won't change in the body. – Leonid Shifrin – 2014-11-05T15:07:25.790

2If you are ok with undocumented features, you can supply multiple arguments to With in Mathematica 10.3 and above: e.g. With[{c = d}, {b = c}, {a = b}, a] (You'll have to also tolerate the red syntax coloring in the front end, or turn it off manually) [and I just realized that it doesn't satisfy your second requirement of threading over List assignments] – QuantumDot – 2016-05-04T20:03:32.060

## Answers

13

With this helper function:

SetAttributes[partThread, HoldAll];
partThread[l___, rhs_] :=
Join @@ Replace[
MapIndexed[Append[#, First@#2] &, Thread[Hold[{l}]]],
Hold[s_, i_] :> Hold[s = rhs[[i]]],
{1}];


The following modification of LetL seems to work according to your specs:

ClearAll[Let, let];
SetAttributes[{Let, let}, HoldAll];

Let /: Verbatim[SetDelayed][lhs_, rhs : HoldPattern[Let[__, _]]] :=
Block[{With}, Attributes[With] = {HoldAll};
lhs := Evaluate[rhs /. HoldPattern[With[{}, b_]] :> b]
];

Let[args___, body_] := let[{args}, body, {}, {}];

let[{}, body_, {}, _] := With[{}, body];

let[{Set[{s___}, rhs_], rest___}, body_, dec_, syms_] :=
Module[{temp},
partThread[s, temp] /. Hold[d___] :>
let[{temp = rhs, d, rest}, body, dec, syms]
];

let[
{Set[sym_, rhs_], rest___},
body_,
{decs___},
{syms___}
] /; FreeQ[Unevaluated[rhs], Alternatives[syms]] :=
let[{rest}, body, {decs, sym = rhs}, {syms, HoldPattern[sym]}];

let[{args___}, body_, {decs__}, _] :=
Block[{With},
Attributes[With] = {HoldAll};
With[{decs},Evaluate[let[{args}, body, {}, {}]]]
];


This works quite similarly to LetL. What it does in addition to LetL is that it collects previous declarations into auxiliary lists stored as extra arguments of let, so that it can group together disjoint declarations. It also threads over arguments, using the partThread helper function. In all other respects it is the same code as LetL.

Here is your example:

f[x_Integer] :=
Let[range = Range[2] + x, {a, b} = range, c = g[range], h[a, b, c]];


we can check what was generated:

?f

Globalf

  f[x_Integer]:=
With[{range=Range[2]+x},
With[{a=range[[1]],b=range[[2]],c=g[range]},h[a,b,c]]]


@MichaelE2 That was an evaluation leak, thanks for reporting. I seem to have fixed it, although perhaps not very elegantly. – Leonid Shifrin – 2014-11-05T14:40:41.057

You're welcome. It seems fixed now. – Michael E2 – 2014-11-05T14:51:07.000

@MichaelE2 Thanks for double-checking. I've not done this sort of things for some while, got a bit rusty. – Leonid Shifrin – 2014-11-05T14:52:36.103

That's a bit more elegant than what I had :). One question though: why doesn't Alternatives leak evaluation here? – Teake Nutma – 2014-11-05T15:09:13.097

@TeakeNutma because I wrapped symbols in HoldPattern, and use Unevaluated for the rhs. HoldPattern is exactly the right tool for the job here, since it is invisible to the pattern-matcher (and so FreeQ works fine). – Leonid Shifrin – 2014-11-05T15:09:57.990

Ah, now I see the HoldPattern on the RHS of that definition. Very nice. I do have one request though (sorry!): in cases like Let[{a, b} = Range[2], {a, b}] Range[2] gets evaluated twice. Any chance to let this expand into With[{temp$=Range[2]},With[{a=temp$[[1]],b=temp\$[[2]]},{a,b}]]? – Teake Nutma – 2014-11-05T19:33:30.987

@TeakeNutma Yes, sure, that wasn't hard at all, see my edit. Thanks for the accept. This was a nice exercise, and this stuff can be actually useful too. – Leonid Shifrin – 2014-11-05T20:13:37.997

I'm having a bit of trouble bullet-proofing this. Upon changing the LHS of the second definition to Let[args : HoldPattern[Set[_?symbolOrListQ, _] ...], body_] (where symbolOrListQ is an appropriately defined recursive function) and adding another definition Let[__] /; Message[Let::lvset] = Null;, the SetDelayed expansion throws a very strange set of errors. Any idea what's going on? (Btw, you can change the attributes of With in the last definition to HoldFirst and remove the Evaluate there). – Teake Nutma – 2014-11-07T16:41:04.260

@TeakeNutma I can't reproduce this. With this definition:ClearAll[symbolOrListQ];SetAttributes[symbolOrListQ,HoldAll];symbolOrListQ[_Symbol|_List]:=True;symbolOrListQ[_]=False;, everything works fine (I tried the same example f:=Let[...]. I'd need to know exactly all the changes you've made to say more. Re: HoldFirst - not so simple. This may lead to evaluation leaks. In particular, here: let[{}, body_, {}, _] := With[{}, body];. Generally, I think this is a bad idea, makes the code potentially fragile. In my method, I explicitly override HoldAll lexically, in specific places. – Leonid Shifrin – 2014-11-07T17:50:25.090

Here's the full code. The problem arises when you give invalid local variable assignments in a SetDelayed. And thanks for pointing out my oversight regarding the HoldFirst -- these subtleties are easily overlooked. – Teake Nutma – 2014-11-07T19:05:58.383

@TeakeNutma This is rather crazy, the stuff going on there. The reason is that With is in fact broken inside SetDelayed (and inside Let/ let), so adding any more code you want to execute (rather than plain expansion) is dangerous, since that code can rely on With being unbroken. So, my original code was a walk on a thin ice, but still Ok, while the code with added message issuing was unlucky. Presumably, messages use With internally. Anyway, I fixed that, here, but for the expense of elegance, to some extent.

– Leonid Shifrin – 2014-11-07T20:18:18.933

@TeakeNutma I'd add that at this point, this stuff becomes a bit scary. My feeling has been that this is pretty close to the limits of what I can control :) – Leonid Shifrin – 2014-11-07T20:22:42.753

@TeakeNutma Another comment on all this code is that one can make it way more robust, but slightly less cool, by introducing a helper head with, making it HoldAll, and then replacing with With` at the end. I just didn't go this way, but then one can safely use all language constructs inside, and perhaps it is better in terms of being more robust. Just less cool :) – Leonid Shifrin – 2014-11-07T20:29:37.560