Why does list assignment with a packed array result in unpacked values?



Bug introduced in 5.2 or earlier and fixed in 10.1.0

This is another question on the design choices in Mathematica. I understand that without direct reply from the developers it may not be possible to give a definitive and exhaustive answer to "why" questions but such topics have often been fruitful.

Why does list assignment of the form {a, b, c} = tensor where tensor is packed result in unpacked values for a, b, and c?

To illustrate:

packedQ = Developer`PackedArrayQ;

tensor = RandomReal[99, {3, 5, 7}];

tensor // packedQ


{a, b, c} = tensor;

packedQ /@ {a, b, c}

{False, False, False}

It is possible to make the assignments without unpacking the sub-arrays of tensor by manually unpacking the outer list using Apply:

{a, b, c} = List @@ tensor;

packedQ /@ {a, b, c}

{True, True, True}

Why doesn't Set operate like this by default?

That is, why doesn't Set only unpack the right-hand-side as far as necessary, to the level of the left-hand-side?


Posted 2013-07-22T14:16:48.150

Reputation: 259 163

@Leonid I see from these comments that a year ago I was unaware that this also unpacks one level: /. h_@x__ :> h@x – Mr.Wizard – 2014-08-03T04:42:47.043

@Mr.Wizard Well, I think this was kind of easy to guess, since the pattern matcher has no way to not unpack for this particular rule. – Leonid Shifrin – 2014-08-03T09:47:07.790

Just a small comment on an old question. Since packed arrays were only introduced in version 5.0, I would think that if it exists in 5.2 it probably was a simple omission in the (AIUI, involved) operation of grafting packed arrays into the language in the first place. I am not quite sure what this meant about its "bug-ness" before the report was filed. – Oleksandr R. – 2015-04-17T10:20:36.033

@Oleksandr Good point. I was only going off Tali's "I've filed a bug for this" but perhaps it is more of a limitation than a bug? – Mr.Wizard – 2015-04-17T17:01:09.363

Your question raises another in my mind: since tensor already has the head List, why does List @@ tensor have any effect on Set at all, considering it's evaluated before Set even sees it? – m_goldberg – 2013-07-22T19:58:30.670

2@m_goldberg Because Apply does unpack the top level of the packed array. It does it always, even if the head to be applied is also a List. – Leonid Shifrin – 2013-07-22T22:59:16.580

@LeonidShifrin. So, although it's not his main point, one thing Mr.Wizard is telling us is that when Set is given tensor with its top-level unpacked, the lower levels will not be disturbed. – m_goldberg – 2013-07-23T02:16:34.580

@m_goldberg Yes, that is what I tried to express that with the final example given. Another way to look at this problem is that apparently when Set gets a packed array on the RHS and a list on the LHS, it fully unpacks the RHS rather that only unpacking it to the level of the LHS. This seems like an unfortunate choice, but I have often learned that there are good reasons for such choices once I asked about them. – Mr.Wizard – 2013-07-23T03:19:42.023

I had thought @m_goldberg's question was why is Apply designed that way. That Apply always unpacks the first level is not an explanation why it does it when it's unnecessary. I would think that it's because List @@ List[..] is not important enough to make a special case. Or it's so that {a, b, c} = List @@ tensor is a workaround for Set's behavior. Or something else like that. – Michael E2 – 2013-07-23T03:49:01.680

1@MichaelE2 I would agree that the special case of List was probably considered not important. In fact, the only reason one would want to do List @@ packed would be if one wants to unpack one level. It also seems to be the only way to do this (i.e. using Apply), if one wants to only unpack one level. – Leonid Shifrin – 2013-07-23T09:10:21.707

@Leonid I believe you could also use #& /@ packed, and even other functions such as Outer, though inelegantly: Outer[# &, packed, {1}, 1][[All, 1]] – Mr.Wizard – 2013-07-24T14:01:02.970

@Mr.Wizard Map only unpacked because the length of the list was smaller than "MapCompileLength" setting. And Outer is rather inelegant, I agree. – Leonid Shifrin – 2013-07-24T14:22:58.440



Thanks Mr.Wizard. I've filed a bug for this, #288440 if you have future correspondence with people inside the company about it...

Taliesin Beynon

Posted 2013-07-22T14:16:48.150

Reputation: 10 244

This does not provide an answer to the question. To critique or request clarification from an author, leave a comment below their post. – m_goldberg – 2015-02-09T19:23:29.043

@m_goldberg Do you think it is reasonable to make an exception for the developers of Mathematica? Given at least some implied authority here this is a form of answer. – Mr.Wizard – 2015-02-09T20:04:42.917

@Mr.Wizard. Perhaps the issue you bring up should be discussed in Meta. Personally, I'd be happy with this being treated as an exception granted by the moderators should that be the moderator position. – m_goldberg – 2015-02-09T20:40:33.770

5@m_goldberg, since Taliesin has filed a bug one might argue that the question has been answered, albeit indirectly. Why does Mathematica do this? Because it has a bug. (As opposed to it being a deliberate design choice). – Simon Woods – 2015-02-09T22:22:39.293

3Aside from these meta comments, this has been fixed. Should be in the next release. – Taliesin Beynon – 2015-02-10T04:15:19.910

@m_goldberg I welcome you to post on Meta if you care. In this case I see it the same way that Simon's described it. As a comparison bugs questions (of which this is now one) have been answered with "confirmed as a but by Wolfram" or "Fixed in 10.0.0" on more than one occasion. If you post on Meta these should also be addressed. – Mr.Wizard – 2015-02-10T06:57:45.613

@Tali I know you must be busy, but if you are willing could you give or draw some attention to these longstanding bugs/issues?: (13347), (13349), (13399), (19656). The first three are about Grid, the last one is about Graphics.

– Mr.Wizard – 2015-02-10T07:07:25.217

@Mr.Wizard. I've come to agree with Simon Woods. I don't see any reason to raise this in Meta. My suggestion about doing so was prompted by my thinking you might want to raise it there :) – m_goldberg – 2015-02-10T08:29:35.753

@Mr.Wizard those have nothing much to do with me... you're better off asking someone else. – Taliesin Beynon – 2015-02-23T11:08:43.857

@Tali I did not mean to imply that these were in any way your responsibility; I had hoped that you would be able to bring these to the attention of the right person(s). Anyway, thanks for replying. – Mr.Wizard – 2015-02-23T17:09:21.080

@Tali Are you comfortable with this being tagged as a bug? (See Oleksandr's comment under the question above.) Thanks again for getting this working right either way! – Mr.Wizard – 2015-04-17T17:02:51.703

1@Mr.Wizard unless we want to decide how many angels can dance on the head of a pin, let's just call it a bug :) – Taliesin Beynon – 2015-04-22T05:35:05.880

@Tali lol -- OK :-) – Mr.Wizard – 2015-04-22T05:37:25.817