How to Set parts of indexed lists?



I would like to assign a list to an indexed variable and then change it using Part and Set like this:

matM[i] = ConstantArray[1, {10, 10}];
matM[i][[1,10]] = 10

Unfortunately I will get the error:

Set::setps: "matM[i] in the part assignment is not a symbol. "

Using Subscript will not work either:

Subscript[matM,i] = ConstantArray[1, {10, 10}];
Subscript[matM,i][[1,10]] = 10

will give the same error.

How can this be done?


Posted 2012-06-22T09:27:50.550

Reputation: 11 083

1Is using a list instead and doing matM[[1]][[1,10]]=10 an option for you application? Could you explain why you need an indexed variable? Then it would be easier to come up with a solution. (As you noticed, assigning to Part doens't work with DownValues...) – Szabolcs – 2012-06-22T09:37:16.523

Szabolcs, as written below, I am a bit amazed at what seems 'illogical' in not being able to address downvalues. I might have to use the list-option as you suggest, but for a more symbolic reference like matM[a] it would be very nice to do it the way I suggested. I should be possible, shouldn't it? – gwr – 2012-06-22T09:50:57.467

I had the impression that you were doing a numeric/programming type of work, not symbolic manipulation (since you were using Part to change arrays element by element). What exactly are you doing? – Szabolcs – 2012-06-22T10:08:51.227

Just to make this clearer, I have edited the original question in order to make it clearer by having the index be a symbol. – gwr – 2012-06-22T10:09:32.023

Szabolcs, I am just trying to have an indexed variable be a list or to be more precise an array. While in principle you are right that I might transfer every index to a numerical range and thus one could come around by simply adding the index-dimension to the arrray dimension, there is good reason to work with the 'sparse' definitions allowed by using an index for the variable itself. My problem at hand is that I am changing code already written. Adding the index in the variable name is a simple replacement. Adding it as an array dimension means a lot of work... – gwr – 2012-06-22T10:14:09.520

I think it is a rather important question, I have run into this issue several times when (as Leonid states below in a comment) I've tried to avoid ReplacePart, but the input expression wasn't simply a symbol. I've changed the title to reflect the generality of the question, please feel free to roll back if you find it less informative. – István Zachar – 2012-06-22T11:29:04.223



I can offer a sort of a solution, which has its shortcomings, but will allow you (more or less) to use the syntax you want. The idea is that you mark symbols you need, as "references", and then execute your code in a special dynamic environment where Part is overloaded.


Here is the code. This function will mark you symbol as a reference.

makeReference[sym_Symbol] :=
  sym /: Set[sym[index_], rhs_] :=
     With[{index = index},
      With[{heldVal = Hold[sym[index]] /. DownValues[sym]},
        If[heldVal === Hold[sym[index]],
                    HoldPattern[sym[index]] :> ref]
        Hold[sym[index]] /. DownValues[sym] /. Hold[s_] :> (s = rhs);

What happens is that we "softly" overload Set on this particular symbol via UpValues, so that an intermediate symbol is inserted where the actual data will be stored, and our symbol (for a given index) refers to that intermediate symbol. Since the latter has no restrictions on part assignments, we can assign parts of it directly at O(1) time.

However, the subtlety is that when we call Set[Part[s[ind],1,2],something], Set holds its first argument, and therefore, s can not communicate to Set that this is special (UpValues won't work here since the s is too deep inside an expression - on level 2 - while UpValues are only looked at at level 1). To solve this problem, we will overload Part, but do it locally within a local dynamic environment, to make this operation safer. This is a dynamic environment:

SetAttributes[withModifiedPart, HoldAll];
withModifiedPart[code_] :=
      Part /: Set[Part[sym_Symbol[index_], inds__], rhs_] :=
        With[{index = index},
          Hold[sym[index]] /. DownValues[sym] /. 
               Hold[s_] :> (s[[inds]] = rhs);


Now, we can test this:


and then

   a[1] = Range[10];
   a[1][[2]] = 100;
 {1, 100, 3, 4, 5, 6, 7, 8, 9, 10}

Let's now measure some timings:

    a[1] = Range[100000];
    Do[a[1][[i]] = a[1][[i]] + 1, {i, a[1]}];
] // Short // AbsoluteTiming

We can compare this to the time it takes for direct assignments:

aa = Range[100000];
Do[aa[[i]] = aa[[i]] + 1, {i, aa}]; // Short // AbsoluteTiming

So, for massive assignments, we are about 6 times slower (which I think is OK). We can also see how costly is the overloaded Part for normal assignments:


from where it looks that those are slower by about 15 percents.


The suggested solution requires 2 modifications to your code:

  • call makeReference on symbols which you wish to use as indexed symbols with part assignment, prior to assigning to them.
  • Execute all your code containing such assignments, inside the withModifiedPart environment.

So, your original code will be changed to


  matM[i] = ConstantArray[1, {10, 10}];
  matM[i][[1, 10]] = 10

What about safety? I would argue that, for this particular case, modifying Part is safe enough. This is because, first, Part is overloaded softly, via UpValues. This means that, when Part is not inside some head which holds arguments, it will execute normally before it would even "think" of a new definition. Now, when it is inside some head which holds arguments, the new definition will only work if that head is Set. Note that no ne rules were added to the Set itself. And since normally, assignments to indexed variables are not allowed anyway, we don't risk breaking some internal behavior.

The performance here is worse than for direct assignments to symbol's parts, but overall seems acceptable.

Leonid Shifrin

Posted 2012-06-22T09:27:50.550

Reputation: 108 027

I can't seem to get this working inside a For loop, ie. I have something like For[i=1, i<=2, i++, withModifiedPart[a[i]={i,i2,i3};a[i][[1]]=5;a[i]] (or something along those lines. Am I doing something wrong? – Andrew Stewart – 2014-05-16T22:22:28.147

@AndrewStewart You've discovered a couple of subtle bugs. I fixed them, should work now (use the updated code). Use as ClearAll[a];makeReference[a]; ClearAll[i]; For[i = 1, i <= 2, i++,withModifiedPart[ a[i] = {i, i ** 2, i ** 3};Print[a[i]];a[i][[1]] = 5; a[i]]]. Thanks for bringing this up. – Leonid Shifrin – 2014-05-16T22:44:41.723

Leonid, what a solution, thank you. I had believed there is some 'easy' way - possibly upvalues - but your answer shows how deep the matter runs. – gwr – 2012-06-22T13:47:12.713

@gwr Thanks :). In fact, since Part is overloaded via UpValues, and then only for Set, and part assignments for indexed symbols are normally prohibited anyway, I would even think that globally overloading Part could be ok (i.e. doing the same thing but without the dynamic invironment), in which case, you don't have to wrap your code in it. But, it is still better to be on the safe side, all it takes is just to wrap your top-level function(s) in withModifiedPart. Thanks for the accept as well. – Leonid Shifrin – 2012-06-22T13:50:22.957

+1, nice one! What's the purpose of setting the DownValue by hand only when there's no other downvalue associated with the symbol, versus simply sym[index]=ref? – Rojo – 2012-06-22T13:52:22.407

@Rojo Thanks:). As to the reason for my construct: because I already overload Set on this symbol, I can not use it (Set) to set the DownValue, since the new definition will be used, and we will enter an infinite recursion. – Leonid Shifrin – 2012-06-22T13:54:29.707

Right, and furthermore you avoid creating a different symbol every time a certain index is reassigned a new value, leaking memory. My yet-non-DownValues-user instinct would have led me to tackle both issues differently. Using explicit DownValues is something that doesn't come straight to mind – Rojo – 2012-06-22T14:07:11.823

@Rojo Yes, you are right, that as well. I though about that as well, but indicated the above as a primary reason. – Leonid Shifrin – 2012-06-22T14:12:55.243

Leonid, this got me thinking about how I would have done it. Posted it as an answer because the code got long for a comment, with the appropriate warnings, and an extra idea to ease my conscience on posting it as an answer – Rojo – 2012-06-22T15:46:13.487

@Rojo I could also have used SetDelayed instead of appending to DownValues explicitly. – Leonid Shifrin – 2012-06-22T17:49:50.183

@LeonidShifrin, nice one, and that could also allow me to remove the $guardOver from my version and make it more readable. Out of curiosity, Set doesn't seem to be recursive, so another alternative for cases where you've already messed with SetDelayed is something like #&[sym][index] = ... – Rojo – 2012-06-22T18:13:16.657

@Rojo Yes, that's a good catch. You exploit that the rule does not match during the pattern-matching stage, and the symbol gets substituted as a part of Set evaluation semantics (it evaluates heads). – Leonid Shifrin – 2012-06-22T19:04:03.223


Unexpectedly I don't favor Leonid's approach. I feel it costs too much performance, and performance is a primary reason for using an assignment like this.

My method is to store the data in a direct assignment to a separate symbol and use a special function to effect the assignment.

SetAttributes[set, HoldFirst]

set[sym_ @ idx_, val_] :=
   Quiet[sym @ idx =.];

   sym @ idx =. ^:= (sym /: sym @ idx =.; x =.);

   set[sym[idx][[part__]], vv_] := x[[part]] = vv;

   sym @ idx = x;
   x = val


set[a[x], {1, 2, 3}];

set[a[x][[2]], 7];

{1, 7, 3}

Care is taken by the function to remove old data when new is assigned. It can also be manually cleared with a[x] =. and the hidden symbol will also be cleared.

Leonid cautiously avoids global modification to Part or Set. One could take a more bold approach for convenience sake and probably be safe as the pattern is rather specific. (In this case it will only match for the exact symbol[index] that is initialized with set.) This could be done with:

set[sym_ @ idx_, val_] :=
   Quiet[sym @ idx =.];

   sym @ idx =. ^:= (sym /: sym @ idx =.; x =.);

   Set[sym[idx][[part__]], vv_] ^:= x[[part]] = vv;

   sym @ idx = x;
   x = val

One would still need the initial set[a[x], . . .] to set up, but assignments to parts would be done with a[x][[i]] = . . .

Timings compared to Leonid's method


  a[1] = Range[100000];
  Do[a[1][[i]] = a[1][[i]] + 1, {i, a[1]}];
]; // AbsoluteTiming
{0.6300009, Null}
  set[a[1], Range[100000]];
  Do[set[a[1][[i]], a[1][[i]] + 1], {i, a[1]}];
{0.1800003, Null}


Posted 2012-06-22T09:27:50.550

Reputation: 259 163

There is no question that by using a special syntax you can gain performance, I realized that from the start. The complexities of my solution are due to my attempt to keep the syntax as close to the original as possible. Performance is a valid concern, but not the one raised in the question, so I did not consider it a primary concern as long as the slowdown is not too dramatic (e.g. full order of magnitude or more). Anyway, nice code, and +1. – Leonid Shifrin – 2012-06-23T12:08:11.083

As to performance, I became much more pragmatic about it recently, as I see more and more that the requirement to always stay within Mathematica is artificial. My current thinking is that Mathematica has several layers where it shines, such as 1. High-level (possibly rule-based) - very easy to put new ideas to code 2. Number crunching and other tasks where one can use Compile 3. Data manipulation, where one can use vectorized operations to make it reasonably fast (of course, I only talk about core proramming). But it also has gaps, in particular in the realm of mutable efficient ... – Leonid Shifrin – 2012-06-23T12:26:51.403

... data structures and efficient general programming. These gaps can often be alleviated by using other languages (e.g. Java, but not only). I can foresee the proliferation of tools similar to my Java reloader, connecting Mathematica to other languages, so I value M mostly as a very interactive medium and development laboratory, a tool to create prototypes. Once I see that something is slow, I will localize the problem and use Java or C or some other language. We spend gigantic efforts on optimization while the code can often be written straightforwardly in other langs and save us time. – Leonid Shifrin – 2012-06-23T12:31:19.027

@Leonid those are very valid points. I am biased because I tried to learn C many years ago and it only gave me headaches; then I discovered Mathematica and the joy of doing with one line of code what was taking dozens and I never looked back. Your Java Reloader makes using code copy and paste easy so it's hard to argue against building a Java library, though if I never learn Java I'll end up with "cargo cult programming." I am (largely thanks to you) slowly coming to see the value of using Mathematica as a framework for multiple languages. – Mr.Wizard – 2012-06-23T17:32:31.863

@Leonid by the way, what is your take on the safety of the second option I present? It seems to me that overloading Part with a specific pattern is unlikely to cause problems. Do you agree? – Mr.Wizard – 2012-06-23T17:37:27.783

I actually started feeling the power of Mathematica as a gluing medium (as compared to Mathematica the language) relativey recently. The problem is that cross-language development has higher entry barrier than the same languages taken separately, since one has to also deal with the impedance mismatch between the langauges. I think we currently just miss the tools to make it easier. As to the second option, it is probably safe enough, but you just never know. And also, it can make Part slower if you have a large number of symbols (but that has to be tested). – Leonid Shifrin – 2012-06-23T19:51:11.083

@Mr.Wizard Hi. I just don't like one point in your solution. We get a unwanted global variable. If we make DownValues@a we get something like {HoldPattern[a[x]] :> x$1280} instead of {HoldPattern[a[x]] :> {1, 7, 3}}. @Leonid solution has the same side effect, if I may call it that. – Murta – 2013-02-19T23:43:55.367

@Murta well yes, there is going to be a proxy symbol. What is your issue with that? It could be moved to a private context if for some reason Global` is bad. If you need to make use of DownValues an additional definition could be made to translate as needed. If you are specific I will try to address the problem. – Mr.Wizard – 2013-02-20T00:15:23.800

I tried an answer. I just don't know why my time performance is so bad. Some clue? – Murta – 2013-02-20T02:43:17.877

@Murta at first glance it appears you are reassigning the entire definition (array) for each set. I'm still trying to understand your objection to my method so that I might address it. – Mr.Wizard – 2013-02-20T02:58:34.147

Seems strange that x jump in the code. Save everything in a seems to be more natural. But maybe it's just something new that I have not yet used. About my code, I like that we can define a = list directly, but it's very slow in your test (more than 10x). – Murta – 2013-02-20T03:08:03.297

@Murta I believe the slow-down is because of the complete re-assignment as I observed. Is your dislike of x$??? merely visual? Because that could be concealed. – Mr.Wizard – 2013-02-20T03:29:13.560

No.. it's not visual. It's more about pollute my Global context. But now I understand better your code, and see that when I clear Symb@indx, x dies together. Symb@indx works just like a pointer. Cool!.. tks for tks patience. – Murta – 2013-02-20T03:38:31.933

@Murta Okay. You are aware that you could create a new context to keep Global` clean? It's more involved and I didn't see the need but I can show you how if you like. – Mr.Wizard – 2013-02-20T03:45:02.967


This is not a precise answer to the actual question, but the list of answers is from 2012 and IMO does urgently lack a note about Association. Since version 10 (2014) these will solve the underlying problem by offering an alternative for such cases:

matM = Association[]
matM[i] = ConstantArray[1, {10, 10}];
matM[[Key[i], 1, 10]] = 10

For every case where one uses downvalues as a hashtable using Association instead like shown will also have many other advantages. I have not made tests but I would be very surprised if the above wouldn't be much more efficient than any of the alternatives. Of course using an Association will not work in all cases, e.g. when one needs a mixture of patterns and constant "indices" in the downvalues of a symbol. But for a vast majority of appications where one runs into the mentioned problem nowadays Association will certainly be the best solution.

Albert Retey

Posted 2012-06-22T09:27:50.550

Reputation: 22 455

Thank you, Albert, that indeed is a good and necessary link to the new features in Mathematica. I cannot imagine doing most anything without using Association. – gwr – 2017-02-07T08:41:25.177

1@gwr: I was almost expecting that you will already use them where appropriate. I thought this answer would be justified especially for new or unexperienced users so they do not follow a complicated pattern which in newer versions is not necessary anymore... – Albert Retey – 2017-02-07T09:33:27.303

+1. Very good point. This indeed seems to be the way now. – Leonid Shifrin – 2017-02-07T16:56:54.130


No need for a custom function, you can use the built in ReplacePart:

matM[1] = ConstantArray[1, {10, 10}];

matM[1] = ReplacePart[matM[1], {2, 5} -> 7]

There are variations of ReplacePart that allow you change groups of elements.


Posted 2012-06-22T09:27:50.550

Reputation: 9 964

4The big problem with ReplacePart is that it copies entire list rather than modify a part in-place. What this means is that part assignment is now O(n) rather than O(1), where n is a number of elements in a list. And also, frequent list copying means an overhead due to excessive memory allocation, and possibly enhanced memory consumption (depending on how efficient is garbage collector / allocator). For all practical purposes, this is a disaster in most cases. – Leonid Shifrin – 2012-06-22T11:22:11.000

@LeonidShifrin That all makes perfect sense, I'm not sure how large the OPs n is. I guess you've traveled this route many times in the past ? – image_doctor – 2012-06-22T11:49:28.740


Yes, I did :)

– Leonid Shifrin – 2012-06-22T12:02:36.923


Ok, I'll offer a solution that I think it's better suited to some of the use cases: use the parsing stage.

A simplistic implementation could be

makeReference[sym_] := 
   RowBox[{ToString@Unevaluated@sym, "[", indexBox_, "]"}], 
   StandardForm] := 
   ToString@Unevaluated@sym <> "\[LetterSpace]" <> 
    ToExpression[indexBox, StandardForm, ToString], StandardForm]

Now we don't need the dynamic environment any more, and we get faster results. Beware: this definition is not attached to the symbol, so to unset it one would need to ipmlement these as a fucntion that unsets the MakeExpression definition, and depending on your goal, maybe removing the intermediate symbols and copying the values straight to sym[index]. Just let know of further complications in trying to implement whatever you need.

Long PD

While looking at Leonid's idea, I thought that I wouldn't have done it that way, not mainly because I know better ones but because I tend to avoid explicit DownValues for whatever reason. What follows is just a reimplementation of @Leonid's answer avoiding the explicit use of DownValues (by basically saving the rules somewhere else). So, bear in mind, it's just an alternative implmeentation of Leonid's great code.

Since some non-experienced users may find his code a little cryptic and mine not, others may find mine a little cryptic for other reasons and his not. Hopefully this adds a few extra guys who end up understanding any of the ways to implement this. That's why I commented it quite a bit. Remember that in this version, just like Leonid's we do need the dynamic environment withModifiedPart that you can find in his answer.

According to a few tests, it is faster than Leonid's for writing new indexes and rewriting already indiced variables, but a little slower on reading and modification of parts of indexed variables, so I guess it depends on the use case

makeReference[sym_Symbol] :=
 Module[{$guardEval = True, $guardOver = True},

  (* This is where we'll be saving the intermediate symbol rules *)
  sym["IntermediateSymbols"] = {};

  (* This definition is the first one tried when setting a value on \
the indexed symbol. It replaces the sym[
  index] by the intermediate symbol and tries setting again. 
  The next trial, 
  this definition won't be matched even if there wasn't any \
intermediate symbol defined *)
  sym /: exp : Set[sym[_], _] /; $guardEval := 
       Block[{$guardEval = False},
    Unevaluated[exp] /. sym["IntermediateSymbols"]

  (* This definition is only found if an intermediate symbol hasn't \
been defined yet. It assigns a new one, 
  appends it to the intermediate symbols list, 
  and sets the desider value *)
  sym /: Set[sym[index_], rhs_] /; $guardOver := 
       Block[{$guardOver = False},
     sym[index] = ref;
      HoldPattern[sym[index]] :> ref];
     ref = rhs



Posted 2012-06-22T09:27:50.550

Reputation: 40 993

+1 Great job, thank you Rojo. I must admit as somewhat 'intermediate' user I certainly will need a bit of time to thoroughly understand the solutions offered by Leonid and you. – gwr – 2012-06-22T15:53:09.553

Thanks! @grw, I just saw a tiny bug in the first solution (the one with MakeExpression) and fixed it. Be sure to use the new one. – Rojo – 2012-06-22T15:57:52.673

+1, Interesting version. I thought about something similar, but did not want to introduce extra dependencies on extra global symbols. On my tests, your code is about 20 percent slower to modify the parts, than mine, however. I am on Win7 64 bit, M 8.0. – Leonid Shifrin – 2012-06-22T17:43:27.400


Here is one way I found doing this:

setIndexed[var_, val_, index__] := Module[


  tempVar = var;
  tempVar[[index]] = val;

matM[i] = ConstantArray[1,{10,10}];

matM[i] = setIndexed[matM[i],10,1,10]

Seems to work fine. But maybe there is a more elegant way to do this?


Posted 2012-06-22T09:27:50.550

Reputation: 11 083

As others have hinted, the more elegant way might be not to use an indexed array, but perhaps we need more information to know why an indexed array is the best solution? – image_doctor – 2012-06-22T10:52:12.950

The reasons for using indexed variables as opposed to lists is addressed in the Mathematica reference ( Why should these reasons - sparse definitions made only when needed - not apply when the variable contains an array?

– gwr – 2012-06-22T11:16:42.583

Clearly this is a good question, as it is generating a lot of debate :) Have you found the built in SparseArray doesn't quite fit your application? – image_doctor – 2012-06-22T11:44:40.510

As I had written to Szabolcs the reason for choosing an index was the more simple way to change existing code. But the SparseArray is simply the list-solution with better allocation of resources in case of lots of empty cells? I can't see how this is a replacement for an indexed var which essentially allows dynamic programming? – gwr – 2012-06-22T13:29:45.173

Testing this runs as fast as direct assignment for a non-indexed var. Does this solution have the same memory disadvantages as ReplacePart does? (the indexed-var needs to be copied and reassigned) – gwr – 2012-06-22T14:53:36.147


Here is another way.


    Set[symb[idx__][[part__]], var_] ^:= set[symb[idx][[part]],var];

testing with:

a[x] = {1, 2, 3}
a[x][[2]] = 7

we get, as in @Mr.Wizard test:

{1, 7, 3}

I like the fact that this solution keep the Downvalues in a, instead of a random named Global variable x. I don't like to Unprotect Part, but in this case doesn't seem to be bad.

In time performance this is not a good answer when compared with @Leonid and @Mr.Wizard.


Posted 2012-06-22T09:27:50.550

Reputation: 23 859


matM[1] is a a call to the function matM using the argument 1, not an index. To assign the array to an index of matM, use:

 matM = {{}};
 matM[[1]] = ConstantArray[1, {10, 10}];
 matM[[1]][[1, 10]] = 10

The first line is just to make sure matM has a first element to write the array into.

Edit: ok so the exact semantics I chose may have been slightly to lighthearted, but the point is that matM[1] is definitely not a symbol, which is also what Mathematica complains about. Even so you can still assign a value to it, just like you would assign a "function" matM[1]=2, compared to matM[x_]=2. The difference being that the pattern is a constant 1 in one and and arbitrary match x_ in the second. But simply because you can assign a value to the pattern does not mean you can treat it like a Symbol, if you consider the example:

 matM[x_] = ConstantArray[1, {10, 10}];     
 matM[1][[1, 10]] = 10

It should be immediately apparent why Mathematica warns you that matM[1] is not a symbol. In your particular case, the "pattern" used for the rule definition just happens to be unique, which means that there is no real conceptual problem in redefining part of the contained matrix.


Posted 2012-06-22T09:27:50.550

Reputation: 14 423

3Saying that "it's a call to the function with the argument 1" is a bit misleading. It's quite common to use variables named x[1], x[2], x[3], ... for symbolic manipulation or other tasks when we don't know beforehand how many variables we'll be needed, or when there's a large number of variables that we need to work with and it's inconvenient to refer to them 'manually'. I think this is clear to the OP. – Szabolcs – 2012-06-22T09:45:07.853

Thanks. But using the 'fixed' argument in a function is explicitly suggested in the Mathematica documentation ( and for example in Sal Mangano's "Mathematica Cookbook". It is of course possible to make this an array adding the index dimension, but if - as in my real case - you want to use symbols like matM[a] is not so easy to do.

– gwr – 2012-06-22T09:47:58.000

1@gwr I expanded my answer a bit, the important note is that matM[a] is not a symbol, and this was also what I was hinting towards originally. You can have some behavior which resembles what you have with symbols, but in other situations it falls short exactly because it isn't a symbol. – jVincent – 2012-06-22T11:37:18.377

@jVincent, thank you for expanding your answer and it does help to clear the issue. Nevertheless it is a 'pity' that even the subscripted variable cannot be addressed by Part and Set. Essentially then you cannot have a truly indexed symbol? – gwr – 2012-06-22T13:24:27.943

1@gwr Indexed variables are really a hash-table under the name of the corresponding symbol, and it points at values rather than assignable memory locations, which is IMO quite natural. I think, my solution gives you the closest to "truly indexed symbols" you can get, since I am using intermediate symbols, which are, indeed, "truly indexed". – Leonid Shifrin – 2012-06-22T13:35:57.193