Functional programming with Associations



Suppose I have an association where keys are dates, and values are returns of some portfolio, and I want to convert the returns to cumulative returns. Now, this can be done as follows:

    Module[ {porfolio = KeySort[port]},
        MapThread[Rule, {Keys[porfolio], Exp[FoldList[Plus, Values[portfolio]]]}]

But this seems clunky. Is there some more elegant way of achieving the aim?

Igor Rivin

Posted 2017-03-29T21:49:22.943

Reputation: 4 896

Can you post a short example of port? – Stitch – 2017-03-29T21:54:56.457

@Stitch Sure, here is a not-so-short example:

– Igor Rivin – 2017-03-29T22:28:06.093

Does anyone know why Accumulate[ <|"a" -> 1, "b" -> 2, "c" -> 3|> ] doesn't work? I thought this kind of thing was missing from version 10 simply because there wasn't time to add it, but it still doesn't work in version 11.1. – Mr.Wizard – 2017-03-30T02:24:45.293

@Mr.Wizard Would that no be because the internal ordering of the association has nothing to do a priori with what you type in, and thus it is not clear what order you are accumulating in? Just a thought... – Igor Rivin – 2017-03-30T04:14:13.603

@Mr. Wizard, to make Igor's thought explicit: would you expect Accumulate[ <|"a" -> 1, "b" -> 2, "c" -> 3|>] and Accumulate[<|"a" -> 1, "c" -> 3, "b" -> 2|>] to give the same results, or not? – J. M.'s ennui – 2017-03-30T04:57:20.140

@J.M. No I would not; KeySort is there if needed. If Association is too vague what about Dataset? – Mr.Wizard – 2017-03-30T05:45:21.997

1@Mr. Wizard, on a Dataset[] without named rows, I can see Accumulate[] working on it. Otherwise... – J. M.'s ennui – 2017-03-30T05:49:49.083



Best I can tell you really just want AssociationThread. It circumvents the Association and the MapThread.

p = AssociationThread[Range[100], Range[100]];
AssociationThread[Keys@p, N@Exp[FoldList[Plus, Values[p]]]]

That does what you were doing before in a few fewer calls. Also note that Associations support lots of functional programming stuff. KeyMap and Map exist of course and KeyValueMap is there too.

If you're wondering why FoldList doesn't do this natively consider that Fold andFoldList are supposed to be direct counter-parts. FoldList could return an Association with the keys realigned, but that wouldn't make sense with Fold. So there's no way to make an intuitive, compatible definition across both of them. And so best to one for neither of them. With AssociationThread rebuilding your association is trivial anyway, so I think they didn't really see the need either.


Posted 2017-03-29T21:49:22.943

Reputation: 42 610

There is a problem with this (which I pointed out in one of my comments to another answer) is that the ordering of keys in an association is (intentionally) non-deterministic, so one pretty much has to do a KeySort before this has any semantic content... Still, I had no idea AssociationThread existed! – Igor Rivin – 2017-03-30T00:07:23.950

1@IgorRivin Yes and no. It's like a hash-map with ordering. So it can be indexed just like a list. If you want something with keys ordered according to their hash values you can check out the internal System`Private`HashTable. Never figured out exactly how it works, so if you do post a Q&A type question on it for me to upvote. – b3m2a1 – 2017-03-30T00:14:26.757

I am guessing that the ordering depends on the hashing algorithm, which is not guaranteed to remain whatever it is today... – Igor Rivin – 2017-03-30T00:18:26.060

@IgorRivin Of course not. But few top-level structures (none I can currently think of) will impose an ordering on their equivalent of the hash table keys. You'll probably have to go internal if you want that. – b3m2a1 – 2017-03-30T00:20:23.010

1The documentation states that Sort and KeySort can be applied to associations to produce well-ordered associations. Furthermore, associations are documented to support numeric part indices and value updating from ordered lists. From these assertions, we must conclude that association element order is being actively preserved independently of the key representation (e.g. like Java's LinkedHashMap). Still, it would be nice if the documentation explicitly stated that association maintain their order rather than just these implicit statements. – WReach – 2017-03-30T05:49:52.380

Also, the distinction between "regular" associations and explicitly unordered associations is very suggestive, e.g. Outer[#2 @#@{"b" -> 1, "a" -> 2} &, {Association, Data`UnorderedAssociation}, {# &, InputForm, Internal`AssociationNodes}] // Grid. – WReach – 2017-03-30T05:51:51.483

@WReach oh I totally misunderstood what Igor was after. I must have assumed it was known that they'd stay ordered. You mention the LinkedHashMap in Java. In python there's the OrderedDict equivalent. – b3m2a1 – 2017-03-30T05:52:05.257

@WReach I'm guessing Data`UnorderedAssociation exists to get you back to constant indexing time rather than simply log2(n). – b3m2a1 – 2017-03-30T05:57:52.760


The shortest (and faster) way is to use Accumulate with pre-computed sorted Association:

eg = <|"2012-08-29" -> -0.2, "2012-08-30" -> 0.2, 
   "2012-08-31" -> -0.05, "2012-09-04" -> 0.05, "2012-09-05" -> -0.5, 
   "2012-09-01" -> -0.2, "2012-09-07" -> -0.3, "2012-09-10" -> 0.9|>;
egs = KeySort@eg;
AssociationThread[Keys@egs, egs // Values // Accumulate // Exp]

<|"2012-08-29" -> 0.818731, "2012-08-30" -> 1., "2012-08-31" -> 0.951229, "2012-09-01" -> 0.778801, "2012-09-04" -> 0.818731, "2012-09-05" -> 0.496585, "2012-09-07" -> 0.367879, "2012-09-10" -> 0.904837|>


Posted 2017-03-29T21:49:22.943

Reputation: 4 085

1That is a good way, though (to play devil's advocate :)) it ONLY works with Plus as the thing being accumulated... – Igor Rivin – 2017-03-30T01:58:48.757

1@IgorRivin Well, you know, they say there is a function for everything in MMA :) – Stitch – 2017-03-30T02:00:28.833


These are not functional solutions, but they are fairly concise owing to the "transparency" of associations to operators like Map and Exp:

Module[{t = 0}, Map[t += # &, eg]] // Exp

or Set:

Module[{e = eg}, e[[All]] = e // Values // Accumulate // Exp; e]

These solutions assume that the association eg is already in the desired order (like the file linked in the question -- use KeySort@eg if necessary).


Posted 2017-03-29T21:49:22.943

Reputation: 62 787


You could write a version of FoldList that works on Associations:

foldList[f_, assoc_Association] := 
    MapThread[Rule, {Keys@assoc, FoldList[f, Values@assoc]}] // Association

Then you can simply do

Exp /@ foldList[ Plus, KeySort@port ]


Posted 2017-03-29T21:49:22.943

Reputation: 3 333

True, but the principle is still the same, and the whole disassembling-reassembling process seems inelegant... – Igor Rivin – 2017-03-29T22:29:15.377

@Igor Absent something built in, I think one has to choose between disassembling-reassembling or for-loop-style iteration. – jjc385 – 2017-03-29T22:33:51.237

@Igor Aside from that issue, are you hoping for something more elegant? I find it hard to imagine something more elegant than a FoldList-like solution. – jjc385 – 2017-03-29T22:36:08.737

2I agree, but it is not obvious that something is not built in (Associations support some, but not all, list operations, so it seems a little underbaked, so maybe more baking had occurred while we were not looking?!) – Igor Rivin – 2017-03-29T22:36:19.937

1I guess part of the point is that there is no canonical order on the keys of an association, so things like Fold don't a a priori make sense, so maybe you are right - this maybe the best one can hope for... – Igor Rivin – 2017-03-29T22:38:05.490