## How to implement the Observer design pattern?

20

11

As a programmer with a history with procedural languages, the Observer design pattern inmediately springs to my mind when attacking certain class of problems, mostly UI/notifications related.

A two-pronged question:

• Is the Observer pattern well suited to Mathematica style(s)? Or, more generally, can event-driven programming, being mostly about state and asynchronous notifications, be implemented efficiently/elegantly in Mathematica? Dynamic seems promising, but is a FE solution.

• In any case --for educational purposes, maybe,-- how would you implement a bare-bones Observer mechanism? I know that taming all *Values and assignments is hard or directly impossible without hooks. Let's settle with somewhat useful, instead of perfect: only OwnValues, simple Protect[], very simple notification callbacks.

Something along the lines of:

ClearAll[a];
a[b_] := b;
a = 2;
Observe[a, observer1 = Print["1.- ", ToString[Unevaluated[#1]], ": ", #2] &];
a = 3


1.- a: 3
3

Observe[a, Print["2.- ", ToString[Unevaluated[#1]], ": ", #2] &];
a = 4


1.- a: 4
2.- a: 4
4

DownValues[a] = DownValues[a]~Join~{HoldPattern[a[c_]] :> c};
a /: b[Unevaluated[a]] := b;

Set::write: "Tag a in DownValues[a] is Protected."
TagSetDelayed::write: "Tag a in b[a] is Protected."

RemoveObservations[a,observer1];
RemoveObservations[a];
a = 5


5

Related:

and possible starting points:

18

### General considerations

To my mind, the only robust way to do this is to build some custom object model in Mathematica, and in particular to restrict the way values can be changed to some well-defined route you can control. Because, as it follows from one of the discussions you linked to, there seems to be no reliable way to intercept arbitrary value changes at the level of the core language, that would've been exposed to the user and would work in absolutely all cases ( I will be happy to retract this statement as soon as we collect enough evidence that there are some means (ValueFunction or some other) which are 100% robust and cover absolutely all routes via which values can be changed).

But even if there is such a function which can trigger changes in all cases, I still think that it would be conceptually wrong to base an observer implementation on it, because you'd leave no space in between the core language and your code, to add more hooks, intermediate layers, etc. The observer pattern is rather closely tied to the OO paradigm, and value change is only one of many events one may want to trigger. The proper way of doing that would involve a full-fledged object model, which is not directly supported by the main Mathematica programming paradigm, primarily because of the different approaches to (im)mutability for the former and the latter.

### Example: OCaml-style references

That said, here is a simple example - an OCaml style references emulated in Mathematica. This is a small helper macro:

ClearAll[withCodeAfter];
SetAttributes[withCodeAfter, HoldRest];
withCodeAfter[before_, after_] := (after; before);


and here is the main code:

ClearAll[Ref, $Observers, MakeRef, Observe, RemoveObserver ];$Observers[_] = {};
SetAttributes[Ref, HoldAll];
MakeRef[expr_] := Module[{sym = expr}, Ref[sym]];
Ref /: Value[Ref[sym_]] := sym;
Ref /: Set[ref : Ref[sym_], new_] :=
withCodeAfter[
sym = new,
Scan[Function[obs, obs[ref, new]], $Observers[ref]] ]; Ref /: Set[lhs_Symbol, rhs_Ref] := withCodeAfter[ OwnValues[lhs] = HoldPattern[lhs] :> rhs; lhs , lhs /: Set[lhs, val_] := (rhs = val) ]; Ref /: Observe[ref_Ref, observer_] := AppendTo[$Observers[ref], observer];
Ref /: RemoveObserver[ref_Ref, observer_] :=
$Observers[ref] = DeleteCases[$Observers[ref], observer];


(some bad formatting here is due to the SE editor bug, related to the treatment of the $ symbol). Here is how you could use it: ClearAll[ref] ref = MakeRef[12] (* Ref[sym$775610] *)

ref = 10

(* 10 *)

Value[ref]

(* 10 *)

Observe[ref, observer1 = Print["1.- ", ToString[Unevaluated[#1]], ": ", #2] &]

(* {Print["1.- ", ToString[Unevaluated[#1]], ": ", #2] &}  *)

ref = 15

During evaluation of In[432]:= 1.- Ref[sym$775610]: 15   (* 15 *) Observe[ref, Print["2.- ", ToString[Unevaluated[#1]], ": ", #2] &] (* {Print["1.- ", ToString[Unevaluated[#1]], ": ", #2] &, Print["2.- ", ToString[Unevaluated[#1]], ": ", #2] &} *) ref = 20  During evaluation of In[434]:= 1.- Ref[sym$775610]: 20

During evaluation of In[434]:= 2.- Ref[sym$775610]: 20  (* 20 *) RemoveObserver[ref, observer1]; ref = 30   During evaluation of In[438]:= 2.- Ref[sym$775610]: 30

(* 30  *)


I've made varibles to which we assign Ref-s to have a "sticky" behavior, which simplifies assigment syntax.

It should be clear however, that what I suggested is only an emulation. Note in particular, that the fact that I explcitly overloaded assigment operator to work on symbols, which I assign to store Ref-s, immediately introduced weaknesses: now my model work for this type of assignments, but not direct DownValue - based assignments. This is because we came too close to the core language layer: should we always stick to Ref's API, and this will always work - as long as we follow this protocol.

### Example: Extensible SetterBar

Here is another example, where the event-handling will make more sense, since this one will be about UI. There is a SetterBar standard widget in Mathematica, but it has a number of limitations. You can't add more choices dynamically, once you have constructed it, and it is hard to make it execute some custom action on user's clicking on some tab.

Let's build an extensible SetterBar, which would allow adding more choices at run-time, as well as adding / removing listeners. I will use Module's kernel variables, which is generally not considered a good practice. It works for me, but be warned.

So, here goes:

ClearAll[makeExtensibleSetterBar]
makeExtensibleSetterBar[labels_List, handler_: Identity] :=
Module[{instance, setter, current, localLabels = labels, handlers = {handler}},
SetAttributes[instance, HoldAll];
setter :=
SetterBar[
Dynamic[current, (Scan[Function[fun, fun@#], handlers]; current = #)&],
localLabels
];
instance @ addHandler[f_] := handlers = DeleteDuplicates[Append[handlers, f]];
instance @ removeHandler[f_] := handlers = DeleteCases[handlers, f];
instance @ add[lab_] := AppendTo[localLabels, lab];
instance @ remove[lab_] := localLabels = DeleteCases[localLabels, lab];
instance @ replace[lab_ -> new_] :=localLabels = localLabels /. lab -> new;
instance @ getCurrent[] := current;
instance @ setCurrent[lab_] := current = lab;
instance @ getLength[] := Length[localLabels];
instance @ render[] := Dynamic[setter];
instance];


I'll show just a couple of examples of how this can be used. Start with this setter bar object:

sb = makeExtensibleSetterBar[{1, 2, 3, 4}]

(* instance$7475964 *)  To render it, call: sb@render[]  Now, you can press buttons on the setter bar a few times - so far, nothing happens, of course. Let's now add another choice:  sb@add[5]  You should see now that the new choice has been dynamically added to the UI, making it available for you to click on. Let's make it a bit more interesting and add some event handler: sb@addHandler[handler1 = Print["Handler1: you picked", #] &];  Now, try clicking on the buttons on the setter bar, and you should see things getting printed as you click. Add another handler: sb@addHandler[handler2 = Print["Handler2: you picked", #] &]  Now, this one will also print, as you click. You can remove them - say, the first one: sb@removeHandler[handler1]  And now only the second one will print stuff. In any case, this example is much closer in spirit to the model which the observer pattern would fit. Because of the enchanced level of encapsulation, it feels more natural to only use the API defined above - it makes no sense to assig anything directly to the object instance itself (which we represent by a symbol we return). Besides, this is a useful piece of functionality, in its own right, since it significantly extends the capabilities of the built-in SetterBar. ### Notes The approaches I outlined may work well or not so well, but one would need a more sophisticated object model to make this look & feel more natural. One can build that in Mathematica too, but there will be a number of limitations of such top-level implementations. They've been discussed here and on SO more than once. I was in this very moment reading your OO hard and other Mathematica OO stuff. Now I must needs study some more. – Fallible – 2014-05-11T18:36:28.600 @Fallible The discussions about OO in Mathematica recur every now and then, and the reason is, I think, that this is a practically important topic. By the way, I think I've figured out how to add a pretty powerful support for class fields via Associations (new in V10), so as soon as get a little extra time, and perhaps a bit more experience with this style, I will update my OO package. Hopefully, this would make it way more powerful and also somewhat simpler to use. I am also thinking of borrowing some features from Python - I am a big fan of Python's object model. – Leonid Shifrin – 2014-05-11T18:40:23.600 Python is a beauty, in at least one aspect similar to M: the batteries-included aspiration. Another wonder, Smalltalk, the granddaddy (IMO still the best, with a great library too) of OO systems pionered MVC/Event-Handling and all that. We tend to associate, even identify, those patterns with OO. I don't believe that relation to be so tight: OO techniques eases but not precludes MVC-type solutions, but this is not the place nor the time to discuss such subtleties (not that I could argue with you). – Fallible – 2014-05-12T08:35:09.363 Your answer exemplifies the strong points of OO: abstraction, extensibility, decoupling, and as a bonus shows us the tricky part of MVC architectures: the controller. I love your OO constructs and will use it a lot, by necessity: I can't avoid thinking OO. BTW, MVC is the canonical foundation of most GUI frameworks. Dynamic is the Mathematica way of building GUIs, and one of the few I'm aware of that it is not MVC based, though it uses probably Observer under the hood. – Fallible – 2014-05-12T08:38:14.457 1I knew my question was problematic, mixing a broad, theoretical issue, with a practical, very constrained problem. You shot at the first brilliantly, and answered the stated question. MrWizard, the second, with a good-enough, simple solution. As a newcomer, I'm interested in quickly adquire the Mathematica mindset, in learning how to attack problems using other, for me more exotic, paradigms. Although I can't raise one single objection to your post, I currently favor other approaches to my question, like Mr.Wizard's. Let's wait a little. – Fallible – 2014-05-12T08:41:53.213 Side note: instance$7475964? LOL, your sessions are long-lasting, for sure, or use a lot of scoping and objects. I'm paranoid and have little confidence in the system, rebooting the kernel frequently. That would be fun, who's seen the higher Unique[] in normal use. – Fallible – 2014-05-12T08:49:54.403

@Fallible Thanks for the interesting comments. Re: Smalltalk - is on my list of languages to learn. More precisely, I am interested in this dialect. Also eventually I'd really like to understand the Seaside web framework. Re: MVC - seems to not be the only game in town, but that depends on who you ask. The dominant opinion is that it is, but as with other tools, I suspect that the mainstream best practices don't really give the most power to the hands of solo developer. Mathematica (Dynamic) is using a version of functional reactive programming, de facto.

– Leonid Shifrin – 2014-05-12T10:31:06.670

@Fallible Re: OO and patterns - I tend to agree with the opinion that all patterns represent certain weaknesses of the language, and are invisible in languages which have that as a language feature. What seems really important is to be able to adapt the language to one's needs, and add those things which you find doing over and over, as a new language feature. That's why Lisp and its dialects are so great. I think, it's always about the balance between starting from scratch vs using some existing infrastructure.

– Leonid Shifrin – 2014-05-12T10:39:07.627

@Fallible Re: instance\$7475964 - you're right on both points - long-lasting sessions and a lot of usage of scoping stuff. Scoping is a powerful programming tool, I think it is generally underappreciated how much one can gain from using it in the right places. I'd say that the two main dimensions for effective programming (in Mathematica at least, but I think this is more general) is scoping / encapsulation / degree of openness, and mutable vs immutable code. – Leonid Shifrin – 2014-05-12T10:44:51.720

@LeonidShifrin your answer is so useful, I think it should be included in the language as a means for asynchronous programming. Maybe Java could be used also in Mathematica for asynchronous programming, although I haven't looked at this yet. – faysou – 2015-01-31T10:10:52.407

@faysou Thanks! Well, I've been using these techniques myself, quite successfully. But, such constructs would need a wide language support, to appear in the docs. For example, using kernel variables in the UI is considered a bad practice, although one can make it work if one knows what to do. As to inclusion in the language - yes, I agree. For example, having native Ocaml-style references integrated into the language would've been great. May be, something like that would appear some day. – Leonid Shifrin – 2015-01-31T12:23:44.990

13

Utilizing ValueFunction as mentioned here, we could do something like this:

SetAttributes[Observe, HoldFirst];

Observe[s_Symbol, fn_] :=
With[{VF := ExperimentalValueFunction[s]},
VF =
If[ValueQ @ VF,
ReplacePart[VF, {2, 2, 2} -> Append[VF[[2, 2, 2]], fn]],
Function[Null, Table[f[##], {f, {fn}}], HoldAll]
];
]


We will need to include Hold attributes in your Print functions for them to work correctly; I shall use:

ofunc1 = Function[Null, Print["1.- ", ToString[Unevaluated[#1]], ": ", #2], HoldFirst];

ofunc2 = Function[Null, Print["2.- ", ToString[Unevaluated[#1]], ": ", #2], HoldFirst];


Now:

ClearAll[a];
a[b_] := b;
a = 2;
Observe[a, ofunc1];
a = 3


1.- a: 3

 3

Observe[a, ofunc2];
a = 4


1.- a: 4

2.- a: 4

4


### Removing observers

SetAttributes[RemoveObservations, HoldFirst]

RemoveObservations[s_Symbol] := (ExperimentalValueFunction[s] =.)

RemoveObservations[s_Symbol, fn_] :=
With[{VF := ExperimentalValueFunction[s]},
If[ValueQ @ VF,
VF = ReplacePart[VF, {2, 2, 2} -> DeleteCases[VF[[2, 2, 2]], fn]];
]
]


Now:

RemoveObservations[a, ofunc1]

a = 7


2.- a: 7

7


So, adding DownValues to ValueFunction (a "external" object) after checking that it's not protected. And no need to overwrite Set. That the kind of things that I'd have never thought with my "procedural/functional" background. Brilliant. – Fallible – 2014-05-11T18:50:48.100

Although I like your solution --in this point of my learning I'm looking for more idiomatic constructs,-- Leonid's is probably more useful for the community at large. – Fallible – 2014-05-14T11:42:57.830

9

In general I would say that the Observer pattern is not applicable to Mathematica code, since in my experience most Mathematica code is not event driven and the Observer pattern makes no sense without events.

But a GUI application taking a state-machine approach and using Refresh with the option TrackedSymbols, however, can certainly implement the Observer pattern by means of a state-machine.

Here is a relatively brief working example:

SeedRandom[42];
Manipulate[
Row[{
Dynamic@Refresh[
If[event != "idle", update[]]; Column[{plot, mean}],
TrackedSymbols -> {event}],
Dynamic @ Refresh[event = "slider-n-changed"; "", TrackedSymbols -> {n}],
Dynamic @ Refresh[event = "slider-k-changed"; "", TrackedSymbols -> {k}],
Dynamic @ Refresh[event = "slider-z-changed"; "", TrackedSymbols -> {z}]}],
{n, 10, 25, 1, Appearance -> "Labeled"},
{k, 1, 10, 1, Appearance -> "Labeled"},
{{z, 200, "zoom"}, 100, 300, 25, Appearance -> "Labeled"},
{{event, "idle"}, ControlType -> None},
{{data, dataF[10]}, ControlType -> None},
{plot, ControlType -> None},
{{mean, meanF[data, 1.]}, ControlType -> None},
TrackedSymbols -> None,
ControlPlacement -> Bottom,
Initialization :> (
dataF[n_] := RandomInteger[{1, 100}, n];
plotF[data_, k_, z_] := ListPlot[k data, ImageSize -> z];
meanF[data_, k_] := Row[{"Mean = ", N@Mean[k data]}];
update[] := Module[{ev = event},
event = "idle";
Switch[ev,
"slider-n-changed",
data = dataF[n]; plot = plotF[data, k, z]; mean = meanF[data, k],
"slider-k-changed",
plot = plotF[data, k, z]; mean = meanF[data, k],
"slider-z-changed",
plot = plotF[data, k, z]];])]


Notice how the controls which produce the events are decoupled from the variables that determine the look-and-feel of the visual output in the content pane. This, essentially, implements the Observer pattern.

When the n-slider is moved, a new data set is computed and the plot and the mean are updated; when the k-slider is moved only the plot and the mean are updated; when the zoom slider is moved only the plot is updated.

Note: this answer is based on work published on-line by John Fultz, but any errors or awkwardness in the above code are entirely mine.

### References

I'm very interested in all things Dynamic, and your example is very interesting. But I think that trying to approximate solutions in the kernel to the kind of problems Observer is suited of will be fruitful and enlightening. – Fallible – 2014-05-11T18:30:54.220

@m_goldberg The Observer (or MVC) pattern is incorporated in Mathematica through Dynamic, Manipulate, etc. So, Observer is very applicable to Mathematica code. – Anton Antonov – 2016-03-20T16:58:59.037

4

This is the code I was working on and the reason I posted the question.

It smells, it' s un-Mathematica-y, it lacks the idioms, the style and the form of sound Mathematica code. But I believe it could be instructive to others that like me are climbing the steep learning-curve of mountain M., most of all in contrast to the other answers.

ClearAll[Observe, ObservedQ, RemoveObservations];
Module[
{Observers},

SetAttributes[Observe, HoldFirst];
Observe[sym_Symbol, callback_] :=
With[{Us = Unevaluated[sym]},
Us /: Observers[Us] = {callback};
Us /: Set[sym, v_] := (
Unprotect[sym]; OwnValues[sym] = HoldPattern[sym] :> v; Protect[sym];
#[Us, v] & /@ Observers[Us];
v);
Protect[Us];
];

Observe[sym_Symbol?ObservedQ, callback_] :=
With[{Us = Unevaluated[sym]},
Unprotect[Us];
Us /: Observers[Us] = Observers[Us] \[Union] {callback};
Protect[Us];
];

SetAttributes[ObservedQ, HoldFirst];
ObservedQ[sym_Symbol] :=
MemberQ[UpValues[sym], HoldPattern[_[Observers[_]]] :> _];

SetAttributes[RemoveObservations, HoldFirst];
RemoveObservations[Shortest[sym_Symbol?ObservedQ], callback__] :=

With[{Us = Unevaluated[sym]},
Unprotect[Us];
If[Length[
Us /: Observers[Us] =
DeleteCases[Observers[Us], Alternatives @@ {callback}]] > 0,
Protect[Us],
RemoveObservations[Us]];
];

RemoveObservations[sym_Symbol?ObservedQ] :=
With[{Us = Unevaluated[sym]},
Unprotect[Us];
Us /: (sym = v_) =.;
Us /: Observers[Us] =.;
]
]


In hindsight, I forgot that in programming a new problem is resolved adding another layer of indirection (and in graphics programming, adding a minus sign), an idea that does not come naturally to the mind in Mathematica. Leonid Shifrin's Ref construct would have streamlined the flow, making irrelevant all those unwieldy Unevaluateds and Protect`s.