SquaresR memory leak?



I have tried the following code in Mathematica on my MacBook:

$HistoryLength = 0;

AbsoluteTiming[Do[SquaresR[2, i], {i, 1000000}];]


There seems to be much more memory in use at the end than at the beginning. I assume the values SquaresR[2,i] (or some related things) are saved, because when I run this a second time, it runs much faster. Is there a way to free this memory, without restarting the kernel?


Posted 2017-01-19T18:24:59.163

Reputation: 133

3At a glance I would consider it a bug or at least oversight that ClearSystemCache[] does not clear these values. – Mr.Wizard – 2017-01-19T18:45:29.913




Under the hood System`SquaresR is still calling functions in the context NumberTheory`.

Partial output of:


PrintDefinitions @ SquaresR
SquaresR[2, NumberTheory`SquaresRDump`n _ Integer?Positive] := 
   NumberTheory`SquaresRDump`res = 
   NumberTheory`SquaresRDump`res /; IntegerQ[NumberTheory`SquaresRDump`res]];

So all these calls to SquaresR[2, i] and handed off to NumberTheory`SquaresRDump`squaresR2.

PrintDefinitions @ NumberTheory`SquaresRDump`squaresR2
(* contexts omitted *)

squaresR2[0] := 1;

squaresR2[1] := 4;

squaresR2[n_] := 
  Block[{res, fact}, 
   fact = FactorInteger[
     n]; (res = 
      If[MemberQ[fact, {p_ /; Mod[p, 4] == 3, _?OddQ}], 0, 
       4*Times @@ Cases[fact, {p_ /; Mod[p, 4] == 1, a_} :> a + 1]]; 
     squaresR2[n] = res) /; FreeQ[fact, $Failed | FactorInteger]];

squaresR2[___] := $Failed;

Note the command squaresR2[n] = res -- this is memoization (caching).


We can clear all these memoized values and reload the SquaresR package as follows:


Get[System`Private`$SystemFileDir <> 
  System`Dump`fixfile["NumberTheory`SquaresR`"] <> "x"]


There used to be a more clear distinction between core functions and those in standard packages. Core functions were typically faster, often being at least partially implemented in C, and better integrated with things like ClearSystemCache. Standard package functions were slower, being written in high-level Mathematica code, and one would not expect them to be entirely integrated into the system.

In recent versions however standard packages have been deprecated and replaced by functions in the System` context. At first one would think these are core functions but in many cases (e.g. SquaresR) they are not. I find this less desirable as one can no longer expect a System` function to be fast or well integrated. So in this case I would not consider the "failure" of ClearSystemCache to be a flaw, but rather I consider it a flaw in the recent namespace organization of Mathematica.


Posted 2017-01-19T18:24:59.163

Reputation: 259 163

With the revision of the RT, I have a feeling that things will get more clear in the future (but this is just a feeling). At the same time, we all know that the language will have to, at a given moment, embrace the open trends. And having a clear division between the core and the extremely big package (the other 98%...), is, in my opinion, a fundamental step (but this is just an opinion based on a feeling...). – P. Fonseca – 2017-01-20T14:13:25.627

@P.Fonseca To what do you refer with the revision of the RT? You touch on something I have long hoped for: the separation of the core language into a stand-alone free distribution, hopefully with new refinements and improvements to performance and perhaps design. The huge function library would then become the commercial product built on top of this language. Is that what you hope for too? – Mr.Wizard – 2017-01-20T20:08:04.797


That's it. Take a look at this https://youtu.be/pzjhFF0wJiw. I have also heard that they are touching the pattern matching artillery. A lot of this is for certainly motivated by the new compiler. I really hope that we will have this on M12 (that should be something like 2018...). This means that everything that is core is being cleaned up, and that everything we can compile within the RT scope will most likely get close to any other "natively" achievable performance and robustness...

– P. Fonseca – 2017-01-21T14:06:26.290

...Then, we have to think on the following question: is the RT library an income competitor of Mathematica? If it isn't (and I don't think it can be), there's the unique opportunity to get to the 21st century business model, while keeping the financial independency of the 80's! – P. Fonseca – 2017-01-21T14:06:31.017

@P.Fonseca A very interesting and encouraging presentation; thank you. – Mr.Wizard – 2017-01-26T14:59:20.817