Detrending a TimeSeries

4

I am trying to de-trend a time series that I know has some seasonality. So I thought I would try to use TimeSeries and DateObject functionality to do it. Here's what I have so far :

input = {{"Date", 
"Count"}, {DateObject[{2009, 8, 24}, 
 TimeObject[{0, 0, 0.`}, TimeZone -> 0.`], TimeZone -> 0.`], 
327.29`}, {DateObject[{2009, 8, 25}, 
 TimeObject[{0, 0, 0.`}, TimeZone -> 0.`], TimeZone -> 0.`], 
347.6`}, {DateObject[{2009, 8, 26}, 
 TimeObject[{0, 0, 0.`}, TimeZone -> 0.`], TimeZone -> 0.`], 
294.`}, {DateObject[{2009, 8, 27}, 
 TimeObject[{0, 0, 0.`}, TimeZone -> 0.`], TimeZone -> 0.`], 
311.`}, {DateObject[{2009, 8, 28}, 
 TimeObject[{0, 0, 0.`}, TimeZone -> 0.`], TimeZone -> 0.`], 
786.`}, {DateObject[{2009, 8, 29}, 
 TimeObject[{0, 0, 0.`}, TimeZone -> 0.`], TimeZone -> 0.`], 
867.`}, {DateObject[{2009, 8, 30}, 
 TimeObject[{0, 0, 0.`}, TimeZone -> 0.`], TimeZone -> 0.`], 
760.`}, {DateObject[{2009, 8, 31}, 
 TimeObject[{0, 0, 0.`}, TimeZone -> 0.`], TimeZone -> 0.`], 
643.`}, {DateObject[{2009, 9, 1}, 
 TimeObject[{0, 0, 0.`}, TimeZone -> 0.`], TimeZone -> 0.`], 
435.`}, {DateObject[{2009, 9, 2}, 
 TimeObject[{0, 0, 0.`}, TimeZone -> 0.`], TimeZone -> 0.`], 
456.`}, {DateObject[{2009, 9, 3}, 
 TimeObject[{0, 0, 0.`}, TimeZone -> 0.`], TimeZone -> 0.`], 
235.`}, {DateObject[{2009, 9, 4}, 
 TimeObject[{0, 0, 0.`}, TimeZone -> 0.`], TimeZone -> 0.`], 
346.`}};

assoc = AssociationThread[First[input] -> #] & /@ Rest[input];
v = Flatten[Values[assoc[[All, {"Count"}]]]];
t = Flatten[Values[assoc[[All, {"Date"}]]]];
ts = TimeSeries[v, {t}];
tsmodel = TimeSeriesModelFit[ts]
fitmodel = Fit[ts, {1, x, x^2}, x]

Questions

  1. Is the Flatten really the best way to go from an association to a TS? I tried a number of ways but that was the only one that worked.

  2. How do I subtract the fitted model(s) from the Time series? I have tried the technique here but no luck.

Gordon Coale

Posted 2016-03-14T23:39:40.310

Reputation: 2 241

An answer to your first question: you can try using TimeSeries directly, e.g. DateListPlot@TimeSeries[Rest@input] . – Anton Antonov – 2016-03-14T23:53:17.843

An answer to your second question. Does the following command do what you want? DateListPlot[{TimeSeries[Rest@input] - TimeSeries[tsmodel /@ input[[2 ;; -1, 1]], {input[[2 ;; -1, 1]]}]} – Anton Antonov – 2016-03-14T23:59:10.307

Answers

11

  1. TimeSeries[Rest[input]] works directly without having to go through an association.

  2. If you really need to make an association: assoc = Inner[#2 -> #1 &, Rest[input], First[input], Association]. From there: TimeSeries[Values[assoc]].

  3. TimeSeriesMapThread[] is useful for detrending:

    ts = TimeSeries[Rest[input]];
    trend = LinearModelFit[ts, {1, x, x^2}, x];
    dts = TimeSeriesMapThread[#2 - trend[#1] &, ts];
    {DateListPlot[ts, PlotLabel -> "Original"], 
     DateListPlot[dts, PlotLabel -> "Detrended"]} // GraphicsRow
    

    before and after

J. M.'s ennui

Posted 2016-03-14T23:39:40.310

Reputation: 115 520

Lovely answer - I'm a database person at heart so my inclination is always to create query-able tabular structures like assoc's and DS's – Gordon Coale – 2016-03-18T08:40:35.797