Add to matrix missing elements (for list of time series)

5

This question is an extension of the 43122. I’ve looked through many interesting post (like this) but (as always) missed the direct matching. So the setting is simple:

datum = {{{2, g},         {6, h}, {7, k}}, 
         {{2, a}, {4, b},         {7, d}}, 
                 {{4, e},         {7, f}}}

Note: numbers {2, 4, 6, 7} are ordered but arbitrary (that differs with related question that seems promising). I need reliable and possibly concise method suitable for working with matrices of the size around 1000 x 1000 (that are as a rule time series). May be some specific ad hoc build-in methods.

Expected result ("X" any constant, i.e. Missing[]):

{
 {{2, g},   {4, "X"}, {6, h},   {7, k}},
 {{2, a},   {4, b},   {6, "X"}, {7, d}},
 {{2, "X"}, {4, e},   {6, "X"}, {7, f}}
}

PS. My current method seems, sort of, naive and long.

sample = Union @@ (#[[All, 1]] & /@ datum)
(* {2, 4, 6, 7} *)
step1 = Complement[sample, #[[All, 1]]] & /@ datum;
step2 = Thread[{#, Array["X" &, Length[#]]}] & /@ step1;
MapThread[Union, {datum, step2}]

Thanks in advance to everyone who respond.

garej

Posted 2016-01-06T21:33:56.010

Reputation: 4 695

Answers

3

Apply[List,
  Normal@*KeySort /@ KeyUnion[
    AssociationThread @@@ Transpose /@ datum
    , "X" &
  ]
  , {2}]

Explanation

AssociationThread @@@ Transpose /@ datum generates a list of Associations with the first elements (2, 4, 6, and 7) as the Keys. KeyUnion then takes each of those Associations and adds the Keys from the other Associations that are missing; the second argument specifies what should be done in the situation where a Key is Missing. KeySort does exactly what it seems to, Normal changes the Association to a list of Rules, and finally, we change each of the Rules to a List using Apply.

Old version:

assocs = Association /@ Apply[Rule, datum, {2}];
assocsUnion = KeySort /@ KeyUnion@assocs /. Missing[__] :> "X";
Apply[List, Normal@assocsUnion, {2}]

march

Posted 2016-01-06T21:33:56.010

Reputation: 20 806

2

datum = {{{2, g}, {6, h}, {7, k}}, {{2, a}, {4, b}, {7, d}}, {{4, e}, {7, f}}};

With[{uni = Union@Cases[datum, a_Integer :> {a, "X"}, 3]},
  Sort /@ DeleteDuplicatesBy[First] /@ (Join[#, uni] & /@ datum)] // MatrixForm

enter image description here

eldo

Posted 2016-01-06T21:33:56.010

Reputation: 34 072

2

This is an entirely pre V10 method:

With[{xtra = Thread[{Union@Flatten@datum[[All, All, 1]], Missing[]}]},
  Sort@DeleteDuplicates[Join[#, xtra], #1[[1]] == #2[[1]] &] & /@ datum
 ]

which gives:

(*

{{{2, g}, {4, Missing[]}, {6, h}, {7, k}}, {{2, a}, {4, b}, {6, 
   Missing[]}, {7, d}}, {{2, Missing[]}, {4, e}, {6, Missing[]}, {7, 
   f}}}

*)

Mike Honeychurch

Posted 2016-01-06T21:33:56.010

Reputation: 36 211

2

With set functions and a pure listable function

With[{indices = Union @@ datum[[All, All, 1]]},
 Union[Function[{v}, {v, Missing[]}, Listable][
     Complement[indices, #[[All, 1]]]], #] & /@ datum]

{{{2, g}, {4, Missing[]}, {6, h}, {7, k}}, {{2, a}, {4, b}, {6, Missing[]}, {7, d}}, {{2, Missing[]}, {4, e}, {6, Missing[]}, {7, f}}}

Hope this helps.

Edmund

Posted 2016-01-06T21:33:56.010

Reputation: 35 657