What's the most "functional" way to do Cholesky decomposition?

14

3

I can do Cholesky in a procedural style, such as:

ProceduralCholesky[matrin_List?PositiveDefiniteMatrixQ] := 
 Module[{dimens,
   ll},
  dimens = Length@matrin;
  ll = ConstantArray[0, {dimens, dimens}];
  Do[
   Do[
     If[i == j,
       ll[[i, i]] = 
         Sqrt[matrin[[i, i]] - 
           Sum[Conjugate[#]*# &@ll[[i, k]], {k, 1, i - 1}]];,
       ll[[i, 
          j]] = (matrin[[i, j]] - 
            Sum[ll[[i, k]]*Conjugate[ll[[j, k]]], {k, 1, j - 1}])/
          Conjugate[ll[[j, j]]];];,
     {i, j, dimens}];,
   {j, dimens}];
  ll
  ]

Moreover, I've seen a "half-functional" implementation, which, however, features a Table function and an outer For loop. So far I have managed to address the need for the Table function by writing:

HalfFunctionalCholesky[matrin_List?PositiveDefiniteMatrixQ] := 
 Module[{dimens,
   uu},
  dimens = Length@matrin;
  uu = ConstantArray[0, {dimens, dimens}];
  Do[uu[[i]] = makerow[matrin, i, uu, dimens], {i, dimens}];
  uu
  ]

makerow[matrin_List, rowindex_Integer, uu_, dimens_Integer] := 
 PadLeft[
  (Join[
      {#},
      offdiagonalelements[matrin, uu, rowindex, dimens, #]]) &@
   diagonalelement[matrin, uu, rowindex],
  dimens]

diagonalelement[matrin_, uu_, rowindex_] := 
 Sqrt[matrin[[rowindex, rowindex]] - Conjugate[#].# &@
   uu[[;; (rowindex- 1), rowindex]]]

offdiagonalelements[matrin_, uu_, rowindex_, dimens_, 
  diag_] := (matrin[[rowindex, (rowindex+ 1) ;; dimens]] -
    Conjugate[
      uu[[;; (rowindex - 1), 
       rowindex]]].uu[[;; (rowindex - 1), (rowindex + 1) ;; 
       dimens]])/diag

I am not satisfied yet. Can I avoid using the outer Do loop by turning it into a Fold or a Nest? The only idea I was able to come up with involved a Nested ReplacePart function, but I think it would be just some pretentious sweeping under the rug. Am I mistaken?

Thank you in advance!

Editortoise-Composerpent

Posted 2012-06-11T14:19:44.773

Reputation: 1 126

3Welcome to the site! – Dr. belisarius – 2012-06-11T14:21:32.790

1In case you are not aware there is a built-in function CholeskyDecomposition. – Mr.Wizard – 2012-06-11T14:23:53.023

3Thanks for welcoming me! Anyway, I am aware of the existence of the built-in function and I have used it to test results for both implementations, but for my Computational Physics course I have to implement a Cholesky decomposition by myself. I am free to do it in whatever programming language I want, so I picked Mathematica since it is the one I am most familiar with. I know that the procedural implementation would be more than enough for my course, but I'm trying to learn functional programming as well by myself in the meanwhile. :) – Editortoise-Composerpent – 2012-06-11T14:25:33.473

4Here's a functional solution: Nest[CholeskyDecomposition, matrix, 1] :P – rm -rf – 2012-06-11T14:43:16.220

@R.M Superb anti-answer! :) – Dr. belisarius – 2012-06-11T15:03:56.857

@R.M that's just being cheeky. :) – rcollyer – 2012-06-11T15:10:40.140

You might get some useful ideas from the notebook available here

– Daniel Lichtblau – 2012-06-11T15:40:26.640

I think one сould also consider LU decomposition as well for completeness. – faleichik – 2012-06-11T20:29:05.343

2As a tiny note, PositiveDefiniteMatrixQ[] internally computes a Cholesky decomposition to prove the positive-definiteness of a matrix, so in effect, you're doing a Cholesky decomposition twice... :) – J. M.'s ennui – 2012-06-12T09:20:12.367

I tried the original posting which seems to work well with a small set of data, then applied a 100 x 100 covariance matrix which worked on CholeskyDecomposition[covMat]), but applying it to the ProceduralCholesk[covMat] I get "Infinite expression 1/0.x10-4 encountered"? Not sure how to copy and paste my covMat data for you to compare if it matters? – sebastian c. – 2013-01-18T12:50:27.973

I guess it's because my code fails to work whenever a zero has been previously computed on the leading diagonal of the lower triangular matrix. I should have added an exception, but I was probably too lazy to do it at the time. :D I'm just wondering if there are any precision issues involved or if you are actually "supposed to" have a zero in that element, though. – Editortoise-Composerpent – 2013-01-21T18:30:52.177

Answers

9

For reference there is a built-in function CholeskyDecomposition.

For improving your existing code Array may be a minor subjective improvement:

HalfFunctionalCholesky2[matrin_List?PositiveDefiniteMatrixQ] := 
 Module[{dimens, uu},
  dimens = Length[matrin]; 
  uu = ConstantArray[0, {dimens, dimens}]; 
  Array[(uu[[#]] = makerow[matrin, #, uu, dimens]) &, dimens];
  uu
 ]

After a look through this function, following your instincts I believe Nest may be used:

HalfFunctionalCholesky3[matrin_List?PositiveDefiniteMatrixQ] :=
  Module[{len = Length[matrin], i = 1},
    Nest[Append[#, makerow[matrin, i++, #, len]] &, {}, len]
  ]

Upon further examination I believe that the change to Nest allows a further simplification as follows (within Part):

diagonalelement[matrin_, uu_, rowindex_] := 
 Sqrt[ matrin[[rowindex, rowindex]] - Conjugate[#].# & @ uu[[All, rowindex]] ]

offdiagonalelements[matrin_, uu_, rowindex_, len_, diag_] :=
 (matrin[[rowindex, rowindex - len ;;]] - 
    Conjugate[uu[[All, rowindex]]].uu[[All, rowindex - len ;;]]) / diag

Mr.Wizard

Posted 2012-06-11T14:19:44.773

Reputation: 259 163

It indeed looks better, but wouldn't Append make it actually run slower than the procedural approach? – Editortoise-Composerpent – 2012-06-11T14:59:37.750

1@Andrea you're not a beginning user, are you? :-) Possibly, but I am working toward elegance, not performance. If your ultimate goal is performance you need to include that in your question. – Mr.Wizard – 2012-06-11T15:06:10.363

@Andrea how large are the matrices you will be using? – Mr.Wizard – 2012-06-11T15:08:00.953

An alternate to Append is a linked list form. It may speed things up, while retaining a measure of elegance. – rcollyer – 2012-06-11T15:11:32.247

@rcollyer I'm not sure that will be true in this case as the intermediate result needs to be used; it cannot be flattened (only) at the end. – Mr.Wizard – 2012-06-11T15:13:02.773

Now that I look at it, you're probably right; makerow would have to be rewritten to work with the linked form, and I'm not sure that is reasonable. – rcollyer – 2012-06-11T15:16:02.160

@Mr.Wizard Well, I've been using Mathematica for little more than a year, so I guess it depends on your definition of "beginning user". :D Anyway, I probably will not test it on matrices larger than 20 x 20, so I realize my concern might be moot. Thank you! – Editortoise-Composerpent – 2012-06-11T15:31:43.483

8

This is not a functional implementation (as it stands, it's rather MATLAB-ish), but I'll leave this snippet around and hope somebody could make something purely functional out of this outer product form of Cholesky decomposition:

m = Array[Min, {4, 4}]; (* example matrix *)

Do[
  m[[k, k]] = b = Sqrt[a = m[[k, k]]];
  m[[k, k + 1 ;;]] = (v = m[[k, k + 1 ;;]])/b;
  m[[k + 1 ;;, k + 1 ;;]] -= Outer[Times, v, v]/a;
  , {k, Length[m]}];
UpperTriangularize[m]

{{1, 1, 1, 1}, {0, 1, 1, 1}, {0, 0, 1, 1}, {0, 0, 0, 1}}

(added 6/12/2012)

As expected, the method given above can be made purely functional, but to me the functional version looks a lot less elegant:

k = 0; n = Length[m];
MapAt[Sqrt,
 Nest[Function[m, ++k;
   Block[{a = m[[k, k]], v = m[[k, k + 1 ;;]], s},
    s = Sqrt[m[[k, k]]];
    ArrayFlatten[{{ReplacePart[m[[1 ;; k, 1 ;; k]], {k, k} -> s], 
       DiagonalMatrix[Append[ConstantArray[1, k - 1], 1/s]].m[[1 ;; k, k + 1 ;;]]},
                 {0, m[[k + 1 ;;, k + 1 ;;]] - Outer[Times, v, v]/a}}]]],
  m, n - 1],
 {n, n}]

J. M.'s ennui

Posted 2012-06-11T14:19:44.773

Reputation: 115 520

3Sometimes loops are the right choice. Despite everyone's best efforts I did not feel that any answer truly improved on the original. – Mr.Wizard – 2012-06-12T14:06:51.587

Well, that question I asked on triangular recursions quite a while back also comes to mind. Still, I feel that outer product Cholesky is much more compact than OP's original route... – J. M.'s ennui – 2012-06-12T14:09:59.807