DSYEVX on a 28000x28000 matrix

[jhelmsen]

Gentlemen,

We have been trying to spectrally separate classes of objects into 2 separate classes using eigenvalue decomposition. To this end, we have calculated a full symmetric distance matrix between the 28000 objects, and are trying to find the second largest eigenvalue and eigenvector. We need the distance matrix for other reasons as well, so it is easily available. This matrix is double precision and has 28000^2 elements, or a little under 6Gb. The machine, however, is a 64bit bladeserver with 16Gb of RAM, so storing the matrix should not be a problem. We have also tested the process on smaller matricies up to 12000 elements, and there does not seem to be a problem. The matrix has all entries between 0 and 1 with no 0's.

To this end, we calculated the matrix and used the LINUX standard rpm release of LAPACK 3.0. This did not work, and the system terminated with a convergence error. We have downloaded, compiled and installed CLAPACK 3.1.1, so we can see what we are doing and are investigating it now. (It takes about 6 hours to run).

My specific question is: Since 64-bit addressing is necessary to work with matricies of this size, is there any reason that CLAPACK will not work, and if there is, can it be fixed, or is there a straightforward alternate approach?

Of course, any comments are very much appreciated.

[Julien Langou]

I do not see a good reason for LAPACK to fail because of the 64-bit or the
size of your matrix.
It's a good idea to have switched to 3.1.1 though.
Can you give a try to DSYEVR? DSYEVD?
(You can try DSYEV but I am afraid, it's going to take more than a day.)
-julien.