Problem with symmetric eigendecompositions on GPU

bdol · Post by **bdol** » Tue May 20, 2014 2:36 pm

Hello - I am seeing the following issue with all variants of the symmetric eigenvalue routines *syev. I am running the testing modules for these functions with the -c option to check their accuracy. With -N <= 128, the checks pass because MAGMA hands this small matrix off to the CPU:

Code: Select all

[bdol@n08 testing]$ ./testing_dsyevd_gpu -N 128 -c -JV
MAGMA 1.5.0 beta1 compiled for CUDA capability >= 1.0
CUDA runtime 5000, driver 5000. OpenMP threads 16. MKL 10.2.4, MKL threads 8. 
device 0: Tesla C1060, 1296.0 MHz clock, 4095.8 MB memory, capability 1.3
device 1: Tesla C1060, 1296.0 MHz clock, 4095.8 MB memory, capability 1.3
Usage: ./testing_dsyevd_gpu [options] [-h|--help]

    N   CPU Time (sec)   GPU Time (sec)
=======================================
  128     ---               0.15
Testing the factorization A = U S U' for correctness:
(1)    | A - U S U' | / (|A| N)     = 1.18e-17
(2)    | I -   U'U  | /  N          = 8.50e-17
(3)    | S(w/ U) - S(w/o U) | / |S| = 1.03e-17

However, as soon as I increase the matrix size to a value that causes MAGMA to hand it off to the GPU, the tests start failing:

Code: Select all

[bdol@n08 testing]$ ./testing_dsyevd_gpu -N 256 -c -JV
MAGMA 1.5.0 beta1 compiled for CUDA capability >= 1.0
CUDA runtime 5000, driver 5000. OpenMP threads 16. MKL 10.2.4, MKL threads 8. 
device 0: Tesla C1060, 1296.0 MHz clock, 4095.8 MB memory, capability 1.3
device 1: Tesla C1060, 1296.0 MHz clock, 4095.8 MB memory, capability 1.3
Usage: ./testing_dsyevd_gpu [options] [-h|--help]

    N   CPU Time (sec)   GPU Time (sec)
=======================================
  256     ---               0.13
Testing the factorization A = U S U' for correctness:
(1)    | A - U S U' | / (|A| N)     = 5.10e-04  failed
(2)    | I -   U'U  | /  N          = 1.40e-02  failed
(3)    | S(w/ U) - S(w/o U) | / |S| = 2.61e-18

The tests also fail for Ssyevd:

Code: Select all

[bdol@n08 testing]$ ./testing_ssyevd -JV -N 128 -c
MAGMA 1.5.0 beta1 compiled for CUDA capability >= 1.0
CUDA runtime 5000, driver 5000. OpenMP threads 16. MKL 10.2.4, MKL threads 8. 
device 0: Tesla C1060, 1296.0 MHz clock, 4095.8 MB memory, capability 1.3
device 1: Tesla C1060, 1296.0 MHz clock, 4095.8 MB memory, capability 1.3
Usage: ./testing_ssyevd [options] [-h|--help]

    N   CPU Time (sec)   GPU Time (sec)
=======================================
  128     ---               0.15
Testing the factorization A = U S U' for correctness:
(1)    | A - U S U' | / (|A| N)     = 5.29e-09
(2)    | I -   U'U  | /  N          = 3.73e-08
(3)    | S(w/ U) - S(w/o U) | / |S| = 2.77e-09


[bdol@n08 testing]$ ./testing_ssyevd -JV -N 129 -c
MAGMA 1.5.0 beta1 compiled for CUDA capability >= 1.0
CUDA runtime 5000, driver 5000. OpenMP threads 16. MKL 10.2.4, MKL threads 8. 
device 0: Tesla C1060, 1296.0 MHz clock, 4095.8 MB memory, capability 1.3
device 1: Tesla C1060, 1296.0 MHz clock, 4095.8 MB memory, capability 1.3
Usage: ./testing_ssyevd [options] [-h|--help]

    N   CPU Time (sec)   GPU Time (sec)
=======================================
  129     ---               0.09
Testing the factorization A = U S U' for correctness:
(1)    | A - U S U' | / (|A| N)     = 9.96e-04  failed
(2)    | I -   U'U  | /  N          = 2.03e-02  failed
(3)    | S(w/ U) - S(w/o U) | / |S| = 7.29e-09

Is there some option I'm setting incorrectly? I was seeing similar issues with CULA, which leads me to believe that there may be something wrong with the GPUs/CUDA configuration I'm using, but hopefully it's just something I missed with MAGMA. Let me know if any additional information about my configuration will help. Thanks!