Errors when testing magma

[braunm]

Hi. I recently installed Magma 1.0 RC2 on my Mac Pro running OSX 10.6.5, with a Quadro 4000 GPU (Fermi). The Magma library now compiles, but I cannot get many of the testing routines to pass. The single precision testing_sgemm test passes with

Code: Select all

device 0: Quadro 4000, 950.0 MHz clock, 2047.6 MB memory

Testing TRANSA = N  TRANSB = N
    N     MAGMA GFLop/s    CUBLAS GFlop/s       error
========================================================
 1024       250.00           240.08         0.000000e+00
 2048       285.11           284.37         0.000000e+00
 3072       306.28           303.81         0.000000e+00
 4096       301.63           291.22         0.000000e+00
 5120       300.09           298.82         0.000000e+00
 6144       307.52           304.75         0.000000e+00
 7168       305.28           298.07         0.000000e+00
 8192       303.59           301.83         0.000000e+00
 9216       308.28           305.31         0.000000e+00
10240       306.52           300.44         0.000000e+00

testing_dsymv also passes, but when I run the double precision matrix multiply test testing_dgemm, I get a "can not bind to texture" error for high dimensional matrices.

Code: Select all

device 0: Quadro 4000, 950.0 MHz clock, 2047.6 MB memory

Testing TRANSA = N  TRANSB = N
    N     MAGMA GFLop/s    CUBLAS GFlop/s       error
========================================================
 1024       135.67           138.92         0.000000e+00
 2048       141.77           141.54         0.000000e+00
 3072       142.48           142.40         0.000000e+00
 4096       142.68           142.41         0.000000e+00
 5120       142.81           142.66         0.000000e+00
 6144       142.87           142.85         0.000000e+00
 7168       143.06           143.03         0.000000e+00
 8192       147.19           183251937.96         0.000000e+00
can not bind to texture 
 9216       43486543.87           260919263.23         0.000000e+00
can not bind to texture 
10240       79536431.41           357913941.33         0.000000e+00

And whenever I try a LAPACK-like routine, lie testing_dgetrf, I get messages like

Code: Select all

device 0: Quadro 4000, 950.0 MHz clock, 2047.6 MB memory

Usage: 
  testing_dgetrf_gpu -M 1024 -N 1024

!!!! cudaMallocHost failed for: h_R

Any ideas on what's wrong?

Thanks.

[Stan Tomov]

A problem is that the testers try to allocate memory (at the beginning of the computation) for matrices of dimension up to ~10,000x10,000. In double precision gemm, allocating 3 matrices of that size exceeds 2GB. We must modify the testers to read the memory available and test only problems that would fit in that memory. For LU we allocate only one matrix so in general there should be space to test on matrix of size 10,000. You can check if memory limitation is a problem, e.g., if you try it on smaller problems:
./testing_dgetrf_gpu -M 1024 -N 1024
Does this work? If not, can you please post your make.inc file.

[braunm]

That might be the problem. Sizes through 7168 seem to work. 8192 and above will fail.

Here's my make.inc file, in case it matters. In any event, thanks for your help.

Code: Select all

GPU_TARGET = 1
CC        = g++
NVCC      = nvcc
FORT      = gfortran
ARCH      = ar
ARCHFLAGS = cr
RANLIB    = ranlib
OPTS      = -O3 -m64 -DADD_ 
NVOPTS    = --compiler-options -fno-strict-aliasing -DUNIX -O3 -m64 -DADD_
LDOPTS    = -fPIC -m64  
LIB       = -lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core  -lpthread -lcublas -lcudart -lcuda -lm -lstdc++.6
CUDADIR   = /usr/local/cuda
LIBDIR    = -L$(MKL_LIBS) \
            -L$(CUDADIR)/lib 
INC       = -I$(CUDADIR)/include
LIBMAGMA     = ../lib/libmagma.a
LIBMAGMABLAS = ../lib/libmagmablas.a