MAGMA Forum

Posted: **Mon Jul 29, 2013 11:56 am**

I have a problem with test code sometimes running the test code testing_sgetri_gpu with increasingly larger matrixes in -c (check) mode.
When I get to a sufficiently large matrix, the test program fails with "!!!! magma_malloc failed for: d_A". That makes perfect sense as I don't have enough memory on the GPU for the matrix. But sometimes the performance of the test code is about half its normal performance, as shown below for one case. I am wondering if the crashed test program sometimes leaves the GPU in a partially cleaned up state, so it doesn't have the compute resources for later tests. The slowness problem goes away only after I reboot my machine.

Example: 285.30 GFlops/s version 123.98:

Code: Select all

   N   CPU GFlop/s (sec)   GPU GFlop/s (sec)   ||R||_F / ||A||_F
 5000    135.72 (   1.23)    285.30 (   0.58)       -inf

versus:

Code: Select all

   N   CPU GFlop/s (sec)   GPU GFlop/s (sec)   ||R||_F / ||A||_F
 5000    136.23 (   1.22)    123.98 (   1.34)       -inf

Is this a known problem?
Is there a way to clean up the GPU's state short of a reboot?

I am running on an iMac 27" 2012 (OS X 10.8.4) with its NVIDIA GeForce GTX 675MX and an Intel Core i5 at 3.2 GHz with 24GB of RAM.
I am using the CUDA 5.5 release candidate and the magma-1.4.0-beta2.

thanks.

Posted: **Fri Aug 09, 2013 1:09 pm**

Sometimes cards will decrease their performance if they overheat. I haven't personally observed this, though. Does waiting a few minutes help?

Are you using the provided make.inc.macos, or did you customize it in some way? If so, can you provide your make.inc?

Also, can you provide the COMPLETE input & output of the tester, including how you ran it? That provides some valuable information about your environment. E.g.,

Code: Select all

magma-trunk/testing> ./testing_sgetri_gpu -N 5000 -c
MAGMA 1.4.0 svn
device 0: GeForce GT 330M, 1100.0 MHz clock, 511.7 MB memory, capability 1.2
Usage: ./testing_sgetri_gpu [options] [-h|--help]

    N   CPU GFlop/s (sec)   GPU GFlop/s (sec)   ||R||_F / ||A||_F
=================================================================
 5000     29.67 (   5.62)     33.43 (   4.99)   2.18e-05

Thanks.
-mark

MAGMA Forum

Slow tests sometimes

Slow tests sometimes

Re: Slow tests sometimes