Guidance on GPU memory assumptions

[fletchjp]

This is a question about the assumptions which can be made about the lifetime of memory on a GPU set by calls to cublas_set_matrix.

I have been converting some Fortran code from CPU (Lapack) to GPU operation (MAGMA). In the code I call magmaf_zgetrs_gpu twice using the same factorization to solve two problems with different input right hand sides.

Code: Select all

! fpm23 is the nrhs
        call cublas_set_matrix(n, fpm23, size_of_elt, Zwork2, N, devptrB, ldda)
        call cublas_set_matrix(n, n, size_of_elt, Z, size, devptrY, ldda)
        call magmaf_zgetrs_gpu('N', n, 1, devptrY, ldda, ipiv, devptrB, ldda, info_gpu)
        call cublas_get_matrix(n, fpm23, size_of_elt, devptrB, ldda, Zwork2, N)
!
!    some cpu only code which has changed Zwork2 but not Z and ipiv.
!
        call cublas_set_matrix(n, fpm23, size_of_elt, Zwork2, N, devptrB, ldda)
        call cublas_set_matrix(n, n, size_of_elt, Z, size, devptrY, ldda)
        call magmaf_zgetrs_gpu('C', n, 1, devptrY, ldda, ipiv, devptrB, ldda, info_gpu)
        call cublas_get_matrix(n, fpm23, size_of_elt, devptrB, ldda, Zwork2, N)

My experience has been that although Z has not changed, and I have not taken any action to change the contents pointed to by devptrY, I do need to transfer the matrix Z again.

That leads me to ask what assumptions can be made about the memory used by a call to cublas_set_matrix? How long is it valid?

I am working with a GPU which is also used to run a display, which I guess may also make use of the memory I am using for CUDA. Magma 1.6.2

Thanks

John

[mgates3]

The lifetime should be until you explicitly free it with cudaFree or your application exits. So I don't know what would be going wrong, unless devptrB is not large enough (ldda x fpm23), so cublas_set_matrix( ..., devptrB, ldda ) obliterates devptrY.

You can examine the contents of your matrices using magmaf_zprint (on CPU) and magmaf_zprint_gpu (on GPU), assuming you can run a small example that exhibits the problem.

NB: you are putting fpm23 number of rhs, but solving only 1 in zgetrs.

-mark

[fletchjp]

Mark

Thank you, thank you for the NB. That is indeed a bug in my code and I am astonished it was working before.

I will do some more exploration as to the other aspect of my problem. I cannot easily print out the matrix data as it is about 7000 square!

This is part of a calculation to locate a small number of generalised eigenvalues.

Thanks again

John