Matlab/nvmex - failure to compile

[Boxed Cylon]

Thanks for the new 0.2 MAGMA!

I am attempting to compile a test matlab mex file that calls magma_sgetrf_gpu and magma_sgetrs_gpu. Alas, the compile fails with:

Code: Select all

#$ gcc -D__CUDA_ARCH__=130 -E -x c++ -DCUDA_DOUBLE_MATH_FUNCTIONS  "-I/usr/local/cuda/bin/../include" "-I/usr/local/cuda/bin/../include/cudart"   -I. -fPIC -D_GNU_SOURCE -pthread -fexceptions -m64 -O3 -funroll-loops -msse2 -DNDEBUG -O3 -I"/usr/local/cuda/include" -I"/usr/local/magma_0.2/include" -I"/usr/local/matlab/extern/include" -m64 -o "/tmp/tmpxft_0000449c_00000000-10_gpu_sgesv_magma.ii" "/tmp/tmpxft_0000449c_00000000-1_gpu_sgesv_magma.cudafe1.cpp"
#$ gcc -c -x c++ "-I/usr/local/cuda/bin/../include" "-I/usr/local/cuda/bin/../include/cudart"   -I. -fPIC -D_GNU_SOURCE -pthread -fexceptions -m64 -O3 -funroll-loops -msse2 -DNDEBUG -O3 -I"/usr/local/cuda/include" -I"/usr/local/magma_0.2/include" -I"/usr/local/matlab/extern/include" -m64 -o "gpu_sgesv_magma.o" "/tmp/tmpxft_0000449c_00000000-10_gpu_sgesv_magma.ii"
/usr/lib64/gcc/x86_64-suse-linux/4.3/../../../../x86_64-suse-linux/bin/ld: /usr/local/magma_0.2/lib/libmagma.a(sgetrf_gpu.o): relocation R_X86_64_32 against `__gxx_personality_v0' can not be used when making a shared object; recompile with -fPIC
/usr/local/magma_0.2/lib/libmagma.a: could not read symbols: Bad value
collect2: ld returned 1 exit status

    mex: link of 'gpu_sgesv_magma.mexa64' failed.

I've put -fPIC's in the Makefile+nvopts.sh wherever I thought it might do some good, tried to put this flag first in the list, etc. all to no avail. I am at the moment at a standstill...any ideas?

I'm using OpenSUSE 11.1

Code: Select all

gcc -v
Using built-in specs.
Target: x86_64-suse-linux
Configured with: ../configure --prefix=/usr --infodir=/usr/share/info --mandir=/usr/share/man --libdir=/usr/lib64 --libexecdir=/usr/lib64 --enable-languages=c,c++,objc,fortran,obj-c++,java,ada --enable-checking=release --with-gxx-include-dir=/usr/include/c++/4.3 --enable-ssp --disable-libssp --with-bugurl=http://bugs.opensuse.org/ --with-pkgversion='SUSE Linux' --disable-libgcj --disable-libmudflap --with-slibdir=/lib64 --with-system-zlib --enable-__cxa_atexit --enable-libstdcxx-allocator=new --disable-libstdcxx-pch --enable-version-specific-runtime-libs --program-suffix=-4.3 --enable-linux-futex --without-system-libunwind --with-cpu=generic --build=x86_64-suse-linux
Thread model: posix
gcc version 4.3.2 [gcc-4_3-branch revision 141291] (SUSE Linux)

Compiling MAGMA's testing seems to work o.k.

[Stan Tomov]

This is interesting. I'll ask our compiler people for a clue. MAGMA 0.2 is compiled with gcc version 4.1.2. You compile the testing suite though with your compiler and link without problem, so it shouldn't be e problem of intermixing the versions. I tried to reproduce your problem and if I try to do the linking with something like

Code: Select all

gcc  -shared  ...

I get what you get, but when I put

Code: Select all

gcc  -fPIC  ...

the linking is successful.

[Boxed Cylon]

A few clues maybe:

I have CUDA 2.3 installed.

I installed gcc 4.1.3 and that made no difference.

The issue may be the line in the nvopts.sh file in the LD section, a file used by nvmex:

Code: Select all

 LDFLAGS="-pthread -shared -Wl,--version-script,$TMW_ROOT/extern/lib/$Arch/$MAPFILE"   

(the -shared in particular).

Since it insisted on a shared library, I tried:

Code: Select all

ar -x libmagma_64.a
gcc -shared *.o -o libmagma_64.so.2
/usr/lib64/gcc/x86_64-suse-linux/4.3/../../../../x86_64-suse-linux/bin/ld: auxiliary.o: relocation R_X86_64_32 against `a local symbol' can not be used when making a shared object; recompile with -fPIC
auxiliary.o: could not read symbols: Bad value
collect2: ld returned 1 exit status

which may or may not be telling...

[Stan Tomov]

If you want to try, here are the shared versions of the libraries.

libmagmablas.so.gz

(716.98 KiB) Downloaded 973 times

libmagma.so.gz

(55.86 KiB) Downloaded 971 times

[Boxed Cylon]

I had to make a shared library of get_nb.cpp (libgetnb.so) and link it in, but the shared libraries compile to a mex file fine. Thanks for those.

The problem now is that matlab is crashing at the call to magma_sgetrf_gpu... Matlab mex routines are very tempermental/strict about memory. I suspect its the MKL libraries that have been compiled in. I may need to find a library that makes matlab happy. Just guessing.

[mfatica]

Try this other method.

From inside Matlab:

!nvcc -c myfun.cu -Xcompiler -fPIC -I /usr/local/matlab/extern/include

mex myfun.o -L /usr/local/cuda/lib64 -lcudart -lcufft

[jpeinado]

Hi Again:

I think, I have the same problem here...

./nvmex -f nvopts.sh sgeidrbdf_cu.cu -I/usr/local/cuda/include -I/home/jpeinado/MAGMA/magma_0.2//include -L/usr/local/cuda/lib64 -L/home/jpeinado/MAGMA/magma_0.2//lib -lcufft -lcudart -lcublas -lmagma -lmagmablas -Wl,-rpath,/usr/local/cuda/lib -lmkl_em64t -lguide -lpthread -lm
/usr/bin/ld: /home/jpeinado/MAGMA/magma_0.2//lib/libmagma.a(sgetrf_gpu.o): relocation R_X86_64_32 against `__gxx_personality_v0' can not be used when making a shared object; recompile with -fPIC
/home/jpeinado/MAGMA/magma_0.2//lib/libmagma.a: could not read symbols: Bad value
collect2: ld returned 1 exit status


Did this solved yet?.

When compiling I based my work in a script done by Boxed Cyclon (¿?), in his excellent tutorial about CUDA/CUBLAS and MATLAB.

Thanks

jpeinado

[jpeinado]

Hi again:

I followed all the "Boxed Cyclon" advices. Now I had to learn to make a shared library of get_nb.cpp (libgetnb.so) and link it in, but the shared libraries compile to a mex file fine.

Thanks for your advices, Boxed Cyclon

jpeinado

[katayama]

Hi,

I am trying to link the above libmagma.so and libmagmablas.so but they have undefined symbols like
...
U magma_get_dgeqlf_nb
U magma_get_dgeqrf_nb
U magma_get_dgetrf_nb
U magma_get_dpotrf_nb
U magma_get_sgehrd_nb
U magma_get_sgelqf_nb
U magma_get_sgeqlf_nb
U magma_get_sgeqrf_nb
U magma_get_sgetrf_nb
U magma_get_spotrf_nb
U magma_get_zgeqrf_nb
U magma_get_zgetrf_nb
U magma_get_zpotrf_nb

Where can I get these symbols?
Thanks

[Stan Tomov]

Hi,
These are defined in testing/get_nb.cpp
Regards,
Stan