make shared problem

[quemener]

Currently I am trying to compile MAGMA on Kubuntu 14.04 LTS for two Tesla C2050 GPU cards with Cuda 6.5 installed in /software/cuda-6.5 and pointed by variable CUDADIR set in .bashrc.
I am using the ATLAS package from the Kubuntu distribution.

In Makefile.internal I changed the following line by removing 'Tesla' and 'Kepler' GPU cards targets:
GPU_TARGET = Fermi

I copied make.inc.atlas into make.inc and modified the following lines in make.inc:

#GPU_TARGET ?= Tesla Fermi Kepler

CFLAGS = -fPIC -O3 -DADD_ -fopenmp -DMAGMA_SETAFFINITY
FFLAGS = -fPIC -O3 -DADD_
F90FLAGS = -fPIC -O3 -DADD_ -x f95-cpp-input
NVCCFLAGS = -O3 -DADD_ -Xcompiler -fno-strict-aliasing
LDFLAGS = -fPIC -fopenmp

LIB = -llapack -lf77blas -lcblas -latlas -lcublas -lcudart -lstdc++ -lm

# define library directories here or in your environment
LAPACKDIR = /usr/lib
ATLASDIR = /usr/lib
#CUDADIR ?= /usr/local/cuda


1) When compiling the static library using 'make -j12', libmagma.a and test programs are succesfully done and run.


2) When trying to compile the dynamic library using 'make -j12 shared' I run into some problems.

Indeed, if I would strictly follow the README, I would add the -fPIC option also to NVCCFLAGS as for the other FLAGS and would get the following error message:

nvcc fatal : Unknown option 'fPIC'

If I do as in make.inc.mkl-shared, i.e. no -fPIC option to NVCCFLAGS, then executing the command 'make -j12 shared' gives:

compile for CUDA arch 2.x (Fermi)
Error: 'make shared' requires CFLAGS, CXXFLAGS, FFLAGS, F90FLAGS, and NVCCFLAGS to have -fPIC.
Please edit your make.inc file. See make.inc.mkl-shared for an example.
After updating make.inc, please 'make clean', then 'make shared', then 'make testing'.


To overcome this last error, I modified the following lines in the Makefile as option -fPIC is not recognized by nvcc and would stop compilation when using 'make -j12 shared':

#fpic = $(and $(findstring -fPIC, $(CFLAGS)), \
# $(findstring -fPIC, $(CXXFLAGS)), \
# $(findstring -fPIC, $(FFLAGS)), \
# $(findstring -fPIC, $(F90FLAGS)), \
# $(findstring -fPIC, $(NVCCFLAGS)))
fpic = $(and $(findstring -fPIC, $(CFLAGS)), \
$(findstring -fPIC, $(CXXFLAGS)), \
$(findstring -fPIC, $(FFLAGS)), \
$(findstring -fPIC, $(F90FLAGS)))

Now 'make -j12 shared' gives the following errors:

compile for CUDA arch 2.x (Fermi)
======================================== magmablas
( cd magmablas && make )
make[1]: Entering directory `/home/quemener/local/magma-1.6.0/magmablas'
make[1]: Nothing to be done for `all'.
make[1]: Leaving directory `/home/quemener/local/magma-1.6.0/magmablas'
======================================== src
( cd src && make )
make[1]: Entering directory `/home/quemener/local/magma-1.6.0/src'
make[1]: Nothing to be done for `all'.
make[1]: Leaving directory `/home/quemener/local/magma-1.6.0/src'
======================================== control
( cd control && make )
make[1]: Entering directory `/home/quemener/local/magma-1.6.0/control'
make[1]: Nothing to be done for `all'.
make[1]: Leaving directory `/home/quemener/local/magma-1.6.0/control'
======================================== interface
( cd interface_cuda && make )
make[1]: Entering directory `/home/quemener/local/magma-1.6.0/interface_cuda'
make[1]: Nothing to be done for `all'.
make[1]: Leaving directory `/home/quemener/local/magma-1.6.0/interface_cuda'
make ./lib/libmagma.so
compile for CUDA arch 2.x (Fermi)
make[1]: Entering directory `/home/quemener/local/magma-1.6.0'
======================================== ./lib/libmagma.so
gcc -fPIC -fopenmp -shared -o ./lib/libmagma.so src/cblas_c.o src/cblas_d.o src/cblas_s.o src/cblas_z.o src/cbulge_applyQ.o src/cbulge_applyQ_v2_m.o src/cbulge_applyQ_v2.o src/cbulge_aux.o src/cbulge_back_m.o src/cbulge_back.o src/cbulge_kernel.o src/cbulge_kernel_v2.o
.......
.......
.......
.......
ztrsm.o magmablas/ztrtri_diag_batched.o magmablas/ztrtri_diag.o magmablas/ztrtri_lower.o magmablas/ztrtri_upper.o \
-L/usr/lib -L/usr/lib/lib -L/software/cuda-6.5/lib64 \
-llapack -lf77blas -lcblas -latlas -lcublas -lcudart -lstdc++ -lm
/usr/bin/ld: magmablas/cbcyclic.o: relocation R_X86_64_32S against `.bss' can not be used when making a shared object; recompile with -fPIC
magmablas/cbcyclic.o: error adding symbols: Bad value
collect2: error: ld returned 1 exit status
make[1]: *** [lib/libmagma.so] Error 1
make[1]: Leaving directory `/home/quemener/local/magma-1.6.0'
make: *** [shared] Error 2

And this is where I need some help as I don't know how to solve this problems.
Thanks a lot for any clue.

[mgates3]

There is no need to modify Makefile.internal. Just set GPU_TARGET in your make.inc file, or even as an environment variable outside make.inc.

As shown in make.inc.mkl-shared, NVCCFLAGS needs to include -fPIC as:
-Xcompiler "-fno-strict-aliasing -fPIC"
This gets passed to the underlying compiler (e.g., g++); nvcc itself doesn't use or understand -fPIC.

-mark