Building MAGMA 1.6.2 and the example

Open discussion for MAGMA library (Matrix Algebra on GPU and Multicore Architectures)
Post Reply
mh1
Posts: 30
Joined: Thu Mar 14, 2013 4:24 pm

Building MAGMA 1.6.2 and the example

Post by mh1 » Wed May 20, 2015 1:53 pm

Hello,

I have downloaded MAGMA 1.6.2.

Platform : Ubuntu 14.02 Mint, GTX 690, Intel 15.0 (which includes MKL).
(NOTE : My Intel MKL libs are here : /opt/intel/mkl/lib/intel64)

I used make.inc.mkl-icc-ilp64 for my make.inc baseline. make.inc is below :

Code: Select all

#//////////////////////////////////////////////////////////////////////////////
#   -- MAGMA (version 1.6.1) --
#      Univ. of Tennessee, Knoxville
#      Univ. of California, Berkeley
#      Univ. of Colorado, Denver
#      @date January 2015
#//////////////////////////////////////////////////////////////////////////////

# GPU_TARGET contains one or more of Tesla, Fermi, or Kepler,
# to specify for which GPUs you want to compile MAGMA:
#     Tesla  - NVIDIA compute capability 1.x cards (no longer supported in CUDA 6.5)
#     Fermi  - NVIDIA compute capability 2.x cards
#     Kepler - NVIDIA compute capability 3.x cards
# The default is "Fermi Kepler".
# See http://developer.nvidia.com/cuda-gpus
#
#GPU_TARGET ?= Fermi Kepler

CC        = icc
CXX       = icpc
NVCC      = nvcc
FORT      = ifort

ARCH      = ar
ARCHFLAGS = cr
RANLIB    = ranlib

# Use -fPIC to make shared (.so) and static (.a) library;
# can be commented out if making only static library.
FPIC      = -fPIC

CFLAGS    = -O3 $(FPIC) -DADD_ -Wall -openmp -DMAGMA_SETAFFINITY -DMAGMA_WITH_MKL
FFLAGS    = -O3 $(FPIC) -DADD_ -warn all -warn nounused
F90FLAGS  = -O3 $(FPIC) -DADD_ -warn all -warn nounused
NVCCFLAGS = -O3         -DADD_       -Xcompiler "-fno-strict-aliasing $(FPIC)"
LDFLAGS   =     $(FPIC)              -openmp

# Defining MAGMA_ILP64 or MKL_ILP64 changes magma_int_t to int64_t in include/magma_types.h
CFLAGS    += -DMKL_ILP64
FFLAGS    += -fdefault-integer-8
F90FLAGS  += -fdefault-integer-8
NVCCFLAGS += -DMKL_ILP64

# Options to do extra checks for non-standard things like variable length arrays;
# it is safe to disable all these
CFLAGS   += -pedantic -Wno-long-long
#CFLAGS   += -Werror  # uncomment to ensure all warnings are dealt with
CXXFLAGS := $(CFLAGS) -std=c++98
CFLAGS   += -std=c99

# IMPORTANT: this link line is for 64-bit int !!!!
# For regular 64-bit builds using 64-bit pointers and 32-bit int,
# use the lp64 library, not the ilp64 library. See make.inc.mkl-gcc or make.inc.mkl-icc.
# see MKL Link Advisor at http://software.intel.com/sites/products/mkl/
# icc with MKL 10.3, Intel threads, 64-bit int
# note -DMAGMA_ILP64 or -DMKL_ILP64, and -fdefault-integer-8 in FFLAGS above
LIB       = -lmkl_intel_ilp64 -lmkl_intel_thread -lmkl_core -lpthread -lcublas -lcudart -lstdc++ -lm

# define library directories preferably in your environment, or here.
# for MKL run, e.g.: source /opt/intel/composerxe/mkl/bin/mklvars.sh intel64
#MKLROOT ?= /opt/intel/composerxe/mkl
#CUDADIR ?= /usr/local/cuda
-include make.check-mkl
-include make.check-cuda

LIBDIR    = -L$(CUDADIR)/lib64 \
            -L$(MKLROOT)/lib/intel64

INC       = -I$(CUDADIR)/include \
            -I$(MKLROOT)/include
After making all the prerequisite changes, I kicked off make and make then compiled the static library libmagma.a. That is the only library Make created. I did receive one compiler error as shown below and my understanding is that is not a critical error. I do not have ifort.

Code: Select all

mark.horn@sid ~/Development/GPU/MAGMA1.6.2/magma-1.6.2/magma-1.6.2 $ make
compile for CUDA arch 2.x (Fermi)
compile for CUDA arch 3.0 (Kepler)
compile for CUDA arch 3.5 (Kepler)
======================================== magmablas
( cd magmablas      && make )
make[1]: Entering directory `/home/mark.horn/Development/GPU/MAGMA1.6.2/magma-1.6.2/magma-1.6.2/magmablas'
make[1]: Nothing to be done for `all'.
make[1]: Leaving directory `/home/mark.horn/Development/GPU/MAGMA1.6.2/magma-1.6.2/magma-1.6.2/magmablas'
======================================== src
( cd src            && make )
make[1]: Entering directory `/home/mark.horn/Development/GPU/MAGMA1.6.2/magma-1.6.2/magma-1.6.2/src'
make[1]: Nothing to be done for `all'.
make[1]: Leaving directory `/home/mark.horn/Development/GPU/MAGMA1.6.2/magma-1.6.2/magma-1.6.2/src'
======================================== control
( cd control        && make )
make[1]: Entering directory `/home/mark.horn/Development/GPU/MAGMA1.6.2/magma-1.6.2/magma-1.6.2/control'
ifort -O3 -fPIC -DADD_ -warn all -warn nounused -fdefault-integer-8 -Dmagma_devptr_t="integer(kind=8)" -I/usr/local/cuda/include -I/opt/intel/composer_xe_2015.0.090/mkl/include -I../include -I../control -I../sparse-iter/include -I../sparse-iter/control  -c magma_param.f90 -o magma_param.o
/bin/sh: 1: ifort: not found
make[1]: *** [magma_param.o] Error 127
make[1]: Leaving directory `/home/mark.horn/Development/GPU/MAGMA1.6.2/magma-1.6.2/magma-1.6.2/control'
make: *** [libmagma] Error 2
I then moved to the "simple" example and tried to compile it. I am lost. I don't understand the instructions it has concerning blas and removing openblas from that example. As stated above, I do have MKL but it isn't clear to me what I am supposed to do in the Makefile. I tried to comment out only the magma startup and initialize logic in example.c just to try and get passed the blas problem. I can't even get that to link as shown by errors below.

Code: Select all

gcc -Wall -o example example.o zfill.o -L../lib -L/usr/local/cuda/lib64 -L/opt/intel/mkl//lib/intel64 -lmagma -lcublas -lcudart
example.o: In function `main':
example.c:(.text+0x10): undefined reference to `magma_init'
example.c:(.text+0x37): undefined reference to `magma_finalize'
zfill.o: In function `magma_zmalloc_cpu':
zfill.c:(.text+0x6b): undefined reference to `magma_malloc_cpu'
zfill.o: In function `zfill_matrix_gpu':
zfill.c:(.text+0x203): undefined reference to `magma_zsetmatrix_internal'
zfill.c:(.text+0x20f): undefined reference to `magma_free_cpu'
collect2: error: ld returned 1 exit status
make: *** [example] Error 1
Can someone please help me get the example working?

mh1
Posts: 30
Joined: Thu Mar 14, 2013 4:24 pm

Re: Building MAGMA 1.6.2 and the example

Post by mh1 » Wed May 20, 2015 3:37 pm

I have managed to get past the compilation by specifying gfortran for the Fortran compiler. By the way, the documentation lead me to believe a Fortran compiler is not required to use Magma 1.6.2 but the following snippet in the Makefile will fail in the "control" build and thus never build the "interface" build. This causes many of the magma signatures (ones belonging to interface) to not be in the final built library. If Fortran is not required I suggest fixing the build scripts to not make it required.

Code: Select all

	@echo ======================================== control
	( cd control        && $(MAKE) )
	@echo ======================================== interface
	( cd interface_cuda && $(MAKE) )
I swapped around the two lines like so :

Code: Select all

	@echo ======================================== interface
	( cd interface_cuda && $(MAKE) )
	@echo ======================================== control
	( cd control        && $(MAKE) )
Then the interface part built clean and failed on control again, but at least the interface part linked its signatures into the library.

I downloaded gfortran and swapped it out for ifort. Another issue in the build file is "-warn all" isn't supported by gfortan. I removed it. After doing all this, and building with gfortran, I now have a clean build and linkage of libmagma.a and libmagma.so.

I then re-built the example and it too now builds clean. When I run it on my system with GTX 690, driver 346.46, CUDA 7.0 I get two crashes. If I choose "cpu_interface" the application just crashed with no errors or warnings. If I choose "gpu_interface" it crashes again with the following dump :

Code: Select all

using MAGMA GPU interface
CUDA runtime error: an illegal memory access was encountered (77) in magma_zgetrf_gpu at zgetrf_gpu.cpp:176
CUBLAS error: memory mapping error (11) in magma_zgetrf_gpu at zgetrf_gpu.cpp:178
CUDA runtime error: an illegal memory access was encountered (77) in magma_zgetrf_gpu at zgetrf_gpu.cpp:194
CUBLAS error: memory mapping error (11) in magma_zgetrf_gpu at zgetrf_gpu.cpp:201
CUDA runtime error: an illegal memory access was encountered (77) in magma_zgetrf_gpu at zgetrf_gpu.cpp:208
CUDA runtime error: an illegal memory access was encountered (77) in magma_zgetrf_gpu at zgetrf_gpu.cpp:176
CUBLAS error: memory mapping error (11) in magma_zgetrf_gpu at zgetrf_gpu.cpp:178
CUDA runtime error: an illegal memory access was encountered (77) in magma_zgetrf_gpu at zgetrf_gpu.cpp:194
CUBLAS error: memory mapping error (11) in magma_zgetrf_gpu at zgetrf_gpu.cpp:201
CUDA runtime error: an illegal memory access was encountered (77) in magma_zgetrf_gpu at zgetrf_gpu.cpp:208
CUDA runtime error: an illegal memory access was encountered (77) in magma_zgetrf_gpu at zgetrf_gpu.cpp:176
CUBLAS error: memory mapping error (11) in magma_zgetrf_gpu at zgetrf_gpu.cpp:178
CUDA runtime error: an illegal memory access was encountered (77) in magma_zgetrf_gpu at zgetrf_gpu.cpp:194
CUBLAS error: memory mapping error (11) in magma_zgetrf_gpu at zgetrf_gpu.cpp:201
CUDA runtime error: an illegal memory access was encountered (77) in magma_zgetrf_gpu at zgetrf_gpu.cpp:208
CUDA runtime error: an illegal memory access was encountered (77) in magma_zgetrf_gpu at zgetrf_gpu.cpp:176
CUBLAS error: memory mapping error (11) in magma_zgetrf_gpu at zgetrf_gpu.cpp:178
CUDA runtime error: an illegal memory access was encountered (77) in magma_zgetrf_gpu at zgetrf_gpu.cpp:194
CUBLAS error: memory mapping error (11) in magma_zgetrf_gpu at zgetrf_gpu.cpp:201
CUDA runtime error: an illegal memory access was encountered (77) in magma_zgetrf_gpu at zgetrf_gpu.cpp:208
CUDA runtime error: an illegal memory access was encountered (77) in magma_zgetrf_gpu at zgetrf_gpu.cpp:176
CUBLAS error: memory mapping error (11) in magma_zgetrf_gpu at zgetrf_gpu.cpp:178
CUDA runtime error: an illegal memory access was encountered (77) in magma_zgetrf_gpu at zgetrf_gpu.cpp:194
CUBLAS error: memory mapping error (11) in magma_zgetrf_gpu at zgetrf_gpu.cpp:201
CUDA runtime error: an illegal memory access was encountered (77) in magma_zgetrf_gpu at zgetrf_gpu.cpp:208
CUDA runtime error: an illegal memory access was encountered (77) in magma_zgetrf_gpu at zgetrf_gpu.cpp:176
CUBLAS error: memory mapping error (11) in magma_zgetrf_gpu at zgetrf_gpu.cpp:178
CUDA runtime error: an illegal memory access was encountered (77) in magma_zgetrf_gpu at zgetrf_gpu.cpp:194
CUBLAS error: memory mapping error (11) in magma_zgetrf_gpu at zgetrf_gpu.cpp:201
CUDA runtime error: an illegal memory access was encountered (77) in magma_zgetrf_gpu at zgetrf_gpu.cpp:208
CUDA runtime error: an illegal memory access was encountered (77) in magma_zgetrf_gpu at zgetrf_gpu.cpp:176
CUBLAS error: memory mapping error (11) in magma_zgetrf_gpu at zgetrf_gpu.cpp:178
CUDA runtime error: an illegal memory access was encountered (77) in magma_zgetrf_gpu at zgetrf_gpu.cpp:194
CUBLAS error: memory mapping error (11) in magma_zgetrf_gpu at zgetrf_gpu.cpp:201
CUDA runtime error: an illegal memory access was encountered (77) in magma_zgetrf_gpu at zgetrf_gpu.cpp:208
CUDA runtime error: an illegal memory access was encountered (77) in magma_zgetrf_gpu at zgetrf_gpu.cpp:176
CUBLAS error: memory mapping error (11) in magma_zgetrf_gpu at zgetrf_gpu.cpp:178
CUDA runtime error: an illegal memory access was encountered (77) in magma_zgetrf_gpu at zgetrf_gpu.cpp:194
CUBLAS error: memory mapping error (11) in magma_zgetrf_gpu at zgetrf_gpu.cpp:201
CUDA runtime error: an illegal memory access was encountered (77) in magma_zgetrf_gpu at zgetrf_gpu.cpp:208
CUDA runtime error: an illegal memory access was encountered (77) in magma_zgetrf_gpu at zgetrf_gpu.cpp:176
CUBLAS error: memory mapping error (11) in magma_zgetrf_gpu at zgetrf_gpu.cpp:178
CUDA runtime error: an illegal memory access was encountered (77) in magma_zgetrf_gpu at zgetrf_gpu.cpp:194
CUBLAS error: memory mapping error (11) in magma_zgetrf_gpu at zgetrf_gpu.cpp:201
CUDA runtime error: an illegal memory access was encountered (77) in magma_zgetrf_gpu at zgetrf_gpu.cpp:208
CUDA runtime error: an illegal memory access was encountered (77) in magma_zgetrf_gpu at zgetrf_gpu.cpp:176
CUBLAS error: memory mapping error (11) in magma_zgetrf_gpu at zgetrf_gpu.cpp:178
CUDA runtime error: an illegal memory access was encountered (77) in magma_zgetrf_gpu at zgetrf_gpu.cpp:194
CUBLAS error: memory mapping error (11) in magma_zgetrf_gpu at zgetrf_gpu.cpp:201
CUDA runtime error: an illegal memory access was encountered (77) in magma_zgetrf_gpu at zgetrf_gpu.cpp:208
CUDA runtime error: an illegal memory access was encountered (77) in magma_zgetrf_gpu at zgetrf_gpu.cpp:176
CUBLAS error: memory mapping error (11) in magma_zgetrf_gpu at zgetrf_gpu.cpp:178
CUDA runtime error: an illegal memory access was encountered (77) in magma_zgetrf_gpu at zgetrf_gpu.cpp:194
CUBLAS error: memory mapping error (11) in magma_zgetrf_gpu at zgetrf_gpu.cpp:201
CUDA runtime error: an illegal memory access was encountered (77) in magma_zgetrf_gpu at zgetrf_gpu.cpp:208
CUDA runtime error: an illegal memory access was encountered (77) in magma_zgetrf_gpu at zgetrf_gpu.cpp:176
CUBLAS error: memory mapping error (11) in magma_zgetrf_gpu at zgetrf_gpu.cpp:178
CUDA runtime error: an illegal memory access was encountered (77) in magma_zgetrf_gpu at zgetrf_gpu.cpp:194
CUBLAS error: memory mapping error (11) in magma_zgetrf_gpu at zgetrf_gpu.cpp:201
CUDA runtime error: an illegal memory access was encountered (77) in magma_zgetrf_gpu at zgetrf_gpu.cpp:208
CUDA runtime error: an illegal memory access was encountered (77) in magma_zgetrf_gpu at zgetrf_gpu.cpp:176
CUBLAS error: memory mapping error (11) in magma_zgetrf_gpu at zgetrf_gpu.cpp:178
CUDA runtime error: an illegal memory access was encountered (77) in magma_zgetrf_gpu at zgetrf_gpu.cpp:194
CUBLAS error: memory mapping error (11) in magma_zgetrf_gpu at zgetrf_gpu.cpp:201
CUDA runtime error: an illegal memory access was encountered (77) in magma_zgetrf_gpu at zgetrf_gpu.cpp:208
CUDA runtime error: an illegal memory access was encountered (77) in magma_zgetrf_gpu at zgetrf_gpu.cpp:176
CUBLAS error: memory mapping error (11) in magma_zgetrf_gpu at zgetrf_gpu.cpp:178
CUDA runtime error: an illegal memory access was encountered (77) in magma_zgetrf_gpu at zgetrf_gpu.cpp:194
CUBLAS error: memory mapping error (11) in magma_zgetrf_gpu at zgetrf_gpu.cpp:201
CUDA runtime error: an illegal memory access was encountered (77) in magma_zgetrf_gpu at zgetrf_gpu.cpp:208
CUBLAS error: memory mapping error (11) in magma_zgetrf_gpu at zgetrf_gpu.cpp:242
CUBLAS error: memory mapping error (11) in magma_zgetrf_gpu at zgetrf_gpu.cpp:255
CUDA runtime error: an illegal memory access was encountered (77) in magma_zgetrf_gpu at zgetrf_gpu.cpp:273
CUDA runtime error: an illegal memory access was encountered (77) in magma_zgetrf_gpu at zgetrf_gpu.cpp:274
CUDA runtime error: an illegal memory access was encountered (77) in magma_zgetrf_gpu at zgetrf_gpu.cpp:276
CUDA runtime error: an illegal memory access was encountered (77) in magma_zgetrf_gpu at zgetrf_gpu.cpp:278
CUBLAS error: memory mapping error (11) in magma_zgetrs_gpu at zgetrs_gpu.cpp:116
Segmentation fault
I then tried run_tests.py. Here is the last few lines of that test ... just mass failure of Magma all the way around.

Code: Select all

  200   300   100    120.99 (   0.10)     153.92 (   0.08)    124.89 (   0.10)        -nan         -nan   failed
  300   100   200    109.18 (   0.11)     144.63 (   0.08)    144.63 (   0.08)        -nan         -nan   failed
  300   200   100    123.66 (   0.10)     155.83 (   0.08)    155.83 (   0.08)        -nan         -nan   failed
  100   300   600    177.43 (   0.20)     346.32 (   0.10)    165.93 (   0.22)        -nan         -nan   failed
  300   100   600    171.59 (   0.21)     382.27 (   0.09)    155.18 (   0.23)        -nan         -nan   failed
  100   600   300    216.64 (   0.17)     487.08 (   0.07)    162.19 (   0.22)        -nan         -nan   failed
  300   600   100    226.38 (   0.16)     199.99 (   0.18)    195.59 (   0.18)        -nan         -nan   failed
  600   100   300    219.47 (   0.16)     409.20 (   0.09)    141.12 (   0.26)        -nan         -nan   failed
  600   300   100    251.66 (   0.14)     197.90 (   0.18)    215.71 (   0.17)        -nan         -nan   failed
 1000  2000  3000    822.76 (  14.59)    1442.13 (   8.32)    265.35 (  45.22)        -nan         -nan   failed
 2000  1000  3000    820.84 (  14.62)    1443.70 (   8.31)    259.89 (  46.17)        -nan         -nan   failed
 1000  3000  2000    817.27 (  14.68)    1445.77 (   8.30)    259.46 (  46.25)        -nan         -nan   failed
 2000  3000  1000    858.05 (  13.99)    1404.81 (   8.54)    257.44 (  46.61)        -nan         -nan   failed
 3000  1000  2000    748.50 (  16.03)    1316.38 (   9.12)    271.77 (  44.16)        -nan         -nan   failed
 3000  2000  1000    786.32 (  15.26)    1278.36 (   9.39)    274.61 (  43.70)        -nan         -nan   failed
  ** 141 tests failed
[enter to continue; M to make and re-run] 
Please help.

mh1
Posts: 30
Joined: Thu Mar 14, 2013 4:24 pm

Re: Building MAGMA 1.6.2 and the example

Post by mh1 » Thu May 21, 2015 12:01 pm

I have managed to solve the problem with a clean build after getting gfortran working. Sorry for inconvenience.

mgates3
Posts: 918
Joined: Fri Jan 06, 2012 2:13 pm

Re: Building MAGMA 1.6.2 and the example

Post by mgates3 » Thu May 21, 2015 8:17 pm

To compile MAGMA without Fortran, remove or comment out FORT in the make.inc file. Sorry this was not made clear. I've added a note to the README for future releases.

-mark

Post Reply