The LAPACK forum has moved to https://github.com/Reference-LAPACK/lapack/discussions.

performance problem of LAPACK timing with threaded GotoBLAS

Open discussion regarding features, bugs, issues, vendors, etc.

performance problem of LAPACK timing with threaded GotoBLAS

Postby ManOnTec » Tue Aug 28, 2007 4:21 am

I've recently installed LAPACK 3.0 and GotoBLAS 1.15 on our blade server, and tried to make the TIMING test on the Cholesky factorization routine(DPOTRF). Since the GotoBLAS claimed to support multi-threads, so I build the threaded GotoBLAS and use it instead of the reference BLAS contained in LAPACK package. but unfortunately, I got no performance gain in the DPOTRF timing test, even if I set the OMP_NUM_THREADS to 2 or 4. It seems that no GotoBLAS thread is spawned. Can anybody figured out what's the problem or what I did wrong, following is my testing details:


1. hardware platform
IBM BladeCenter LS21, with 2 "Dual-Core AMD Opteron(tm) Processor 2216 HE"


2. OS
Redhat Linux Enterprise, kernel version is 2.6.9-22


3. GotoBLAS options defined in Makefile.rule (only the modified part)
Code: Select all
...
# Which C compiler do you prefer? Default is gcc.
# I recommend you to use GCC because inline assembler is required.
# C_COMPILER = GNU
# C_COMPILER = INTEL

# Which Fortran compiler do you prefer? Default is GNU g77.
# F_COMPILER = G77
# F_COMPILER = G95
# F_COMPILER = GFORTRAN
# F_COMPILER = INTEL
# F_COMPILER = PGI
# F_COMPILER = PATHSCALE
# F_COMPILER = IBM
# F_COMPILER = COMPAQ
# F_COMPILER = SUN
# F_COMPILER = F2C

# If you need 64bit binary; some architecture can accept both 32bit and
# 64bit binary(EM64T, Opteron, SPARC and Power/PowerPC).
BINARY64  = 1

# If you want to build threaded version.
# You can specify number of threads by environment value
# "OMP_NUM_THREADS", otherwise, it's automatically detected.
SMP = 1

# You may specify Maximum number of threads. It should be minimum.
# For OpenMP users; you have to specify MAX_THREADS even if you don't
# use threaded BLAS(MAX_THREADS >= OMP_NUM_THREADS * GOTO_NUM_THREADS)
MAX_THREADS = 4
...


4. LAPACK options in make.inc (only the modified part)
Code: Select all
...
FORTRAN  = g77
OPTS     = -funroll-all-loops -O3
DRVOPTS  = $(OPTS)
NOOPT    =
LOADER   = g77
LOADOPTS = -funroll-all-loops -O3

#
#  The archiver and the flag(s) to use when building archive (library)
#  If you system has no ranlib, set RANLIB = echo.
#
ARCH     = ar
ARCHFLAGS= cr
RANLIB   = ranlib
#
#  The location of the libraries to which you will link.  (The
#  machine-specific, optimized BLAS library should be used whenever
#  possible.)
#
BLASLIB      = /home/jascha/01.lapack/lapack-3.0.complete-smp/libgoto_opteronp-r1.15.a -lpthread
...



5. test input DTIME.in
Code: Select all
LAPACK timing, DOUBLE PRECISION square matrices
6                                Number of values of M
50 100 200 300 400 500           Values of M (row dimension)
4                                Number of values of N
100 500 1000 2000 250            Values of N (column dimension)
4                                Number of values of K
1 2 16 100                       Values of K
5                                Number of values of NB
8 64 96 128 256 1 8 16            Values of NB (blocksize)
0 48 128 128 128                 Values of NX (crossover point)
1                                Number of values of LDA
2001                              Values of LDA (leading dimension)
0.0                              Minimum time in seconds
#DGE    T T T
#DGT    T T T
DPO    T F F
#DPP    T T T
#DPT    T T T
#DSY    T T T
#DSP    T T T
#DTR    T T
#DTP    T T
#DQR    T T T
#DLQ    T T T
#DQL    T T T
#DRQ    T T T
#DQP    T
#DHR    T T T T
#DTD    T T T T
#DBR    T T T
#DLS    T T T T T T


6. test result
6.1 OMP_NUM_THREADS=1
Code: Select all
*** Speed of DGEMM  in megaflops ***
     with LDA =  2001

         N   100     500    1000    2000
       
             0.0  4386.8  4367.5  4423.0

 *** Speed of DPOTRF in megaflops ***

     DPOTRF with UPLO = 'U'

         N   100     500    1000    2000
     NB
      8      0.0  2322.3  2351.3  2318.9
     64      0.0  3800.0  3669.1  3785.9
     96      0.0  3799.4  3751.5  3936.7
    128    337.9  3483.5  3751.5  4001.6
    256      0.0  2985.5  3669.1  4087.4

6.2 OMP_NUM_THREADS=2
Code: Select all
 *** Speed of DGEMM  in megaflops ***
     with LDA =  2001

         N   100     500    1000    2000
       
             0.0  4238.0  4301.7  4391.4

 *** Speed of DPOTRF in megaflops ***

     DPOTRF with UPLO = 'U'

         N   100     500    1000    2000
     NB
      8      0.0  1990.5  2140.3  2067.4
     64      0.0  3215.1  3629.2  3626.5
     96    338.5  2786.6  3590.2  3791.3
    128      0.0  3215.6  3590.1  3818.4
    256      0.0  2786.6  3478.0  3873.8

6.3 OMP_NUM_THREADS=4
Code: Select all
 *** Speed of DGEMM  in megaflops ***
     with LDA =  2001

         N   100     500    1000    2000
       
          2001.1  4238.0  4311.0  4326.2

 *** Speed of DPOTRF in megaflops ***

     DPOTRF with UPLO = 'U'

         N   100     500    1000    2000
     NB
      8    338.5   746.4   558.3   494.3
     64    337.9  2089.9  2419.4  2442.0
     96      0.0  3215.1  2954.7  2815.5
    128      0.0  3215.1  3091.5  3043.4
    256      0.0  2786.6  2954.8  3370.0



Thanks for your consideration!

Xiangqian Wang
ManOnTec
 
Posts: 1
Joined: Mon Aug 27, 2007 10:10 pm
Location: Beijing, RPC

Return to User Discussion

Who is online

Users browsing this forum: No registered users and 4 guests