LAPACK/ScaLAPACK Development

Hi All,
I have been utilizing a program for automated clustering - that is - identifying clusters in data in order to sort it. The original procedure did not use lapack, however, I have recently added a small piece to the program that requires the use of eigenvalues and so I have linked against blas and lapack. The problem, however, is that simply linking WITHOUT ANY ADDITIONAL CODE has caused the program to slow down considerably, in fact, it's run time went from 5 to 10 minutes, simply by linking against these packages. Is there any reason anyone sees for this?
Ben

Hi Ben,

Could you tell us which LAPACK routine(s) you are calling and what is approximately the size of your problem?

For good performance, you have to use an optimized BLAS and not the reference BLAS provided with Lapack that is there only for testing purpose.

Julie

I'm calling the routines dsyev and dgesvd. I'm sorting about 340,000 neuronal spikes, using their first 3 PCA coefficients to describe them. This makes about 1,020,000 floats that are in the text file that are being sorted using a K-meand clustering algorithm. I'm linking against what I believe to be a optimized version of blas, but perhaps I'm mistaken. Is atlas a better way to go for ubuntu linux? The makefile is as follows if it helps:



# makefile for KlustaKwik
#
#
CC = g++-3.3
FC = g77
LINK = g77
SHELL = /bin/sh

MATLIB = /usr/local/matlab/bin/glnx86
MATINC = /usr/local/matlab/extern/include
HEADERFILES = /mnt/ldaphome/bdg4/ctesting

DEBUG = -g

BLASLIB = /usr/lib/sse2

CFLAGS = -Wall -ansi -pedantic -I$(MATINC) -I $(HEADERFILES)



FFLAGS = -Wall

LDFLAGS = -Wl,-rpath $(MATLIB) -L$(MATLIB) -lmat -lut -lmx -L$(BLASLIB) -llapack -lf77blas -latlas
SOURCES = KlustaKwik.cpp param.c KK.cpp
HEADERS = KlustaKwik.h Array.h param.h KK.h KlustaSave.h
KlustaKwik: $(HEADERS)
$(CC) $(CFLAGS) $(LDFLAGS) $(SOURCES) -lm -o KlustaKwik /mnt/ldaphome/bdg4/ctesting/matrix.o /mnt/ldaphome/bdg4/ctesting/print.o /mnt/ldaphome/bdg4/ctesting/matfile.o

test:
rm -f tempout test.model.1 test.clu.1
KlustaKwik test 1 -MinClusters 2 -Debug 1 > tempout
diff test.clu.1 test_res.clu.1
cvs diff test.model.1

vg: KlustaKwik.h KlustaKwik.C Array.h param.c param.h
g++ -g -O0 -o KlustaKwik KlustaKwik.C param.c -lm
rm -f tempout
valgrind KlustaKwik test 1 -MinClusters 2 -Debug 1 > tempout

clean:
rm KlustaKwik

Ben

Yes according to your makefile you are linking with the ATLAS BLAS. ATLAS BLAS is an optimized BLAS library so that for sure is better than using the reference BLAS implementation that is provided with LAPACK. Now you might want to check how ATLAS has been installed on your machine. In general you (or your sys-admin) need to install ATLAS and then the ATLAS install procedure runs some experiments to tune the implementation for you architecture.

Is ATLAS a better way to go for ubuntu linux?

It's not much a question of Linux distribution you are working with than the machine you are working on. The fact that you are using Ubuntu, Fedora, Gentoo, etc. have very little importance for the performance of the BLAS. What is important is your machine.

Now which BLAS is the best for a given machine is an open question. Say that you can definitely consider GotoBLAS, ATLAS BLAS, your vendor BLAS implementation (Intel=MKL, AMD=ACML, Sun=Sunperf, IBM=ESSL, Apple=VecLib, etc.) or an independent vendor BLAS library. Then either you test all of those BLAS libraries and pick the best one or you simply pick the one you prefer knowing that all of those libraries are doing a very good job and are way better than a reference implementation.

Since you are concerned about speed on the Symmetric Eigenvalue Problem (SEP), maybe you can have a try with other alternatives than dsyev. For example for SEP, you can give a try with dsyevr, dsyevx, and dsyevd. In particular, dsyevx might be a good pick since you just want to compute the three largest eigenvalues and eigenvectors.

Julie

the problem is I haven't even inserted any of these routines. I simply linked these libraries in my make file and added no additioanl code. I didn't call either lapack function, just included the dsyev header file, that's it, no additional code was entered and nothing in the program is using an lapack procedures. The program would still compile without the aid of lapack, just futuristically I plan on using these functions.

Well, that's a problem sure. So your statement is that you have a code taht runs fine. You are adding one header line and then the code runs twice slower.... Sure you did not change anything else? Julien.

No i mean the whole make file is new, it links against the matrix.h and matrix.o files as well as mat.h and mat.o. However, other than that, no difference, just additional headers(3 or 4)