Hello,
i am trying to run testing_dgetrf and testing_dgetrf_mgpu.
when i run testing_dgetrf_mgpu with 4 GPU k40 using the command (--ngpu 4), i get about 1300 Gflops in the GPU section.
And when i run testing_dgetrf with 4 GPU k40 , i get about 2400 Gflops.
Why does these two tests give different results?
Is the number of Gflops in the ouput of testing_dgetrf_mgpu related to only one GPU or for the 4 GPU ?
Thank you in advance
Multi Gpu interface
Multi Gpu interface
Last edited by ritadia on Sun Jul 19, 2015 8:41 am, edited 5 times in total.
Re: Multi Gpu interface
The Gflop/s is simply 2/3 n^3 / time. The formula does not change based on the number of GPUs used.
Please copy & paste your input and output so we can understand the problem you are seeing. Also any additional details about your system such as CPU model, number of CPU cores, etc. E.g., on 2 socket x 8 core Sandy Bridge Intel Xeon E5-2670:
Please copy & paste your input and output so we can understand the problem you are seeing. Also any additional details about your system such as CPU model, number of CPU cores, etc. E.g., on 2 socket x 8 core Sandy Bridge Intel Xeon E5-2670:
Code: Select all
magma-trunk/testing> ./testing_dgetrf --ngpu 3 -N 40000
% MAGMA 1.6.2 svn compiled for CUDA capability >= 3.5, 32-bit magma_int_t, 64-bit pointer.
% CUDA runtime 7000, driver 7000. OpenMP threads 16. MKL 11.2.3, MKL threads 16.
% device 0: Tesla K40c, 745.0 MHz clock, 11519.6 MB memory, capability 3.5
% device 1: Tesla K40c, 745.0 MHz clock, 11519.6 MB memory, capability 3.5
% device 2: Tesla K40c, 745.0 MHz clock, 11519.6 MB memory, capability 3.5
% ngpu 3
% M N CPU GFlop/s (sec) GPU GFlop/s (sec) |PA-LU|/(N*|A|)
%========================================================================
40000 40000 --- ( --- ) 2675.90 ( 15.94) ---
Re: Multi Gpu interface
Here are some more results for comparison. Note that numactl improves the results a bit.
Code: Select all
bunsen magma-trunk/testing> echo $OMP_NUM_THREADS
16
bunsen magma-trunk/testing> ./testing_dgetrf -N 20000 -N 30000 -N 40000 --ngpu 3
% MAGMA 1.6.2 svn compiled for CUDA capability >= 3.5, 32-bit magma_int_t, 64-bit pointer.
% CUDA runtime 7000, driver 7000. OpenMP threads 16. MKL 11.2.3, MKL threads 16.
% device 0: Tesla K40c, 745.0 MHz clock, 11519.6 MB memory, capability 3.5
% device 1: Tesla K40c, 745.0 MHz clock, 11519.6 MB memory, capability 3.5
% device 2: Tesla K40c, 745.0 MHz clock, 11519.6 MB memory, capability 3.5
% Mon Jul 20 08:08:59 2015
% Usage: ./testing_dgetrf [options] [-h|--help]
% ngpu 3
% M N CPU GFlop/s (sec) GPU GFlop/s (sec) |PA-LU|/(N*|A|)
%========================================================================
20000 20000 --- ( --- ) 1400.29 ( 3.81) ---
30000 30000 --- ( --- ) 2638.76 ( 6.82) ---
40000 40000 --- ( --- ) 2960.97 ( 14.41) ---
bunsen magma-trunk/testing> ./testing_dgetrf_mgpu -N 20000 -N 30000 -N 40000 --ngpu 3
[... cut ...]
% M N CPU GFlop/s (sec) GPU GFlop/s (sec) |PA-LU|/(N*|A|)
%========================================================================
20000 20000 --- ( --- ) 1783.55 ( 2.99) ---
30000 30000 --- ( --- ) 2548.81 ( 7.06) ---
40000 40000 --- ( --- ) 2898.20 ( 14.72) ---
bunsen magma-trunk/testing> numactl --interleave=all ./testing_dgetrf -N 20000 -N 30000 -N 40000 --ngpu 3
[... cut ...]
% M N CPU GFlop/s (sec) GPU GFlop/s (sec) |PA-LU|/(N*|A|)
%========================================================================
20000 20000 --- ( --- ) 1449.97 ( 3.68) ---
30000 30000 --- ( --- ) 2586.57 ( 6.96) ---
40000 40000 --- ( --- ) 2909.64 ( 14.66) ---
bunsen magma-trunk/testing> numactl --interleave=all ./testing_dgetrf_mgpu -N 20000 -N 30000 -N 40000 --ngpu 3
[... cut ...]
% M N CPU GFlop/s (sec) GPU GFlop/s (sec) |PA-LU|/(N*|A|)
%========================================================================
20000 20000 --- ( --- ) 1981.37 ( 2.69) ---
30000 30000 --- ( --- ) 2766.23 ( 6.51) ---
40000 40000 --- ( --- ) 2995.65 ( 14.24) ---
Re: Multi Gpu interface
would you please explain to me what is the difference between testing_dgetrf and testing_dgetrf_mgpu ? I thought they were supposed to act the same once we run them on multiple gpu
Re: Multi Gpu interface
magma_dgetrf is a CPU interface. The matrix A is initially in CPU memory, is transferred to GPUs and factored using a hybrid algorithm on both CPU & GPUs, then the result is transferred back to CPU memory. It can handle matrices larger than the combined GPUs' memory using an out-of-GPU-memory algorithm.
magma_dgetrf_mgpu is a multi-GPU interface. The matrix A is distributed in the multiple GPUs' memory by the user before calling it, factored using a hybrid algorithm on both CPU & GPUs, then the result is left in the GPUs' memory. Obviously the matrix must fit into the combined GPUs' memory. It is usually a little faster than magma_dgetrf because it avoids allocating and transferring data.
-mark
magma_dgetrf_mgpu is a multi-GPU interface. The matrix A is distributed in the multiple GPUs' memory by the user before calling it, factored using a hybrid algorithm on both CPU & GPUs, then the result is left in the GPUs' memory. Obviously the matrix must fit into the combined GPUs' memory. It is usually a little faster than magma_dgetrf because it avoids allocating and transferring data.
-mark
Re: Multi Gpu interface
For magma_dgetrf_mgpu, you said it's faster because it avoids transferring data , but how does it do the factorization on CPU if there's no transfer of data? (since the matrix is initially distributed in the multiple GPU's memory)
And just to make sure: You said that the GPU Gflops shown in the output is "2/3 n^3 /time", Then "time" is the time took to do the factorization with the hybrid algorithm on both CPU and GPU ? If so, then what does the CPU Gflops refer to? how do you compute the CPU Gflops since you have an hybrid algorithm and since the algorithm is not independent between GPU and CPU ?
And just to make sure: You said that the GPU Gflops shown in the output is "2/3 n^3 /time", Then "time" is the time took to do the factorization with the hybrid algorithm on both CPU and GPU ? If so, then what does the CPU Gflops refer to? how do you compute the CPU Gflops since you have an hybrid algorithm and since the algorithm is not independent between GPU and CPU ?
Re: Multi Gpu interface
In magma_dgetrf, initially the entire matrix is in the CPU memory. So the first thing it does is allocate memory on the GPU and transfer the entire matrix to the GPU. At the end, it transfers the entire result back to the CPU. In magma_dgetrf_gpu/mgpu, those two transfers are not needed.
Of course there are transfers during the factorization for both algorithms, since one block is factorized on the CPU at each step.
Please look at the testing code (testing_dgetrf.cpp) to see exactly what it does. The time is the wall time around the dgetrf time, either the call to LAPACK ("CPU time" and "CPU Gflop/s"), or the call to MAGMA ("GPU time" and "GPU Gflop/s"). So the "GPU Gflop/s" is the performance for the hybrid implementation, while "CPU Gflop/s" is the performance for the LAPACK CPU-only implementation.
Of course there are transfers during the factorization for both algorithms, since one block is factorized on the CPU at each step.
Please look at the testing code (testing_dgetrf.cpp) to see exactly what it does. The time is the wall time around the dgetrf time, either the call to LAPACK ("CPU time" and "CPU Gflop/s"), or the call to MAGMA ("GPU time" and "GPU Gflop/s"). So the "GPU Gflop/s" is the performance for the hybrid implementation, while "CPU Gflop/s" is the performance for the LAPACK CPU-only implementation.
Code: Select all
cpu_time = magma_wtime();
lapackf77_dgetrf(&M, &N, h_A, &lda, ipiv, &info);
cpu_time = magma_wtime() - cpu_time;
gpu_time = magma_wtime();
magma_dgetrf( M, N, h_A, lda, ipiv, &info);
gpu_time = magma_wtime() - gpu_time;