Partial initialisation

[sevin]

Hi,

I create a version of magma_init to initialise only few GPU :

Code: Select all

magma_int_t magma_init( int nb_devices=0, magma_device_t *devices=NULL)

Why ? Good question... In fact, in our simulation code, we didn't gain too much using more than 4 GPU and we have an access to a machine with 8 K80 (so 16 GPU).
So instead using the 16 GPU for one simulation, we wanted to launch 4 simulations on 4 GPU.

If you call magma_init( ) without arguments, it will generate the correct array calling magma_getdevices method (initialisation to [0,1,2,...,nb_devices-1])
If you call it with correct arguments, if will use the devices argument.

I take care to change magma_setdevice and magma_getdevice method.

It saves resources on used GPU (~100MB) and makes the nvidia-smi more readable

Any comments are welcome.

Arnaud

[mgates3]

Thanks for the contribution.

Updating to magma_v2.h may also alleviate this problem. MAGMA v2 doesn't need to initialize all the GPUs the same way that the MAGMA v1 interface does.

-mark

[sevin]

Do you think it's possible to integrate it in a future release of magma or I need to keep it ?

[mgates3]

We would prefer to not change the magma_init() syntax, as that breaks everyone's code.

However, we did rearrange the code a bit so that, if using MAGMA v1 compatibility (magma.h), it allocates an internal queue only on the first use of a NULL stream. When using only the MAGMA v2 interface (magma_v2.h), a NULL stream should never occur. The effect of this is it doesn't start a context on every GPU, but only on the ones it uses. This avoids some overhead and nvidia-smi doesn't display extraneous processes.

Compare old (2.0):

Code: Select all

+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID  Type  Process name                               Usage      |
|=============================================================================|
|    0    113285    C   ./testing_dsyrk                                668MiB |
|    1    113285    C   ./testing_dsyrk                                109MiB |
+-----------------------------------------------------------------------------+

with new (upcoming 2.1):

Code: Select all

+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID  Type  Process name                               Usage      |
|=============================================================================|
|    0    113332    C   ./testing_dsyrk                                668MiB |
+-----------------------------------------------------------------------------+

-mark

[mgates3]

MAGMA 2.1 was just released. I think it will address your concern about MAGMA v1 compatibility. It initializes a default queue on each GPU when the default queue is first used, rather than during magma_init().

Of course, we highly recommend updating to MAGMA v2 interfaces, by including magma_v2.h instead of magma.h. The major difference is you have to pass a queue into various BLAS and set/get functions. Higher level routines like LU and Cholesky haven't changed their interface.

-mark

[sevin]

Thanks, I will take a look soon

Arnaud