Page 1 of 1
example_sparse.c failing to compile
Posted: Fri Nov 02, 2018 10:25 am
by dariofigueira
I'm in Windows, VS2017. Project solution generated by CMake with parameters:
- GPU_TARGET=Pascal
LAPACK_LIBRARIES=mkl_intel_lp64.lib;mkl_intel_thread.lib;mkl_core.lib;libiomp5md.lib (copied from IntelSWTools/parallel_studio_xe_2019.0.045/ compilers_and_libraries_2019/windows/mkl/lib/intel64_win)
MKLROOT=D:/Program Files (x86)/IntelSWTools/parallel_studio_xe_2019.0.045/compilers_and_libraries_2019/windows/mkl
USE_FORTRAN=Yes
Compiled and then Statically linked (/MT linker flag)
I compiled and ran
testing_dgels in Debug and in Release mode successfully.
Copied example_sparse.c to a new folder, renamed it example_sparse.cpp
Created a simple CMakeLists.txt
Code: Select all
add_executable(magma-test example_sparse.cpp)
find_package( CUDA ) # just to set CUDA_INCLUDE_DIRS
target_include_directories(magma-test PUBLIC D:/Work/Magma/magma-2.4.0/include D:/Work/Magma/magma-2.4.0/sparse/include ${CUDA_INCLUDE_DIRS})
target_link_libraries(magma-test debug D:/Work/Magma/magma-2.4.0/build/lib/Debug/magma.lib debug D:/Work/Magma/magma-2.4.0/build/lib/Debug/magma_sparse.lib)
set(CMAKE_CXX_FLAGS_RELEASE "${CMAKE_CXX_FLAGS_RELEASE} /MT")
set(CMAKE_CXX_FLAGS_DEBUG "${CMAKE_CXX_FLAGS_DEBUG} /MTd")
which upon compilation yields
2>magma_sparse.lib(magma_sparse_generated_djacobisetup.cu.obj) : error LNK2038: mismatch detected for '_ITERATOR_DEBUG_LEVEL': value '0' doesn't match value '2' in magma.lib(interface.obj)
(what?..)
Just grasping at straws I also compiled MAGMA in Release mode, and then tried to compile the example code in Release mode, which yields
1>magma_sparse.lib(magma_dmio.obj) : error LNK2038: mismatch detected for '_ITERATOR_DEBUG_LEVEL': value '2' doesn't match value '0' in magma_sparse.lib(magma_sparse_generated_djacobisetup.cu.obj)
What may be happening here?
What more information do we need to divine this issue?
Shall I recompile MAGMA from scratch?
Re: example_sparse.c failing to compile
Posted: Fri Nov 02, 2018 2:04 pm
by mgates3
This appears to be an issue with compiling with Microsoft Visual Studio. For instance, see this post.
https://stackoverflow.com/questions/766 ... t/12950627
Sorry I can't help any further, as I don't use VS.
BTW, I noticed you listed mkl_intel_ilp64 again, which might give problems, instead of mkl_intel_lp64.
-mark
Re: example_sparse.c failing to compile
Posted: Fri Nov 02, 2018 2:06 pm
by dariofigueira
(copy paste error on the lp64, fixed in the post now)
Let me investigate further then, thanks
EDIT:
- I've "clean" MAGMA project on "Release", which deleted all the *.obj and the .lib from the Release folder
- I've ran again testing_dgels in "Debug" successfully (which does not call magma_sparse.lib)
- But the example_sparse project
1>magma_sparse.lib(magma_sparse_generated_djacobisetup.cu.obj) : error LNK2038: mismatch detected for '_ITERATOR_DEBUG_LEVEL': value '0' doesn't match value '2' in magma.lib(interface.obj)
1>magma_sparse.lib(magma_sparse_generated_djacobisetup.cu.obj) : error LNK2038: mismatch detected for 'RuntimeLibrary': value 'MT_StaticRelease' doesn't match value 'MTd_StaticDebug' in magma.lib(interface.obj)
So, now we're sure he's not getting the magma_sparse.lib from the Release folder.
This suggests there is something wrong with the magma_sparse.lib in the Debug folder.
EDIT2:
- I've searched the hard-drive and the magma.lib and magma_sparse.lib in the Debug folder are the only exemplars in the whole hard-drive
- I've renamed magma.lib and magma_sparse.lib in the Debug folder to make sure they are the ones being used and , indeed it says "1>LINK : fatal error LNK1104: cannot open file 'D:\Work\Magma\magma-2.4.0\build\lib\Debug\magma_sparse.lib'
Re: example_sparse.c failing to compile
Posted: Sat Nov 03, 2018 8:45 am
by dariofigueira
the magma compilation command line parameters are:
/MP /GS /TP /W4 /Zc:wchar_t /I"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.0\include" /I"D:\Program Files (x86)\IntelSWTools\parallel_studio_xe_2019.0.045\compilers_and_libraries_2019\windows\mkl\include" /I"D:\Work\Magma\magma-2.4.0\include" /I"D:\Work\Magma\magma-2.4.0\control" /I"D:\Work\Magma\magma-2.4.0\magmablas" /I"D:\Work\Magma\magma-2.4.0\sparse\include" /I"D:\Work\Magma\magma-2.4.0\sparse\control" /I"D:\Work\Magma\magma-2.4.0\testing" /Zi /Gm- /Od /Ob0 /Fd"magma.dir\Debug\magma.pdb" /Zc:inline /fp:precise /D "WIN32" /D "_WINDOWS" /D "MAGMA_NOAFFINITY" /D "HAVE_CUBLAS" /D "MIN_CUDA_ARCH=600" /D "MAGMA_WITH_MKL" /D "CMAKE_INTDIR=\"Debug\"" /D "_MBCS" /errorReport:prompt /WX- /Zc:forScope /RTC1 /GR /Gd /MTd /openmp /Fa"Debug/" /EHsc /nologo /Fo"magma.dir\Debug\" /Fp"magma.dir\Debug\magma.pch" /diagnostics:classic
the magma_sparse compilation command line parameters are:
/MP /GS /TP /W4 /Zc:wchar_t /I"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.0\include" /I"D:\Program Files (x86)\IntelSWTools\parallel_studio_xe_2019.0.045\compilers_and_libraries_2019\windows\mkl\include" /I"D:\Work\Magma\magma-2.4.0\include" /I"D:\Work\Magma\magma-2.4.0\control" /I"D:\Work\Magma\magma-2.4.0\magmablas" /I"D:\Work\Magma\magma-2.4.0\sparse\include" /I"D:\Work\Magma\magma-2.4.0\sparse\control" /I"D:\Work\Magma\magma-2.4.0\testing" /Zi /Gm- /Od /Ob0 /Fd"magma_sparse.dir\Debug\magma_sparse.pdb" /Zc:inline /fp:precise /D "WIN32" /D "_WINDOWS" /D "MAGMA_NOAFFINITY" /D "HAVE_CUBLAS" /D "MIN_CUDA_ARCH=600" /D "MAGMA_WITH_MKL" /D "CMAKE_INTDIR=\"Debug\"" /D "_MBCS" /errorReport:prompt /WX- /Zc:forScope /RTC1 /GR /Gd /MTd /openmp /Fa"Debug/" /EHsc /nologo /Fo"magma_sparse.dir\Debug\" /Fp"magma_sparse.dir\Debug\magma_sparse.pch" /diagnostics:classic
the magma librarian command line parameters are:
/OUT:"D:\Work\Magma\magma-2.4.0\build\lib\Debug\magma.lib" /MACHINE:X64 /NOLOGO
the magma_sparse librarian command line parameters are:
/OUT:"D:\Work\Magma\magma-2.4.0\build\lib\Debug\magma_sparse.lib" /MACHINE:X64 /NOLOGO
Do you spot anything off?
Re: example_sparse.c failing to compile
Posted: Sat Nov 03, 2018 5:44 pm
by dariofigueira
Ok, I just determined this is reproducible in my machine:
- I just deleted MAGMA folder completely
- and did a complete rebuild, *just* in Debug mode, as described in
https://stackoverflow.com/questions/531 ... 2017-magma
- and the same behavior was observed
I'm now at a point of attempting to reproduce this in a separate computer.
Would any kind soul attempt to reproduce this in your computer? To further this investigation
Re: example_sparse.c failing to compile
Posted: Mon Nov 05, 2018 10:44 am
by dariofigueira
Just attempted to compile in Release and
Code: Select all
1>magma.lib(interface.obj) : error LNK2019: unresolved external symbol MKL_Get_Version referenced in function magma_print_environment
1>magma.lib(interface.obj) : error LNK2019: unresolved external symbol MKL_Get_Max_Threads referenced in function magma_print_environment
1>magma.lib(interface.obj) : error LNK2019: unresolved external symbol cudaGetLastError referenced in function magma_is_devptr
1>magma.lib(interface.obj) : error LNK2019: unresolved external symbol cudaGetDeviceCount referenced in function magma_getdevices
1>magma.lib(interface.obj) : error LNK2019: unresolved external symbol cudaGetDeviceProperties referenced in function magma_init
1>magma.lib(interface.obj) : error LNK2019: unresolved external symbol cudaSetDevice referenced in function magma_mem_size
1>magma.lib(interface.obj) : error LNK2019: unresolved external symbol cudaGetDevice referenced in function magma_getdevice
1>magma.lib(interface.obj) : error LNK2019: unresolved external symbol cudaStreamCreate referenced in function magma_queue_create_internal
1>magma.lib(interface.obj) : error LNK2019: unresolved external symbol cudaStreamDestroy referenced in function magma_finalize
1>magma.lib(interface.obj) : error LNK2019: unresolved external symbol cudaStreamWaitEvent referenced in function magma_queue_wait_event
1>magma.lib(interface.obj) : error LNK2019: unresolved external symbol cudaStreamSynchronize referenced in function magma_queue_sync_internal
1>magma.lib(copy_v2.obj) : error LNK2001: unresolved external symbol cudaStreamSynchronize
1>magma.lib(interface.obj) : error LNK2019: unresolved external symbol cudaEventCreate referenced in function magma_event_create
1>magma.lib(interface.obj) : error LNK2019: unresolved external symbol cudaEventRecord referenced in function magma_event_record
1>magma.lib(interface.obj) : error LNK2019: unresolved external symbol cudaEventSynchronize referenced in function magma_event_sync
1>magma.lib(interface.obj) : error LNK2019: unresolved external symbol cudaEventDestroy referenced in function magma_event_destroy
1>magma.lib(interface.obj) : error LNK2019: unresolved external symbol cudaMemGetInfo referenced in function magma_mem_size
1>magma.lib(interface.obj) : error LNK2019: unresolved external symbol cudaPointerGetAttributes referenced in function magma_is_devptr
1>magma.lib(interface.obj) : error LNK2019: unresolved external symbol cudaDriverGetVersion referenced in function magma_print_environment
1>magma.lib(interface.obj) : error LNK2019: unresolved external symbol cudaRuntimeGetVersion referenced in function magma_print_environment
1>magma.lib(interface.obj) : error LNK2019: unresolved external symbol cublasCreate_v2 referenced in function magma_queue_create_from_cuda_internal
1>magma.lib(interface.obj) : error LNK2019: unresolved external symbol cublasDestroy_v2 referenced in function magma_finalize
1>magma.lib(interface.obj) : error LNK2019: unresolved external symbol cublasSetStream_v2 referenced in function magma_queue_create_from_cuda_internal
1>magma_sparse.lib(magma_dcuspmm.obj) : error LNK2001: unresolved external symbol cusparseCreate
1>magma_sparse.lib(dcustomilu.obj) : error LNK2001: unresolved external symbol cusparseCreate
1>magma_sparse.lib(dcustomic.obj) : error LNK2001: unresolved external symbol cusparseCreate
1>magma_sparse.lib(magma_d_blaswrapper.obj) : error LNK2001: unresolved external symbol cusparseCreate
1>magma_sparse.lib(magma_sparse_generated_dmergecg.cu.obj) : error LNK2001: unresolved external symbol cusparseCreate
...
Mark, does the above even compile in linux?
example_sparse.cpp
CMakeLists.txt
Re: example_sparse.c failing to compile
Posted: Mon Nov 05, 2018 2:46 pm
by mgates3
Yes, it compiles on Unix. (I'm actually using macOS.)
Code: Select all
g++ -Wall -fopenmp \
-I ${MAGMADIR}/include \
-I ${MAGMADIR}/sparse/include \
-o example_sparse example_sparse.cpp \
-L ${MAGMADIR}/lib -lmagma -lmagma_sparse \
-L ${CUDADIR} -lcublas -lcudart -lcusparse \
-L /opt/OpenBLAS/lib -lopenblas
From the missing symbols, it looks like you didn't add the CUDA runtime (cudart), cuBLAS, cuSparse, and MKL libraries.
-mark
Re: example_sparse.c failing to compile
Posted: Mon Nov 05, 2018 3:01 pm
by dariofigueira
The example code compiler arguments line generated is
Code: Select all
/GS
/TP
/W3
/Zc:wchar_t
/I"D:\Work\Magma\magma-2.4.0\include"
/I"D:\Work\Magma\magma-2.4.0\sparse\include"
/I"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.0\include"
/Zi
/Gm-
/Od
/Ob0
/Fd"magma-test.dir\Debug\vc141.pdb"
/Zc:inline
/fp:precise
/D "WIN32"
/D "_WINDOWS"
/D "CMAKE_INTDIR=\"Debug\""
/D "_MBCS" /errorReport:prompt
/WX-
/Zc:forScope
/RTC1
/GR
/Gd
/MTd
/Fa"Debug/"
/EHsc
/nologo
/Fo"magma-test.dir\Debug\"
/Fp"magma-test.dir\Debug\magma-test.pch"
/diagnostics:classic
The example code linker arguments line generated is
Code: Select all
/OUT:"D:\Work\bob-domino-master\magma-test-clean\build\Debug\magma-test.exe"
/MANIFEST
/NXCOMPAT
/PDB:"D:/Work/bob-domino-master/magma-test-clean/build/Debug/magma-test.pdb"
/DYNAMICBASE "D:\Work\Magma\magma-2.4.0\build\lib\Debug\magma.lib" "D:\Work\Magma\magma-2.4.0\build\lib\Debug\magma_sparse.lib" "kernel32.lib" "user32.lib" "gdi32.lib" "winspool.lib" "shell32.lib" "ole32.lib" "oleaut32.lib" "uuid.lib" "comdlg32.lib" "advapi32.lib"
/IMPLIB:"D:/Work/bob-domino-master/magma-test-clean/build/Debug/magma-test.lib"
/DEBUG
/MACHINE:X64
/INCREMENTAL
/PGD:"D:\Work\bob-domino-master\magma-test-clean\build\Debug\magma-test.pgd"
/SUBSYSTEM:CONSOLE
/MANIFESTUAC:"level='asInvoker' uiAccess='false'"
/ManifestFile:"magma-test.dir\Debug\magma-test.exe.intermediate.manifest"
/ERRORREPORT:PROMPT
/NOLOGO
/TLBID:1
Mark, you never used CMake? I think newcomers would benefit from a working example CMakeLists.txt, are you aware of anyone that uses CMake that we could ask?
Re: example_sparse.c failing to compile
Posted: Wed Nov 07, 2018 11:12 am
by dariofigueira
Finally compiling and running.
Here is the CMakeLists.txt that did it
Code: Select all
add_executable(magma-test example_sparse.cpp)
find_package( CUDA )
set( MKLROOT "D:/Program Files (x86)/IntelSWTools/parallel_studio_xe_2019.0.045/compilers_and_libraries_2019/windows/mkl" )
set( LAPACK_LIBRARIES
"D:/Program Files (x86)/IntelSWTools/parallel_studio_xe_2019.0.045/compilers_and_libraries_2019/windows/mkl/lib/intel64_win/mkl_intel_lp64.lib"
"D:/Program Files (x86)/IntelSWTools/parallel_studio_xe_2019.0.045/compilers_and_libraries_2019/windows/mkl/lib/intel64_win/mkl_intel_thread.lib"
"D:/Program Files (x86)/IntelSWTools/parallel_studio_xe_2019.0.045/compilers_and_libraries_2019/windows/mkl/lib/intel64_win/mkl_core.lib"
"D:/Program Files (x86)/IntelSWTools/compilers_and_libraries_2019.0.117/windows/compiler/lib/intel64_win/libiomp5md.lib")
target_include_directories(magma-test PUBLIC
"D:/Work/Magma/magma-2.4.0/include"
"D:/Work/Magma/magma-2.4.0/sparse/include"
${CUDA_INCLUDE_DIRS}
${MKLROOT}/include)
target_link_libraries(magma-test
${CUDA_CUDART_LIBRARY}
${CUDA_CUBLAS_LIBRARIES}
${CUDA_cusparse_LIBRARY}
${LAPACK_LIBRARIES}
debug D:/Work/Magma/magma-2.4.0/build/lib/Debug/magma.lib
debug D:/Work/Magma/magma-2.4.0/build/lib/Debug/magma_sparse.lib
optimized D:/Work/Magma/magma-2.4.0/build/lib/Release/magma.lib
optimized D:/Work/Magma/magma-2.4.0/build/lib/Release/magma_sparse.lib)
set(CMAKE_CXX_FLAGS_RELEASE "${CMAKE_CXX_FLAGS_RELEASE} /MT")
set(CMAKE_CXX_FLAGS_DEBUG "${CMAKE_CXX_FLAGS_DEBUG} /MTd")
Re: example_sparse.c failing to compile
Posted: Fri Nov 09, 2018 9:41 am
by dariofigueira
The above compiles and runs only in Release mode..
Anyone can think of a reason for that? (which then I'll try to verify)
EDIT: Dear Mike, or Hartwig,
are you able to compile
with debug symbols? (-g flag in linux correct?)
and statically? (-static flag in linux I believe)
(https://stackoverflow.com/questions/369 ... ary-exists instructs on how to choose only a selection of libraries to be statically linked while others dinamic)