CUDAOpenCV

Build/Compile OpenCV 3.4 in Windows with CUDA 9.1 and Intel MKL + TBB

Because the pre-built Windows libraries available for OpenCV 3.4 do not include the CUDA modules, or the support for Intel’s Math Kernel Libraries (MKL) or Intel Threaded Building Blocks (TBB) performance libraries, I have included the build instructions, below for anyone who is interested. If you just need the Windows libraries then go to Download OpenCV 3.4 with CUDA 9.1. To see the performance boost from calling the OpenCV CUDA functions with these libraries see the OpenCV 3.4 GPU CUDA Performance Comparisson (nvidia vs intel).

The guide below details instructions on compiling the 64 bit version of OpenCV 3.4 shared libraries with Visual Studio 2017 (will also work with Visual Studio 2015 if selected in CMake as long as you do not include the CUDA_HOST_COMPILER option in the commands below), CUDA 9.1, support for both the Intel Math Kernel Libraries (MKL) and Intel Threaded Building Blocks (TBB).

Before continuing there are a few things to be aware of:

  1. CUDA 9.1 is not supported by the latest versions of Visual Stutio 2017, 15.5 onwards, to follow the guide you need to install a supported version, e.g. 15.45 from here, or earlier. If you have already installed the latest version you can downgrade to a previous version or use the precompiled binaries from my download page.
  2. The procedure outlined has only been tested on Visual Studio Community 2017 and 2015.
  3. I have not included instructions for compiling the python bindings because you cannot call the CUDA modules from within python. If you require python support it is better to install directly through conda.
  4. If you have built OpenCV with CUDA support then to use those libraries and/or redistribute applications built with them on any machines without the CUDA toolkit installed, you will need to ensure those machines have,
    • an Nvidia capable GPU with driver version of 388.19 or later, and
    • the following dll’s
      cublas64_91.dll
      cudart64_91.dll
      cufft64_91.dll
      nppc64_91.dll
      nppial64_91.dll
      nppicc64_91.dll
      nppidei64_91.dll
      nppif64_91.dll
      nppig64_91.dll
      nppim64_91.dll
      nppist64_91.dll
      nppitc64_91.dll
      npps64_91.dll
      

      which can be found in your

      C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v9.1\bin

      directory.

  5. The latest version of Intel TBB uses a shared library, therefore if you build with Intel TBB you need to add
    C:\Program Files (x86)\IntelSWTools\compilers_and_libraries\windows\redist\intel64_win\tbb\vc_mt

    to your path variable, and make sure you redistribute that dll with any of your applications.

 

Prerequisites

Assuming you already have a compatible version of Visual Studio (2017 or 2015) installed there are a couple of additional components you need to download before you can get started, you first need to:

  • Download the source files, available on GitHub. Either clone the git repo making sure to checkout the 3.4.0 tag or download this archive containing all the source file.
  • Install CMake – Version 3.10.1 is used in the guide.
  • Install The CUDA 9.1 Toolkit (cuda_9.1.85_windows.exe) + Patch 1 (cuda_9.1.85.1_windows.exe).
  • Optional – Install both the Intel MKL and TBB by registering for community licensing, and downloading for free. MKL version 2018.0.124 and TBB version 2018.0.124 are used in this guide, I cannot guarantee that other versions will work correctly.

 

Generating OpenCV Visual Studio solution files with CMake

In the next section we are going to generate the Visual Studio solution files with CMake. There are two ways to do this, from the command prompt or with the CMake GUI, however quickest and easiest way to proceed is to use the command prompt to generate the base configuration. Then if you want to add any aditional configuration options, you can open up the build directory in the CMake GUI as described here.
 

Generating Visual Studio solution files for OpenCV 3.4 with CUDA 9.1 and Intel MKL + TBB, from the command prompt (cmd)
    1. Open up the command prompt (windows key + r, then type cmd and press enter) and enter
      "C:\Program Files (x86)\IntelSWTools\compilers_and_libraries\windows\tbb\bin\tbbvars.bat" intel64

      to temporarily set the environmental variables for locating your TBB installation.

    2. Set the location of the source files, and your Visual Studio edition, by entering the text shown below, first setting PATH_TO_SOURCE to the root of the OpenCV files you downloaded or cloned (the directory containing 3rdparty,apps,build,etc.). If you not using Visual Studio Community 2017 then you also need to replace Community with the edition you have installed (Professional or Enterprise).
      set "openCvSource=PATH_TO_SOURCE"
      set "vsEdition=Community"
    3. Then choose your configuration from below and copy to the command prompt. To build with Visual Studio 2015 instead of 2017 replace -G”Visual Studio 15 2017 Win64″ with -G”Visual Studio 14 2015 Win64″ and remove -DCUDA_HOST_COMPILER onwards:
      • OpenCV 3.4 with CUDA 9.1
        "C:\Program Files\CMake\bin\cmake.exe" -B"%openCvSource%\build" -H"%openCvSource%/" -G"Visual Studio 15 2017 Win64" -DBUILD_opencv_world=ON -DCUDA_FAST_MATH=ON -DWITH_CUBLAS=ON -DINSTALL_TESTS=ON -DINSTALL_C_EXAMPLES=ON -DBUILD_EXAMPLES=ON -DCUDA_HOST_COMPILER="C:/Program Files (x86)/Microsoft Visual Studio/2017/%vsEdition%/VC/Tools/MSVC/$(VCToolsVersion)/bin/Host$(Platform)/$(PlatformTarget)/cl.exe"
      • OpenCV 3.4 with CUDA 9.1 and MKL multi-threaded with TBB
        "C:\Program Files\CMake\bin\cmake.exe" -B"%openCvSource%\build" -H"%openCvSource%/" -G"Visual Studio 15 2017 Win64" -DBUILD_opencv_world=ON -DCUDA_FAST_MATH=ON -DWITH_CUBLAS=ON -DWITH_MKL=ON -DMKL_USE_MULTITHREAD=ON -DMKL_WITH_TBB=ON -DINSTALL_TESTS=ON -DINSTALL_C_EXAMPLES=ON -DBUILD_EXAMPLES=ON -DCUDA_HOST_COMPILER="C:/Program Files (x86)/Microsoft Visual Studio/2017/%vsEdition%/VC/Tools/MSVC/$(VCToolsVersion)/bin/Host$(Platform)/$(PlatformTarget)/cl.exe"
      • OpenCV 3.4 with CUDA 9.1, MKL multi-threaded with TBB and TBB
        "C:\Program Files\CMake\bin\cmake.exe" -B"%openCvSource%\build" -H"%openCvSource%/" -G"Visual Studio 15 2017 Win64" -DBUILD_opencv_world=ON -DCUDA_FAST_MATH=ON -DWITH_CUBLAS=ON -DWITH_MKL=ON -DMKL_USE_MULTITHREAD=ON -DMKL_WITH_TBB=ON -DWITH_TBB=ON -DINSTALL_TESTS=ON -DINSTALL_C_EXAMPLES=ON -DBUILD_EXAMPLES=ON -DCUDA_HOST_COMPILER="C:/Program Files (x86)/Microsoft Visual Studio/2017/%vsEdition%/VC/Tools/MSVC/$(VCToolsVersion)/bin/Host$(Platform)/$(PlatformTarget)/cl.exe"
    4. Your OpenCV.sln solution file should now be in your PATH_TO_SOURCE/build directory, open it in Visual Studio and select your Configuration.
    5. Click Solution Explorer, expand CMakeTargets, right click on INSTALL and click Build.

      This will both build the library and copy the necessary redistributable parts to the install directory, PATH_TO_SOURCE/build/install in this example. All that is required now to run any programs compiled against these libs is to add the directory containing opencv_world340.dll (and tbb.dll if you have build with Intel TBB) to you path environmental variable.

      If everything was successful, congratulations, you now have OpenCV 3.4 built with CUDA 9.1.

 

Adding additional configuration options with the CMake GUI

Once you have generated the base Visual Studio solution file from the command prompt the easiest way to make any aditional configuration changes is through the CMake GUI. To do this:

  1. Fire up the CMake GUI.
  2. Making sure that the Grouped checkbox is ticked, click on the browse build button
     

     
    and navigate to your PATH_TO_SOURCE/build directory. If you have selected the correct directory the main CMake window should resemble the below.
     

     
  3. Now any additional configuration changes can be made by just expanding any of the grouped items and ticking or unticking the values displayed. Once you are happy just press Configure,
     

     
    if the bottom window displays configuration successful press Generate, and you should see
     

     
    Now you can open up the Visual Studio solution file and proceed as before.
  4. Troubleshooting:
    • If you are using Visual Studio 2017, make sure you are using a version earlier than 15.5, as this version is not compatible with CUDA 9.1.
    • Not all options are compatible with each other and the configuration step may fail as a result. If so examine the error messages given in the bottom window and look for a solution.
    • If the build is failing after making changes to the base configuration, I would advise you to remove the build directory and start again making sure that you can at least build the base Visual Studio solution files produces from the command line

26 thoughts on “Build/Compile OpenCV 3.4 in Windows with CUDA 9.1 and Intel MKL + TBB

  1. I try to build opencv3.4.0 with cuda9.1 using VS2017. I can generate Visual Studio solution use cmake 3.10.1. But when I do this step “Click Solution Explorer, expand CMakeTargets, right click on INSTALL and click “. I get something wrong. I can’t build opencv_world340.dll/opencv_world340d.dll. Do you know how can I solve this problem? Thanks a lot.

  2. 1>—— 已启动生成: 项目: opencv_world, 配置: Debug x64 ——
    1>Building NVCC (Device) object modules/world/CMakeFiles/cuda_compile.dir/__/core/src/cuda/Debug/cuda_compile_generated_gpu_mat.cu.obj
    1>Failed to run C:/Program Files (x86)/Microsoft Visual Studio/2017/Community/VC/Tools/MSVC/$(VCToolsVersion)/bin/Host$(Platform)/$(PlatformTarget)/cl.exe
    1>
    1>).
    1>CMake Error at cuda_compile_generated_gpu_mat.cu.obj.cmake:206 (message):
    1> Error generating
    1> D:/opencv/build/modules/world/CMakeFiles/cuda_compile.dir/__/core/src/cuda/Debug/cuda_compile_generated_gpu_mat.cu.obj
    1>
    1>
    1>C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Common7\IDE\VC\VCTargets\Microsoft.CppCommon.targets(171,5): error MSB6006: “cmd.exe”已退出,代码为 1。
    1>已完成生成项目“opencv_world.vcxproj”的操作 – 失败。

    1. Hi,
      As you have Visual Studio enterprise installed the path to cl.exe will be different. When you generate the cmake file, the following line
      -DCUDA_HOST_COMPILER=”C:/Program Files (x86)/Microsoft Visual Studio/2017/Community/VC/Tools/MSVC/$(VCToolsVersion)/bin/Host$(Platform)/$(PlatformTarget)/cl.exe”
      should be corrected for your version, I imagine it is something like
      -DCUDA_HOST_COMPILER=”C:/Program Files (x86)/Microsoft Visual Studio/2017/Enterprise/VC/Tools/MSVC/$(VCToolsVersion)/bin/Host$(Platform)/$(PlatformTarget)/cl.exe”

  3. Firstly, thank you James Bowley and Parallel Vision for these well done and timely instructions. Before beginning the OpenCV build I upgraded to Visual Studio to the most recent 15.5.5 version, and experienced a number of compiler errors that prevented a successful build of opencv_world340.dll, which appears to be the same error that Zexi Shao saw. After uninstalling VS 2017 15.5.5 and installing the 15.4.5 version I was able to build opencv_world340.dll and all of OpenCV. Note that the CUDA Forum also has many reports of compatibility issues with the latest VS2017.

    1. Hi, thank you for bringing this to my attention, I did not realize there was a problem with the latest version of Visual Studio. I have amended the guide, as per your recomendation, and verified that 15.45 is working.

  4. Hi,
    Thanks for this post.
    Just a quick note I found: for future installers, make sure your CMake is set to the right generator before running step 3.

    If you have run step 3 with the wrong generator, just delete the build folder you already made, and try step 3 again after fixing your generator.

    1. Hi, thank you for your comment, but I am not sure what you mean by the right generator? The guide is for VS2017, with alternative instructions if you want to build with VS2015?

      1. Hi,
        You may be able to explain better than me, or maybe I have just not followed the instructions properly.
        However, my install of CMake defaulted to the VS2015 generator, where my step 3 was build using VS 2017 (15.42). Once I changed my CMake tothe VS2017 generator, and repeated the steps, everything worked.

        Cheers

        1. Hi, just to confirm you ran the below command with -G”Visual Studio 15 2017 Win64″
          “C:\Program Files\CMake\bin\cmake.exe” -B”%openCvSource%\build” -H”%openCvSource%/” -G”Visual Studio 15 2017 Win64″ -DBUILD_opencv_world=ON -DCUDA_FAST_MATH=ON -DWITH_CUBLAS=ON -DINSTALL_TESTS=ON -DINSTALL_C_EXAMPLES=ON -DBUILD_EXAMPLES=ON -DCUDA_HOST_COMPILER=”C:/Program Files (x86)/Microsoft Visual Studio/2017/%vsEdition%/VC/Tools/MSVC/$(VCToolsVersion)/bin/Host$(Platform)/$(PlatformTarget)/cl.exe”
          and cmake defaulted to VS 2015 when you opened it up in the CMake GUI?

  5. I’m seeing the same bug as Zexi, using VS 2017 15.6.3, CMake 3.11.0-rc4, Cuda 9.1 (with all applied patches). The .CU files aren’t compiling. I have the 16299 SDK installed…
    Anybody find a fix for this yet? “Using the correct generator” (commented above) doesn’t help. I’m using Visual Studio 15 2017 Win64…

  6. First, I want to say thank you for this tutorial!
    Now, I’ve been trying to compile OpenCV 3.4.1 with Cuda 9.1 following these steps for longer than I would care to admit. I keep getting this error, and I was hoping for a little help. I can get it to compile without opencv_world or without Cuda, but as soon as both are enabled I’m getting this error. I’m using VS 2017 community 15.4.5.

    Any help would be greatly appreciated!

    10>—— Build started: Project: opencv_world, Configuration: Debug x64 ——
    10>Building NVCC (Device) object modules/world/CMakeFiles/cuda_compile.dir/__/core/src/cuda/Debug/cuda_compile_generated_gpu_mat.cu.obj
    10>CMake Warning (dev) at cuda_compile_generated_gpu_mat.cu.obj.cmake:79:
    10> Syntax Warning in cmake code at column 108
    10>
    10> Argument not separated from preceding token by whitespace.
    10>This warning is for project developers. Use -Wno-dev to suppress it.
    10>
    10>CMake Warning (dev) at cuda_compile_generated_gpu_mat.cu.obj.cmake:79:
    10> Syntax Warning in cmake code at column 227
    10>
    10> Argument not separated from preceding token by whitespace.
    10>This warning is for project developers. Use -Wno-dev to suppress it.
    10>
    10>nvcc fatal : A single input file is required for a non-link phase when an outputfile is specified
    10>CMake Error at cuda_compile_generated_gpu_mat.cu.obj.cmake:206 (message):
    10> Error generating
    10> C:/git/opencv/build/modules/world/CMakeFiles/cuda_compile.dir/__/core/src/cuda/Debug/cuda_compile_generated_gpu_mat.cu.obj
    10>
    10>
    10>C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\Common7\IDE\VC\VCTargets\Microsoft.CppCommon.targets(171,5): error MSB6006: “cmd.exe” exited with code 1.
    10>Done building project “opencv_world.vcxproj” — FAILED.

    1. Hi Michael, I haven’t seen that error before, just to confirm, have you checked out the 3.4.0 tag from the git repo?

      1. I got the 3.4.1 from git, and the 3.4.0 by downloading a zip. I have tried this with all combinations of VS 2015, VS 2017 15.4.5, opencv 3.4.0, and opencv 3.4.1 source code (deleting the build folder between each try), and I get the same error when both opencv_world and cuda are enabled (but they will all compile with one or the other enabled). Because of the consistency, I’m getting the feeling that it’s an error on my end, but I’m at a loss and getting desperate to locate the problem.

        1. Hi MiChael, I suffers the same error. Have you solved it? Could you give me some hints? Thank you very much.

          1. Michael had this problem on a specific machine. He managed to compile it successfully on another one. Are you getting exactly the same error, are you using 3.4.0, on VS2017 15.4.5?

          2. Hi, I am using OpenCV 3.4.1 and OpenCV Contrib 3.4.1 with VS2017 15.4.5. But I haved installed VS 2017 15.6.5 on this machine before, and uninstalled it. I don’t know whether it is the cause. I will try on another machine. Thank you for your help.

          3. The guide is for 3.4.0 but I have tested it on 3.4.1 and apart from two examples everything should compile. I have not however tested the contrib modules, have you tried compiling without the contrib modules, have your followed the guide exactly if so what command line options are you passing to CMake? Did you check out the 3.4.1 tag or a more recent commit.

            What are the errors you are seeing, are they in CMake or during compilation?

  7. James, thanks a ton for this!
    I was able to build 3.4.0 in VS 2015 Pro with MKL, and TBB and it built flawlessly, with almost not effort at all.
    I did notice that it looks like I’ll have to find the relevant header files for opencv cuda and move them into the opencv include folder.
    Did I miss something? I would have hoped their config file would have moved them as part of the build.
    Thanks again.

    1. Hi, if you build the INSTALL solution, all of the headers should be contained insided PATH_TO_SOURCE/build/install/include, are they missing?

      1. There is a build/win_install folder, but not build/install
        I did notice that in one of your screenshots it looks like ALL_BUILD is still the startup project.
        I built the INSTALL project, but switched it to be startup. could that be it?

        1. The win_install folder is different and the startup project should not make any difference. Try building the INSTALL project again, if it was successful last time, it should just create and populate and install directory and not build anything.

          1. OK, I simply didn’t follow directions carefully enough.
            In step 5 I changed the default startup project and selected build from the menu.
            If right clicking on the INSTALL project it will build if needed and install.
            I suppose I could check install in config manager, but why bother. 🙂
            Thanks again for a great tutorial.
            Now to get on with learning more opencv.

Leave a Reply

Your email address will not be published. Required fields are marked *