OpenCV DNN speed compare in Python, C#, C++

Last time I tried SSD on Python + OpenCV DNN, this time I will implement same test using C# and C++ to test performance and the difficult of implement.

To be fair, I will use same model and same test image on those 3 tests, model will be "VGG_VOC0712Plus_SSD_512x512_ft_iter_160000.caffemodel" from here (07++12+COCO: SSD512), image will be one picture I shot at Bail few year ago.

Test environment

• Hardware: Intel Core i7-7820HQ @ 2.90GHz (OpenCV DNN can't enable GPU so didn't list)
• Python: Python 3.6.3, OpenCv-Python 3.3.0.10
• C#: .NET Framework 4.7, EMGU.CV 3.3.0.2824
• C++: OpenCv 3.3.1

SSD with Python OpenCV DNN

Detail please check my previous post.

RunTime: 1510 ms

SSD with C# EmguCV DNN

Because OpenCvSharp still didn't support OpenCV 3.3, so I have to change to EmguCV, it can easy install via NuGet, but EmguCV API namespace are way different with OpenCV and it can't direct access Mat's pixel, so I spent a lot of time on implement my demo code from OpenCv to EmguCV.

RunTime: 4041 ms


And here has a problem, using EmguCV to evaluate test image, the result will different with other framework, you can obviously see here have a wrong rectangle at bottom of image labeled as person.

I think it is Emgu CV bug, not sure come from DNN module or not, if you evaluate other image, the wrong rectangle still appear here.

SSD with C++ OpenCV DNN

I running C++/OpenCV on Windows, follow this introduction can easy install OpenCv to your pc, and make sure follow this guide to setup your environment and solution if you want to run C++ on Visual Studio like me.

RunTime: 9306 ms

Summary

Speed: Python > C# > C++

Yes, it was weird, but i can't figure it out the real reason, I only can guess it come from framework or not native code or something else...

Anyway, my test result was here, please left comments if you have any opinions, I really want to know why running OpenCv Dnn in C++ was 6 times slow than running in python.

---

Source Code

All source code already put on GuiHub/OpenCvDnnSpeedCompare.


Python code in previous post.

C++ code part from OpenCv Dnn Moudle Sample, so I didn't add much comments.

#include "stdafx.h"
#include <opencv2/opencv.hpp>
#include <opencv2/dnn.hpp>
using namespace cv;
using namespace cv::dnn;
using namespace std;

const char* classNames[] = { "background","aeroplane", "bicycle", "bird", "boat","bottle", "bus", "car", "cat", "chair","cow", "diningtable", "dog", "horse","motorbike", "person", "pottedplant","sheep", "sofa", "train", "tvmonitor" };

int main()
{
 Scalar colors[21];
 srand((unsigned)time(NULL));
 for (int i = 0; i < 21; i++) {
  colors[i] = Scalar(rand() % 256, rand() % 256, rand() % 256);
 }

 String modelTxt = "deploy.prototxt";
 String modelBin = "VGG_VOC0712Plus_SSD_512x512_ft_iter_160000.caffemodel";
 String imageFile = "bali-crop.jpg";

 Net net = dnn::readNetFromCaffe(modelTxt, modelBin);
 Mat img = cv::imread(imageFile);

 Mat inputBlob = blobFromImage(img, 1, Size(512, 512));   //Convert Mat to batch of images
 Mat prob;

 cv::TickMeter t;
 net.setInput(inputBlob, "data");        //set the network input
 t.start();
 prob = net.forward("detection_out");                          //compute output
 t.stop();
 cout << "Runtime: " << (double)t.getTimeMilli() << " ms " << endl;

 Mat detectionMat(prob.size[2], prob.size[3], CV_32F, prob.ptr());

 for (int i = 0; i < detectionMat.rows; i++)
 {
  float confidence = detectionMat.at(i, 2);
  if (confidence > 0.4)
  {
   size_t objectClass = (size_t)(detectionMat.at(i, 1));

   int xLeftBottom = static_cast(detectionMat.at(i, 3) * img.cols);
   int yLeftBottom = static_cast(detectionMat.at(i, 4) * img.rows);
   int xRightTop = static_cast(detectionMat.at(i, 5) * img.cols);
   int yRightTop = static_cast(detectionMat.at(i, 6) * img.rows);

   Rect object(xLeftBottom, yLeftBottom, xRightTop - xLeftBottom, yRightTop - yLeftBottom);

   rectangle(img, object, colors[objectClass],2);
   String label = String(classNames[objectClass]) + ": " + to_string(confidence);
   std::cout << label << endl;

   int baseLine = 0;
   Size labelSize = getTextSize(label, FONT_HERSHEY_TRIPLEX, 0.5, 1, &baseLine);
   rectangle(img, Rect(Point(xLeftBottom, yLeftBottom - labelSize.height),
    Size(labelSize.width, labelSize.height + baseLine)), colors[objectClass], CV_FILLED);
   putText(img, label, Point(xLeftBottom, yLeftBottom), FONT_HERSHEY_TRIPLEX, 0.5, Scalar(0,0,0));
  }
 }

 cv::imshow("image", img);
 cv::waitKey(0);
 return 0;
}

C# code was here, seems no one using Emgu CV Dnn with SSD(at least I didn't see it on google search), so this sample may help you if you want to use OpenCV DNN on C# (until OpenCvSharp support 3.3).

But be honest, I don't recommend Emgu CV, as a wrapper of C# over openCV, Emgu CV has part different with OpenCV, so most of time you can't using OpenCV resource in troubleshooting since Emgu CV got fewer user and fewer support.

using System;
using System.Diagnostics;
using System.Drawing;
using Emgu.CV;
using Emgu.CV.CvEnum;
using Emgu.CV.Dnn;
using Emgu.CV.Structure;

namespace emguCvSsd
{
    class Program
    {
        private static readonly string[] Labels = {"background", "aeroplane", "bicycle", "bird", "boat","bottle", "bus", "car", "cat", "chair","cow", "diningtable", "dog", "horse", "motorbike","person", "pottedplant", "sheep", "sofa", "train", "tvmonitor"};
        private static readonly MCvScalar[] Colors = new MCvScalar[21];

        static void Main()
        {
            //set random color
            Random rnd = new Random();
            for (int i = 0; i < 21; i++)
            {
                Colors[i] = new Rgb(rnd.Next(0,256), rnd.Next(0, 256), rnd.Next(0, 256)).MCvScalar;
            }
            //get image and set model
            Mat img = CvInvoke.Imread("bali-crop.jpg");
            var blob = DnnInvoke.BlobFromImage(img, 1, new Size(512, 515));
            var prototxt = "deploy.prototxt";
            var model = "VGG_VOC0712Plus_SSD_512x512_ft_iter_160000.caffemodel";
            var net = new Net();
            var import = Importer.CreateCaffeImporter(prototxt, model);
            import.PopulateNet(net);
            net.SetInput(blob, "data");
            
            Stopwatch sw = new Stopwatch();
            sw.Start();
            //forward model
            var prob = net.Forward("detection_out");
            sw.Stop();
            Console.WriteLine($"Runtime:{sw.ElapsedMilliseconds} ms");

            //copy result to byte due to egmucv can not access Mat pixel.
            byte[] data = new byte[5600];
            prob.CopyTo(data);

            //draw result
            for (int i = 0; i < prob.SizeOfDimemsion[2]; i++)
            {
                var d = BitConverter.ToSingle(data, i * 28 + 8);
                if (d > 0.4)
                {
                    var idx = (int)BitConverter.ToSingle(data, i * 28 + 4);
                    var w1 = (int) (BitConverter.ToSingle(data, i * 28 + 12) * img.Width);
                    var h1 = (int)(BitConverter.ToSingle(data, i * 28 + 16) * img.Height);
                    var w2 = (int)(BitConverter.ToSingle(data, i * 28 + 20) * img.Width);
                    var h2 = (int)(BitConverter.ToSingle(data, i * 28 + 24) * img.Height);

                    var label = $"{Labels[idx]} {d * 100:0.00}%";
                    Console.WriteLine(label);
                    CvInvoke.Rectangle(img,new Rectangle(w1,h1,w2-w1,h2-h1),Colors[idx],2);
                    int baseline = 0;
                    var textSize = CvInvoke.GetTextSize(label, FontFace.HersheyTriplex, 0.5, 1, ref baseline);
                    var y = h1 - textSize.Height < 0 ? h1 + textSize.Height : h1;
                    CvInvoke.Rectangle(img,new Rectangle(w1,y-textSize.Height,textSize.Width, textSize.Height),Colors[idx],-1);
                    CvInvoke.PutText(img, label, new Point(w1, y), FontFace.HersheyTriplex, 0.5, new Bgr(0, 0, 0).MCvScalar);
                }
            }

            //Show the image
            CvInvoke.Imshow("image", img); 
            CvInvoke.WaitKey();  
            CvInvoke.DestroyAllWindows();
        }
    }
}