Thursday, January 10, 2019

[C#] YOLO3 with OpenCvSharp4



One year ago, I writed [C#] OpenCvSharp DNN with YOLO2, since YOLO3 come out for a while and OpenCV released version 4 on Dec 2018, I think it's time to make a upgrade.



For windows user, you can get OpenCvSharp4.Windows on nuget(make sure the version are 4.0.0.20190108 or newer), special thank shimat@opencvsharp for his efforts.

Let's jump to code, it almost same as previous project, so I only highlight on change.


//load model and config, if you got error: "separator_index < line.size()", check your cfg file, must be something wrong.
var net = CvDnn.ReadNetFromDarknet(cfg, model);
net.SetPreferableBackend(3);
/*
0:DNN_BACKEND_DEFAULT 
1:DNN_BACKEND_HALIDE 
2:DNN_BACKEND_INFERENCE_ENGINE
3:DNN_BACKEND_OPENCV 
 */
net.SetPreferableTarget(0);
/*
0:DNN_TARGET_CPU 
1:DNN_TARGET_OPENCL
2:DNN_TARGET_OPENCL_FP16
3:DNN_TARGET_MYRIAD 
4:DNN_TARGET_FPGA 
 */
You may notice here have two new setting command, if you didn't need to set prefer option can just mark it using default.

And another thins, I saw some people asking about error "separator_index < line.size()", the error message came from YOLO can't get right config, so please check your cfg file, must be something wrong, maybe you download it as html not cfg.



//get output layer name
var outNames = net.GetUnconnectedOutLayersNames();

//create mats for output layer
Mat[] outs = outNames.Select(_ => new Mat()).ToArray();

//forward model
net.Forward(outs, outNames);
YOLO3 have multi output layer not just one, so we have to find out all output layer and get result from them, if you just get one layer like previous, you will wonder why the detect object and confidence are so low.



//using non-maximum suppression to reduce overlapping low confidence box
CvDnn.NMSBoxes(boxes, confidences, threshold, nmsThreshold, out int[] indices);

foreach (var i in indices)
{
 var box = boxes[i];
 Draw(image, classIds[i], confidences[i], probabilities[i], box.X, box.Y, box.Width, box.Height);
}
at here we using NMSBoxes to remove some low confidence overlapping box, you can turn it off to check what different.


Here is showcase of the result.


Kamikochi, Japan. A place you must to been.


Kite from ssd_mobilenet.


Dog from YOLO.


Shinjuku, Japan.


Bali, I used it multi times.


Rebun Island, Japan. A remote island on north of Japan.



Full code please get from github YOLO3-With-OpenCvSharp4.

Remember, due to github 100MB size limit, you have to down YOLO weight myself from YOLO webite.


Tha's all, hope you enjoy it.

3 comments:

  1. Hey thanks for the articles I have now basic understanding of Yolo I have a question since I am using opencv or opencvsharp it can detect faces using haar cascade classifier now my question is how do we train this faces in yolo so it can generate the .cfg, .names and weights then use this files to detect certain person with corresponding names?

    ReplyDelete
  2. Hi died, Im trying to implement your code to run on webcam to do live detection but i can't seem to figure out what is wrong. I changed the image into videocapture and I used the latest version of the opencvsharpv4. Please help!!

    ReplyDelete