[Python] Using deep learning model in OpenCV | 在OpenCV中使用深度學習模型

OpenCV 3.3 included dnn module in the main repository, now we can use popular framework like Caffe, TensorFlow in OpenCV via dnn module.
OpenCV 3.3將dnn模組包入了主庫，現在我們可以在OpenCV中藉由dnn模組使用像是Caffe，TensorFlow這些熱門框架。

This post we will follow this dnn sample to test it (with some modify), because OpenCvSharp didn't support OpenCV 3.3 yet, so we will using python to do it.
這篇文章我們將照著這篇dnn範例來測試(有做一點修改)，因為OpenCvSharp還沒支援OpenCV 3.3，所以我們將使用python來做這測試。



At first, load image and calculate mean, make blob object. Note: because we are using googlenet, it only support 224x224 img , so we need resize it.
首先，讀入圖片並計算平均值，產生blob物件。注意：因為我們是用googlenet，他只支援224x224的圖，所以我們要對圖做縮放的動作。

image = cv2.imread('green_apple.jpg')
mean,std_dev = cv2.meanStdDev(image, np.zeros((224, 224)), np.zeros((224, 224)))
blob = dnn.blobFromImage(image, 1, (224, 224), (mean[0][0], mean[1][0], mean[2][0]), False)

Load googlenet pre-trained model and forward it, get top 3 result for later use. You can get the model file from bvlc_googlenet.
載入googlenet預先訓練過的模型然後計算結果，取得最佳的三個結果之後使用。你可以在bvlc_googlenet取得模型資料。

net = dnn.readNetFromCaffe('bvlc_googlenet.prototxt', 'bvlc_googlenet.caffemodel')
net.setInput(blob)
prob = net.forward()
timeit_forward(net)        #Uncomment to check performance
idxs = np.argsort(prob[0])[::-1][:3]

Finally, display top 3 prediction result and source image.
最後，顯示最佳三個預測結果與來源圖片

for (i, idx) in enumerate(idxs):
    print("#{0} {1:.6f} Label:{2} ".format(i + 1, prob[0][idx] , classes[idx].replace("\n","")))
cv2.imshow("Image", image)

The full code is here.
完整程式碼如下。

import numpy as np
import cv2
from cv2 import dnn
import timeit

def timeit_forward(net):
    print("Runtime:", timeit.timeit(lambda: net.forward(), number=10))

def get_class_list():
    with open('synset_words.txt', 'rt') as f:
        return [x[x.find(" ") + 1:] for x in f]

#Get image and calculate mean
image = cv2.imread('green_apple.jpg')
mean,std_dev = cv2.meanStdDev(image, np.zeros((224, 224)), np.zeros((224, 224)))
blob = dnn.blobFromImage(image, 1, (224, 224), (mean[0][0], mean[1][0], mean[2][0]), False)

#Input image and forward in model
net = dnn.readNetFromCaffe('bvlc_googlenet.prototxt', 'bvlc_googlenet.caffemodel')
net.setInput(blob)
prob = net.forward()
timeit_forward(net)        #Uncomment to check performance
idxs = np.argsort(prob[0])[::-1][:3]

#Get label list
classes = get_class_list()

#Print top 3 result and show image
for (i, idx) in enumerate(idxs):
    print("#{0} {1:.6f} Label:{2} ".format(i + 1, prob[0][idx] , classes[idx].replace("\n","")))
cv2.imshow("Image", image)
cv2.waitKey(0)

You can find more pre-trained model on Model Zoo
你可以在Model Zoo找到更多預先訓練過的模型。


Here coming more test result.
這邊多來幾個測試結果。


Granny Smith is a apple cultivar, the green one, so it is right answer.
Granny Smith是個蘋果品種，綠色的那種，所以是正確答案。



I guess they don't have Bugatti Veyron label, so sport car count as right answer.
我猜它們沒有Bugatti Veyron這個標籤，所以跑車算是正確答案。



Sorrel is a term means reddish-brown color horses, one of the most common equine coat colors in horses. So it's right answer too.
Sorrel一詞是指紅棕色的馬(栗毛馬)，最普遍的馬匹毛色的其中一種。所以也是正確答案。


Hope you enjoy it.