目錄
1.安裝依賴
2.從項目目錄下運作安裝程式
3.下載下傳預處理好的COCO權重模型，
4.安裝資料标記工具labelme
5.資料集準備
6.做資料标記
7. 轉換标記資料
8.整理資料檔案
9.建立訓練檔案
10.運作訓練檔案
11.預測模型
12.結果顯示
準備:先從https://github.com/matterport/Mask_RCNN下載下傳Mask_RCNN。
注意:一定要注意環境問題！!如果你是tensorflow-gpu2.x的很多函數都需要改，因為tf2中已經廢除或者直接删除了一些函數，你需要用tf2中的新方法替代掉原來Mask_RCNN的tf1老方法，Mask_rcnn是tf1.x版本的，當然最好用tf1.x版運作，tf2.x的話得改不少......這裡我改的過程就不說了(挺費勁的).....下面給的訓練代碼和預測代碼為了盡量的通用性給的都是1.x版本的。
1.安裝依賴
首先我們需要安裝所需依賴，我們的依賴是全部寫在了requirement檔案裡，可直接通過下面的指令Pip。
pip install -i https://pypi.mirrors.ustc.edu.cn/simple/ -r requirements.txt
requirements檔案内容是這樣的，這包含了我們所需要的依賴包。
（注意這裡的tensorflow以及keras可以改成你那個版本，也可以直接去掉)
2.從項目目錄下運作安裝程式
python setup.py install
3.下載下傳預處理好的COCO權重模型，
可以去github也可以去csdn下載下傳，總共二百多M，github比較慢
github: https://github.com/matterport/Mask_RCNN/releases
csdn:https://download.csdn.net/download/wanghuiqiang1/14965340
4.安裝資料标記工具labelme
1.打開Anaconda Prompt,連接配接清華鏡像:
conda config --add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/msys2/
conda config --add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge/
conda config --add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/free/
conda config --set show_channel_urls yes
2.輸入
conda create --name=labelme python=3.7
激活lableme指令
activate labelme
安裝pyqt5
pip install pyqt5 -i https://pypi.mirrors.ustc.edu.cn/simple
安裝lableme
pip install labelme -i https://pypi.mirrors.ustc.edu.cn/simple
（建議将labelme的啟動寫成一個bat腳本
這樣免得每次啟動labelme都要conda activate相應環境，激活，比較麻煩，舉個例子，bat腳本内容可以如下：
@echo.
@echo ***進入conda指令行，并直接切換到labelme環境***
@echo.
@echo.
@echo.
CALL C:\ProgramData\Anaconda3\Scripts\activate.bat C:\ProgramData\Anaconda3\envs\labelme
@echo.
@echo.
@echo.
@echo ***啟動labelme***
@echo.
@echo.
labelme.exe
）
這樣就可以每次輕按兩下bat就可以啟動Lableme了。
5.資料集準備
python爬取百度圖檔(我這裡爬的是送你一朵小紅花劇照)
# -*- coding: utf-8 -*-
import re
import requests
from urllib import error
from bs4 import BeautifulSoup
import os
num = 0
numPicture = 0
file = ''
List = []
def Find(url, A):
global List
print('正在檢測圖檔總數，請稍等.....')
t = 0
i = 1
s = 0
while t < 1000:
Url = url + str(t)
try:
# 這裡搞了下
Result = A.get(Url, timeout=7, allow_redirects=False)
except BaseException:
t = t + 60
continue
else:
result = Result.text
pic_url = re.findall('"objURL":"(.*?)",', result, re.S) # 先利用正規表達式找到圖檔url
s += len(pic_url)
if len(pic_url) == 0:
break
else:
List.append(pic_url)
t = t + 60
return s
def recommend(url):
Re = []
try:
html = requests.get(url, allow_redirects=False)
except error.HTTPError as e:
return
else:
html.encoding = 'utf-8'
bsObj = BeautifulSoup(html.text, 'html.parser')
div = bsObj.find('div', id='topRS')
if div is not None:
listA = div.findAll('a')
for i in listA:
if i is not None:
Re.append(i.get_text())
return Re
def dowmloadPicture(html, keyword):
global num
# t =0
pic_url = re.findall('"objURL":"(.*?)",', html, re.S) # 先利用正規表達式找到圖檔url
print('找到關鍵詞:' + keyword + '的圖檔，即将開始下載下傳圖檔...')
for each in pic_url:
print('正在下載下傳第' + str(num + 1) + '張圖檔，圖檔位址:' + str(each))
try:
if each is not None:
pic = requests.get(each, timeout=7)
else:
continue
except BaseException:
print('錯誤，目前圖檔無法下載下傳')
continue
else:
string = file + r'\\' + keyword + '_' + str(num) + '.jpg'
fp = open(string, 'wb')
fp.write(pic.content)
fp.close()
num += 1
if num >= numPicture:
return
if __name__ == '__main__': # 主函數入口
##############################
# 這裡加了點
headers = {
'Accept-Language': 'zh-CN,zh;q=0.8,zh-TW;q=0.7,zh-HK;q=0.5,en-US;q=0.3,en;q=0.2',
'Connection': 'keep-alive',
'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/60.0',
'Upgrade-Insecure-Requests': '1'
}
A = requests.Session()
A.headers = headers
###############################
word = input("請輸入搜尋關鍵詞(可以是人名，地名等): ")
# add = 'http://image.baidu.com/search/flip?tn=baiduimage&ie=utf-8&word=%E5%BC%A0%E5%A4%A9%E7%88%B1&pn=120'
url = 'https://image.baidu.com/search/flip?tn=baiduimage&ie=utf-8&word=' + word + '&pn='
# 這裡搞了下
tot = Find(url, A)
Recommend = recommend(url) # 記錄相關推薦
print('經過檢測%s類圖檔共有%d張' % (word, tot))
numPicture = int(input('請輸入想要下載下傳的圖檔數量 '))
file = input('請建立一個存儲圖檔的檔案夾，輸入檔案夾名稱即可')
y = os.path.exists(file)
if y == 1:
print('該檔案已存在，請重新輸入')
file = input('請建立一個存儲圖檔的檔案夾，)輸入檔案夾名稱即可')
os.mkdir(file)
else:
os.mkdir(file)
t = 0
tmp = url
while t < numPicture:
try:
url = tmp + str(t)
# 這裡搞了下
result = A.get(url, timeout=10, allow_redirects=False)
except error.HTTPError as e:
print('網絡錯誤，請調整網絡後重試')
t = t + 60
else:
dowmloadPicture(result.text, word)
t = t + 60
print('目前搜尋結束，感謝使用')
print('猜你喜歡')
for re in Recommend:
print(re, end=' ')
6.做資料标記
使用資料标記工具，直接打開之前做好的腳本或者conda activate相應環境，激活，labelme也可以。在pycharm中可以直接labelme。
這個就是苦力活了.......
7. 轉換标記資料
單個轉換:labelme_json_to_dataset xxxx.json(替換成你生成的json檔案名)
批量轉換的代碼:
import os
path = r'C:\Users\apple\Desktop\labelme\json' # path是你存放json的路徑
json_file = os.listdir(path)
for file in json_file:
os.system("labelme_json_to_dataset %s" % (path + '/' + file))
會生成這堆東西，一定要注意一個都别少，特别是那個info.yml。
8.整理資料檔案
建立四個這種空檔案夾
注意:
放原來用lambel轉換後的json檔案，就是這些
labelme_json檔案夾放用json檔案批量轉換後生成的東西，就是這些
pic檔案夾放原圖
最後要注意的就是我們需要把原來生成的那五個檔案中的label.png依次改名，改成這樣
，然後移動到cv2_mask，當然這裡你自己一個個手動改名然後複制是比較麻煩的，可以寫個python腳本實作，這裡提供一個python腳本。
import os
import shutil
filename = "F:/deeplearning/mask_image/json/" # 存放json轉化得到的檔案夾名稱，需要保證沒有隐藏的檔案夾
fileList = os.listdir(filename)
"""
抽取json轉化得到的5個檔案中的label.png，并重新命名
"""
for i in range(len(fileList)):
path = filename + fileList[i];
# 如果不是檔案夾，跳過
if os.path.isfile(path):
continue
no = fileList[i][:-5] # 從檔案夾上取出檔案名
mask_source = path + "/label.png"
mask_target = "F:/deeplearning/mask_image/cv2_mask/{}.png".format(no) # 命名為“檔案名.png”
shutil.copy(mask_source, mask_target) # 利用shutil直接copy過去
最後要確定的是這四個檔案夾中的命名要注意的就是命名格式要統一，除了labelme_json裡面的檔案夾命名是自定義命名+編号_json ，其他的檔案名都是自定義命名+編号.檔案類型。
9.建立訓練檔案
然後我們需要在項目的根目錄下建立用于訓練的python檔案（我定好的檔案名是mask_train.py），代碼是
# -*- coding: utf-8 -*-
import os
import sys
import random
import math
import re
import time
import numpy as np
import cv2
import matplotlib
import matplotlib.pyplot as plt
import tensorflow as tf
from mrcnn.config import Config
# import utils
from mrcnn import model as modellib, utils
from mrcnn import visualize
import yaml
from mrcnn.model import log
from PIL import Image
# os.environ["CUDA_VISIBLE_DEVICES"] = "0"
# Root directory of the project
ROOT_DIR = os.getcwd()
# ROOT_DIR = os.path.abspath("../")
# Directory to save logs and trained model
MODEL_DIR = os.path.join(ROOT_DIR, "logs")
iter_num = 0
# Local path to trained weights file
COCO_MODEL_PATH = os.path.join(ROOT_DIR, "mask_rcnn_coco.h5")
# Download COCO trained weights from Releases if needed
if not os.path.exists(COCO_MODEL_PATH):
utils.download_trained_weights(COCO_MODEL_PATH)
class ShapesConfig(Config):
"""Configuration for training on the toy shapes dataset.
Derives from the base Config class and overrides values specific
to the toy shapes dataset.
"""
# Give the configuration a recognizable name
NAME = "shapes"
# Train on 1 GPU and 8 images per GPU. We can put multiple images on each
# GPU because the images are small. Batch size is 8 (GPUs * images/GPU).
GPU_COUNT = 1
IMAGES_PER_GPU = 1
# Number of classes (including background)
NUM_CLASSES = 1 + 1 # background + 1 shapes
# Use small images for faster training. Set the limits of the small side
# the large side, and that determines the image shape.
IMAGE_MIN_DIM = 320
IMAGE_MAX_DIM = 384
# Use smaller anchors because our image and objects are small
RPN_ANCHOR_SCALES = (8 * 6, 16 * 6, 32 * 6, 64 * 6, 128 * 6) # anchor side in pixels
# Reduce training ROIs per image because the images are small and have
# few objects. Aim to allow ROI sampling to pick 33% positive ROIs.
TRAIN_ROIS_PER_IMAGE = 100
# Use a small epoch since the data is simple
STEPS_PER_EPOCH = 100
# use small validation steps since the epoch is small
VALIDATION_STEPS = 50
config = ShapesConfig()
config.display()
class DrugDataset(utils.Dataset):
# 得到該圖中有多少個執行個體（物體）
def get_obj_index(self, image):
n = np.max(image)
return n
# 解析labelme中得到的yaml檔案，進而得到mask每一層對應的執行個體标簽
def from_yaml_get_class(self, image_id):
info = self.image_info[image_id]
with open(info['yaml_path']) as f:
temp = yaml.load(f.read())
labels = temp['label_names']
del labels[0]
return labels
# 重新寫draw_mask
def draw_mask(self, num_obj, mask, image, image_id):
# print("draw_mask-->",image_id)
# print("self.image_info",self.image_info)
info = self.image_info[image_id]
# print("info-->",info)
# print("info[width]----->",info['width'],"-info[height]--->",info['height'])
for index in range(num_obj):
for i in range(info['width']):
for j in range(info['height']):
# print("image_id-->",image_id,"-i--->",i,"-j--->",j)
# print("info[width]----->",info['width'],"-info[height]--->",info['height'])
at_pixel = image.getpixel((i, j))
if at_pixel == index + 1:
mask[j, i, index] = 1
return mask
# 重新寫load_shapes，裡面包含自己的自己的類别
# 并在self.image_info資訊中添加了path、mask_path 、yaml_path
# yaml_pathdataset_root_path = "/tongue_dateset/"
# img_floder = dataset_root_path + "rgb"
# mask_floder = dataset_root_path + "mask"
# dataset_root_path = "/tongue_dateset/"
def load_shapes(self, count, img_floder, mask_floder, imglist, dataset_root_path):
"""Generate the requested number of synthetic images.
count: number of images to generate.
height, width: the size of the generated images.
"""
# Add classes
self.add_class("shapes", 1, "person")
for i in range(count):
# 擷取圖檔寬和高
print(i)
filestr = imglist[i].split(".")[0]
# print(imglist[i],"-->",cv_img.shape[1],"--->",cv_img.shape[0])
# print("id-->", i, " imglist[", i, "]-->", imglist[i],"filestr-->",filestr)
# filestr = filestr.split("_")[1]
mask_path = mask_floder + "/" + filestr + ".png"
yaml_path = dataset_root_path + "labelme_json/" + filestr + "_json/info.yaml"
print(dataset_root_path + "labelme_json/" + filestr + "_json/img.png")
cv_img = cv2.imread(dataset_root_path + "labelme_json/" + filestr + "_json/img.png")
self.add_image("shapes", image_id=i, path=img_floder + "/" + imglist[i],
width=cv_img.shape[1], height=cv_img.shape[0], mask_path=mask_path, yaml_path=yaml_path)
# 重寫load_mask
def load_mask(self, image_id):
"""Generate instance masks for shapes of the given image ID.
"""
global iter_num
print("image_id", image_id)
info = self.image_info[image_id]
count = 1 # number of object
img = Image.open(info['mask_path'])
num_obj = self.get_obj_index(img)
mask = np.zeros([info['height'], info['width'], num_obj], dtype=np.uint8)
mask = self.draw_mask(num_obj, mask, img, image_id)
occlusion = np.logical_not(mask[:, :, -1]).astype(np.uint8)
for i in range(count - 2, -1, -1):
mask[:, :, i] = mask[:, :, i] * occlusion
occlusion = np.logical_and(occlusion, np.logical_not(mask[:, :, i]))
labels = []
labels = self.from_yaml_get_class(image_id)
labels_form = []
for i in range(len(labels)):
if labels[i].find("person") != -1:
# print "car"
labels_form.append("person")
elif labels[i].find("leg") != -1:
# print "leg"
labels_form.append("leg")
elif labels[i].find("well") != -1:
# print "well"
labels_form.append("well")
class_ids = np.array([self.class_names.index(s) for s in labels_form])
return mask, class_ids.astype(np.int32)
def get_ax(rows=1, cols=1, size=8):
"""Return a Matplotlib Axes array to be used in
all visualizations in the notebook. Provide a
central point to control graph sizes.
Change the default size attribute to control the size
of rendered images
"""
_, ax = plt.subplots(rows, cols, figsize=(size * cols, size * rows))
return ax
# 基礎設定
dataset_root_path = "samples/trinmy/myinfo/"
img_floder = dataset_root_path + "pic"
mask_floder = dataset_root_path + "cv2_mask"
# yaml_floder = dataset_root_path
imglist = os.listdir(img_floder)
count = len(imglist)
# train與val資料集準備
dataset_train = DrugDataset()
dataset_train.load_shapes(count, img_floder, mask_floder, imglist, dataset_root_path)
dataset_train.prepare()
# print("dataset_train-->",dataset_train._image_ids)
dataset_val = DrugDataset()
dataset_val.load_shapes(count, img_floder, mask_floder, imglist, dataset_root_path)
dataset_val.prepare()
# print("dataset_val-->",dataset_val._image_ids)
# Load and display random samples
# image_ids = np.random.choice(dataset_train.image_ids, 4)
# for image_id in image_ids:
# image = dataset_train.load_image(image_id)
# mask, class_ids = dataset_train.load_mask(image_id)
# visualize.display_top_masks(image, mask, class_ids, dataset_train.class_names)
# Create model in training mode
model = modellib.MaskRCNN(mode="training", config=config,
model_dir=MODEL_DIR)
# Which weights to start with?
init_with = "coco" # imagenet, coco, or last
if init_with == "imagenet":
model.load_weights(model.get_imagenet_weights(), by_name=True)
elif init_with == "coco":
# Load weights trained on MS COCO, but skip layers that
# are different due to the different number of classes
# See README for instructions to download the COCO weights
# print(COCO_MODEL_PATH)
model.load_weights(COCO_MODEL_PATH, by_name=True,
exclude=["mrcnn_class_logits", "mrcnn_bbox_fc",
"mrcnn_bbox", "mrcnn_mask"])
elif init_with == "last":
# Load the last model you trained and continue training
model.load_weights(model.find_last()[1], by_name=True)
# Train the head branches
# Passing layers="heads" freezes all layers except the head
# layers. You can also pass a regular expression to select
# which layers to train by name pattern.
model.train(dataset_train, dataset_val,
learning_rate=config.LEARNING_RATE,
epochs=10,
layers='heads')
# Fine tune all layers
# Passing layers="all" trains all layers. You can also
# pass a regular expression to select which layers to
# train by name pattern.
model.train(dataset_train, dataset_val,
learning_rate=config.LEARNING_RATE / 10,
epochs=10,
layers="all")
1.注意的是要在代碼的33行改成我們之前下載下傳好的預訓練模型所在路徑(一般放在項目根目錄下)。比如這是我的
2.要注意的是第53行中修改我們的類别數，這個類别數是1(背景有的類别)+X(你的類别)
3.在第186行代碼中我們需要改成我們之前制作資料源的路徑(那個路徑下有之前準備好的四個檔案夾)
4.在代碼122行中改一下相應的類别，比如我的是三個類别
在代碼160-169中也改成相應的類别，比如我的是三個類别，照着我這個相應改就可
其他的參數有需要的話可以在45-71行修改
10.運作訓練檔案
python mask_train.py
成功運作的話我們應該是能看到這種的界面
11.預測模型
因為之前的前面的預設設定是每訓練100步儲存一次模型，我們可以到項目根目錄的logs下尋找我們訓練好的模型檔案。
找最下面的檔案夾，我們剛訓練好的模型檔案就在那。
然後這些就是我們生成好的模型檔案了(因為我訓練了400步，是以有四個模型檔案，當然最下面的那個最準了）
然後我們就可以在項目根目錄下建立mask_test.py。代碼是
# -*- coding: utf-8 -*-
import os
import sys
import random
import math
import numpy as np
import skimage.io
import matplotlib
import matplotlib.pyplot as plt
import cv2
import time
from mrcnn.config import Config
from datetime import datetime
# Root directory of the project
ROOT_DIR = os.getcwd()
# Import Mask RCNN
sys.path.append(ROOT_DIR) # To find local version of the library
from mrcnn import utils
import mrcnn.model as modellib
from mrcnn import visualize
# Import COCO config
# sys.path.append(os.path.join(ROOT_DIR, "samples/coco/")) # To find local version
# from samples.coco import coco
# Directory to save logs and trained model
MODEL_DIR = os.path.join(ROOT_DIR, "logs")
# Local path to trained weights file
COCO_MODEL_PATH = os.path.join(MODEL_DIR ,"mask_rcnn_shapes_0010.h5")
# Download COCO trained weights from Releases if needed
if not os.path.exists(COCO_MODEL_PATH):
utils.download_trained_weights(COCO_MODEL_PATH)
print("cuiwei***********************")
# Directory of images to run detection on
IMAGE_DIR = os.path.join(ROOT_DIR, "images")
class ShapesConfig(Config):
"""Configuration for training on the toy shapes dataset.
Derives from the base Config class and overrides values specific
to the toy shapes dataset.
"""
# Give the configuration a recognizable name
NAME = "shapes"
# Train on 1 GPU and 8 images per GPU. We can put multiple images on each
# GPU because the images are small. Batch size is 8 (GPUs * images/GPU).
GPU_COUNT = 1
IMAGES_PER_GPU = 1
# Number of classes (including background)
NUM_CLASSES = 1 + 1 # background + 3 shapes
# Use small images for faster training. Set the limits of the small side
# the large side, and that determines the image shape.
IMAGE_MIN_DIM = 320
IMAGE_MAX_DIM = 384
# Use smaller anchors because our image and objects are small
RPN_ANCHOR_SCALES = (8 * 6, 16 * 6, 32 * 6, 64 * 6, 128 * 6) # anchor side in pixels
# Reduce training ROIs per image because the images are small and have
# few objects. Aim to allow ROI sampling to pick 33% positive ROIs.
TRAIN_ROIS_PER_IMAGE =100
# Use a small epoch since the data is simple
STEPS_PER_EPOCH = 100
# use small validation steps since the epoch is small
VALIDATION_STEPS = 50
#import train_tongue
#class InferenceConfig(coco.CocoConfig):
class InferenceConfig(ShapesConfig):
# Set batch size to 1 since we'll be running inference on
# one image at a time. Batch size = GPU_COUNT * IMAGES_PER_GPU
GPU_COUNT = 1
IMAGES_PER_GPU = 1
config = InferenceConfig()
model = modellib.MaskRCNN(mode="inference", model_dir=MODEL_DIR, config=config)
# Create model object in inference mode.
model = modellib.MaskRCNN(mode="inference", model_dir=MODEL_DIR, config=config)
# Load weights trained on MS-COCO
model.load_weights(COCO_MODEL_PATH, by_name=True)
# COCO Class names
# Index of the class in the list is its ID. For example, to get ID of
# the teddy bear class, use: class_names.index('teddy bear')
class_names = ['BG', 'person']
# Load a random image from the images folder
file_names = next(os.walk(IMAGE_DIR))[2]
image = skimage.io.imread("./images/5951960966_d4e1cda5d0_z.jpg")
a=datetime.now()
# Run detection
results = model.detect([image], verbose=1)
b=datetime.now()
# Visualize results
print("shijian",(b-a).seconds)
r = results[0]
visualize.display_instances(image, r['rois'], r['masks'], r['class_ids'],
class_names, r['scores'])
在運作之前我們需要把代碼的31行，修改模型路徑，把路徑替換成訓練好的模型路徑。
比如我的是這個
第二個需要修改的就是99行中要改成你需要測試圖檔的路徑
第三個的話就是我們需要在107行中将class_names修改成我們的類名
（注意BG不要動，其他改成你增加的類名)
這樣就OK了。
12.結果顯示
模型其實還是不夠準确，是因為我們訓練的時間沒足夠長（其實也跟我用lambelme資料标注的時候沒好好标注的原因有關.....）......不過多訓練會就行了。