天天看點

(訓練一)王朗自然保護區---目标檢測資料集介紹 & balancap/ssd訓練介紹1資料集準備2用于訓練

1資料集準備

1 王朗DIY
(訓練一)王朗自然保護區---目标檢測資料集介紹 & balancap/ssd訓練介紹1資料集準備2用于訓練

2 這個資料集要放在ssd目錄下:

/home/hp/z../Tensorflow/Deeplearning/SSD-Tensorflow-master

我們使用的tf架構下的ssd是github上面balancap/SSD-Tensorflow版本: https://github.com/balancap/SSD-Tensorflow

(訓練一)王朗自然保護區---目标檢測資料集介紹 & balancap/ssd訓練介紹1資料集準備2用于訓練

3 關于訓練集的結構如下

Anotations:      自己制作xml

ImageSets:     裡面有三個檔案夾Layout(空檔案夾), Main(自己制作Main), Segmentation(空檔案夾)

       自己制作Main/test.tst, Main/train.txt, Main/trianval.txt, Main/val.txt

JEPGImages: 檔案夾裡面存放有原圖檔檔案.jpg

SegmentationClass: 空檔案夾

SegmentationObject: 空檔案夾

(訓練一)王朗自然保護區---目标檢測資料集介紹 & balancap/ssd訓練介紹1資料集準備2用于訓練

1)其中Annotations檔案裡面存儲的是xml檔案

(使用labelme工具進行标注得到,可以參考https://blog.csdn.net/zjc910997316/article/details/82993420但是這個寫的并不好)

(訓練一)王朗自然保護區---目标檢測資料集介紹 & balancap/ssd訓練介紹1資料集準備2用于訓練

2)ImageSet檔案夾裡面三個檔案夾 Layout(空檔案夾), Main(自己制作Main), Segmentation(空檔案夾)

(訓練一)王朗自然保護區---目标檢測資料集介紹 & balancap/ssd訓練介紹1資料集準備2用于訓練

Main檔案裡面四個.txt檔案Main/test.tst, Main/train.txt, Main/trianval.txt, Main/val.txt

3)JEPGImages檔案夾裡面存放有原圖檔檔案.jpg

(訓練一)王朗自然保護區---目标檢測資料集介紹 & balancap/ssd訓練介紹1資料集準備2用于訓練

4)SegmentationClass: 空檔案夾

(訓練一)王朗自然保護區---目标檢測資料集介紹 & balancap/ssd訓練介紹1資料集準備2用于訓練

5)SegmentationObject: 空檔案夾

(訓練一)王朗自然保護區---目标檢測資料集介紹 & balancap/ssd訓練介紹1資料集準備2用于訓練

2用于訓練

(訓練一)王朗自然保護區---目标檢測資料集介紹 & balancap/ssd訓練介紹1資料集準備2用于訓練

Training

The script train_ssd_network.py  is in charged of training the network.

腳本 train_ssd_network.py 負責網絡教育訓練。

Similarly to TF-Slim models, one can pass numerous options to the training process

(dataset, optimiser, hyper-parameters, model, ...).

與TF-Slim模型類似,可以在訓練過程中傳遞許多選項(資料集,優化器,超參數,模型,…)

In particular, it is possible to provide a checkpoint file which can be use as starting point in order to fine-tune a network.

特别是,可以提供檢查點檔案,可以将其用作微調網絡的起點。

(訓練一)王朗自然保護區---目标檢測資料集介紹 & balancap/ssd訓練介紹1資料集準備2用于訓練

Fine-tuning existing SSD checkpoints

The easiest way to fine the SSD model is to use as pre-trained SSD network (VGG-300 or VGG-512).

For instance, one can fine a model starting from the former as following:

微調現有的SSD檢查點

優化SSD模型最簡單的方法是使用預先訓練的SSD網絡(VGG-300或VGG-512)。

例如,可以從前者開始細化模型如下:

DATASET_DIR=./tfrecords
TRAIN_DIR=./logs/
CHECKPOINT_PATH=./checkpoints/ssd_300_vgg.ckpt
python train_ssd_network.py \
    --train_dir=${TRAIN_DIR} \
    --dataset_dir=${DATASET_DIR} \
    --dataset_name=pascalvoc_2012 \
    --dataset_split_name=train \
    --model_name=ssd_300_vgg \
    --checkpoint_path=${CHECKPOINT_PATH} \
    --save_summaries_secs=60 \
    --save_interval_secs=600 \
    --weight_decay=0.0005 \
    --optimizer=adam \
    --learning_rate=0.001 \
    --batch_size=32
           
(訓練一)王朗自然保護區---目标檢測資料集介紹 & balancap/ssd訓練介紹1資料集準備2用于訓練

Note that in addition to the training script flags, one may also want to experiment with data augmentation parameters

(random cropping, resolution, ...) in

ssd_vgg_preprocessing.py

or/and network parameters (feature layers, anchors boxes, ...) in

ssd_vgg_300/512.py

Furthermore, the training script can be combined with the evaluation routine in order to monitor the performance of saved checkpoints on a validation dataset.

For that purpose, one can pass to training and validation scripts a GPU memory upper limit such that both can run in parallel on the same device.

If some GPU memory is available for the evaluation script, the former can be run in parallel as follows:

在ssd_vgg_preprocessing.py , 注意,除了訓練腳本标志之外,還可能需要試驗資料增強參數 (随機裁剪,分辨率,…)

或者/和 ssd_vgg_300/512.py 中的網絡參數(特性層、錨框等)

此外,可以将訓練腳本與評估例程結合使用,以監視在驗證資料集中儲存的檢查點的性能。

為此,可以将GPU記憶體上限傳遞給訓練和驗證腳本,這樣兩者就可以在同一裝置上并行運作。

如果計算腳本有GPU記憶體可用,則可以并行運作GPU記憶體:

EVAL_DIR=${TRAIN_DIR}/eval
python eval_ssd_network.py \
    --eval_dir=${EVAL_DIR} \
    --dataset_dir=${DATASET_DIR} \
    --dataset_name=pascalvoc_2007 \
    --dataset_split_name=test \
    --model_name=ssd_300_vgg \
    --checkpoint_path=${TRAIN_DIR} \
    --wait_for_checkpoints=True \
    --batch_size=1 \
    --max_num_batches=500
           

(訓練一)王朗自然保護區---目标檢測資料集介紹 & balancap/ssd訓練介紹1資料集準備2用于訓練

Fine-tuning a network trained on ImageNet

One can also try to build a new SSD model based on standard architecture (VGG, ResNet, Inception, ...) and set up on top of it the

multibox

layers (with specific anchors, ratios, ...).

For that purpose, you can fine-tune a network by only loading the weights of the original architecture, and initialize randomly the rest of network.

For instance, in the case of the VGG-16 architecture, one can train a new model as following:

微調一個基于ImageNet的網絡

您還可以嘗試基于标準體系結構(VGG、ResNet、Inception,……)建構一個新的SSD模型,并在其之上設定多盒層(使用特定的錨、比率,……)。

為此,您可以通過隻加載原始體系結構的權重來微調網絡,并随機初始化網絡的其餘部分。

例如,在VGG-16架構的情況下,可以訓練一個新的模型如下:

DATASET_DIR=./tfrecords
TRAIN_DIR=./log/
CHECKPOINT_PATH=./checkpoints/vgg_16.ckpt
python train_ssd_network.py \
    --train_dir=${TRAIN_DIR} \
    --dataset_dir=${DATASET_DIR} \
    --dataset_name=pascalvoc_2007 \
    --dataset_split_name=train \
    --model_name=ssd_300_vgg \
    --checkpoint_path=${CHECKPOINT_PATH} \
    --checkpoint_model_scope=vgg_16 \
    --checkpoint_exclude_scopes=ssd_300_vgg/conv6,ssd_300_vgg/conv7,ssd_300_vgg/block8,ssd_300_vgg/block9,ssd_300_vgg/block10,ssd_300_vgg/block11,ssd_300_vgg/block4_box,ssd_300_vgg/block7_box,ssd_300_vgg/block8_box,ssd_300_vgg/block9_box,ssd_300_vgg/block10_box,ssd_300_vgg/block11_box \
    --trainable_scopes=ssd_300_vgg/conv6,ssd_300_vgg/conv7,ssd_300_vgg/block8,ssd_300_vgg/block9,ssd_300_vgg/block10,ssd_300_vgg/block11,ssd_300_vgg/block4_box,ssd_300_vgg/block7_box,ssd_300_vgg/block8_box,ssd_300_vgg/block9_box,ssd_300_vgg/block10_box,ssd_300_vgg/block11_box \
    --save_summaries_secs=60 \
    --save_interval_secs=600 \
    --weight_decay=0.0005 \
    --optimizer=adam \
    --learning_rate=0.001 \
    --learning_rate_decay_factor=0.94 \
    --batch_size=32
           
(訓練一)王朗自然保護區---目标檢測資料集介紹 & balancap/ssd訓練介紹1資料集準備2用于訓練
Hence, in the former command, the training script randomly initializes the weights belonging to the

checkpoint_exclude_scopes

and load from the checkpoint file

vgg_16.ckpt

the remaining part of the network.

是以,在前一個指令中,訓練腳本随機初始化屬于checkpoint_exclude_scope的權重,并從檢查點檔案vgg_16.ckpt加載網絡的其餘部分。

Note that we also specify with the

trainable_scopes

parameter to first only train the new SSD components and left the rest of VGG network unchanged.

注意,我們還使用 trainable_scope 參數指定,首先隻訓練新的SSD元件,其餘的VGG網絡保持不變。

--------------------------------------------------------------------------------------------------------------------------------------------

Once the network has converged to a good first result (~0.5 mAP for instance), you can fine-tuned the complete network as following:

一旦網絡收斂到良好的第一個結果(例如~0.5 mAP),您可以對整個網絡進行微調,如下所示:

DATASET_DIR=./tfrecords
TRAIN_DIR=./log_finetune/
CHECKPOINT_PATH=./log/model.ckpt-N
python train_ssd_network.py \
    --train_dir=${TRAIN_DIR} \
    --dataset_dir=${DATASET_DIR} \
    --dataset_name=pascalvoc_2007 \
    --dataset_split_name=train \
    --model_name=ssd_300_vgg \
    --checkpoint_path=${CHECKPOINT_PATH} \
    --checkpoint_model_scope=vgg_16 \
    --save_summaries_secs=60 \
    --save_interval_secs=600 \
    --weight_decay=0.0005 \
    --optimizer=adam \
    --learning_rate=0.00001 \
    --learning_rate_decay_factor=0.94 \
    --batch_size=32
           

A number of pre-trained weights of popular deep architectures can be found on TF-Slim models page.

在TF-Slim模型頁面上可以找到一些流行的深度架構的預訓練權重。

繼續閱讀