caffe源碼學習（六） 自定義層

經過前面對google protocol buffer、Blob、SyncedMemory 與 shared_ptr、layer、data layer的初步學習，已經能夠仿照caffe中已有的層來寫自定層了。是以接下來就在caffe深度學習架構下寫自定義層。首先應該注意，不同版本的caffe可能會有一些差別，是以要根據官網指南來寫自定義層。為了友善，把目前版本的指南（20160605）貼出來：

Developing new layers

1. Add a class declaration for your layer to

include/caffe/layers/your_layer.hpp

.

(1) Include an inline implementation of

type

overriding the method

virtual inline const char* type() const { return "YourLayerName"; }

replacing

YourLayerName

with your layer’s name.

(2) Implement the

{*}Blobs()

methods to specify blob number requirements; see /caffe/include/caffe/layers.hpp to enforce strict top and bottom Blob counts using the inline

{*}Blobs()

methods.

(3) Omit the

*_gpu

declarations if you’ll only be implementing CPU code.

2. Implement your layer in

src/caffe/layers/your_layer.cpp

.

(1) (optional)

LayerSetUp

for one-time initialization: reading parameters, fixed-size allocations, etc.

(2)

Reshape

for computing the sizes of top blobs, allocating buffers, and any other work that depends on the shapes of bottom blobs

(3)

Forward_cpu

for the function your layer computes

(4)

Backward_cpu

for its gradient (Optional – a layer can be forward-only)

3. (Optional) Implement the GPU versions

Forward_gpu

and

Backward_gpu

in layers/your_layer.cu.

4. If needed, declare parameters in

proto/caffe.proto

, using (and then incrementing) the “next available layer-specific ID” declared in a comment above

message LayerParameter

.

5. Instantiate and register your layer in your cpp file with the macro provided in

layer_factory.hpp

. Assuming that you have a new layer

MyAwesomeLayer

, you can achieve it with the following command:

INSTANTIATE_CLASS(MyAwesomeLayer);
REGISTER_LAYER_CLASS(MyAwesome);
           

6. Note that you should put the registration code in your own cpp file, so your implementation of a layer is self-contained.

7. Optionally, you can also register a Creator if your layer has multiple engines. For an example on how to define a creator function and register it, see

GetConvolutionLayer

in

caffe/layer_factory.cpp

.

8. Write tests in

test/test_your_layer.cpp

. Use

test/test_gradient_check_util.hpp

to check that your Forward and Backward implementations are in numerical agreement.

Forward-Only Layers

If you want to write a layer that you will only ever include in a test net, you do not have to code the backward pass. For example, you might want a layer that measures performance metrics at test time that haven’t already been implemented. Doing this is very simple. You can write an inline implementation of

Backward_cpu

(or

Backward_gpu

) together with the definition of your layer in

include/caffe/your_layer.hpp

that looks like:

virtual void Backward_cpu(const vector<Blob<Dtype>*>& top, const vector<bool>& propagate_down, const vector<Blob<Dtype>*>& bottom) {
  NOT_IMPLEMENTED;
}
           

The

NOT_IMPLEMENTED

macro (defined in

common.hpp

) throws an error log saying “Not implemented yet”. For examples, look at the accuracy layer (

accuracy_layer.hpp

) and threshold layer (

threshold_layer.hpp

) definitions.

官網指南中已經說的比較詳細，下面自定義層主要用到1,2,3,4,5,6中提到的，簡單的實作了

y = x^n

點對點的簡單計算，其中n從prototxt檔案中讀取。7和8目前還沒有實作，後面有需要的話再更新。然後通過matlab接口進行驗證該層的計算是否正确。

還是按照老習慣，先給出源碼，再做總結。

1.源碼

mysquare.hpp(該代碼是仿照caffe中已經實作的

absval_layer

和

dropout_layer

來寫的，是以保留原有的一些内容。最開始要是實作

y = x^2

，接下來為了實作能夠從prototxt中讀取資料，實作

y = x^n

，n從prototxt中讀取，是以命名為mysquare。)

#ifndef MYSQUARE_LAYER_HPP
#define MYSQUARE_LAYER_HPP

#include <vector>

#include "caffe/blob.hpp"
#include "caffe/layer.hpp"
#include "caffe/proto/caffe.pb.h"

#include "caffe/layers/neuron_layer.hpp"

namespace caffe {

/**
 * @brief Computes @f$ y = x^2 @f$
 *
 * @param bottom input Blob vector (length 1)
 *   -# @f$ (N \times C \times H \times W) @f$
 *      the inputs @f$ x @f$
 * @param top output Blob vector (length 1)
 *   -# @f$ (N \times C \times H \times W) @f$
 *      the computed outputs @f$ y = x^2 @f$
 */
template <typename Dtype>
class MySquareLayer : public NeuronLayer<Dtype> {
 public:
  explicit MySquareLayer(const LayerParameter& param)
      : NeuronLayer<Dtype>(param) {}
  virtual void LayerSetUp(const vector<Blob<Dtype>*>& bottom,
      const vector<Blob<Dtype>*>& top);

  virtual inline const char* type() const { return "MySquare"; }
  virtual inline int ExactNumBottomBlobs() const { return ; }
  virtual inline int ExactNumTopBlobs() const { return ; }

 protected:
  /// @copydoc MySquareLayer
  virtual void Forward_cpu(const vector<Blob<Dtype>*>& bottom,
      const vector<Blob<Dtype>*>& top);
  virtual void Forward_gpu(const vector<Blob<Dtype>*>& bottom,
      const vector<Blob<Dtype>*>& top);

  /**
   * @brief Computes the error gradient w.r.t. the absolute value inputs.
   *
   * @param top output Blob vector (length 1), providing the error gradient with
   *      respect to the outputs
   *   -# @f$ (N \times C \times H \times W) @f$
   *      containing error gradients @f$ \frac{\partial E}{\partial y} @f$
   *      with respect to computed outputs @f$ y @f$
   * @param propagate_down see Layer::Backward.
   * @param bottom input Blob vector (length 2)
   *   -# @f$ (N \times C \times H \times W) @f$
   *      the inputs @f$ x @f$; Backward fills their diff with
   *      gradients @f$
   *        \frac{\partial E}{\partial x} =
   *            \mathrm{sign}(x) \frac{\partial E}{\partial y}
   *      @f$ if propagate_down[0]
   */
 virtual void Backward_cpu(const vector<Blob<Dtype>*>& top,
      const vector<bool>& propagate_down, const vector<Blob<Dtype>*>& bottom);
 virtual void Backward_gpu(const vector<Blob<Dtype>*>& top,
      const vector<bool>& propagate_down, const vector<Blob<Dtype>*>& bottom);

 float power_;
};

}  // namespace caffe

#endif // MYSQUARE_LAYER_HPP
           

mysquare_layer.cpp

#include <vector>

#include "caffe/layers/mysquare_layer.hpp"
#include "caffe/util/math_functions.hpp"

namespace caffe {

template <typename Dtype>
void MySquareLayer<Dtype>::LayerSetUp(const vector<Blob<Dtype>*>& bottom,
      const vector<Blob<Dtype>*>& top) {
  NeuronLayer<Dtype>::LayerSetUp(bottom, top);
  CHECK_NE(top[], bottom[]) << this->type() << " Layer does not "
    "allow in-place computation.";
  power_ = this->layer_param_.mysquare_param().power();
}

template <typename Dtype>
void MySquareLayer<Dtype>::Forward_cpu(
    const vector<Blob<Dtype>*>& bottom, const vector<Blob<Dtype>*>& top) {
  const int count = top[]->count();
  Dtype* top_data = top[]->mutable_cpu_data();
  caffe_powx(count, bottom[]->cpu_data(), Dtype(power_), top_data);
}

template <typename Dtype>
void MySquareLayer<Dtype>::Backward_cpu(const vector<Blob<Dtype>*>& top,
    const vector<bool>& propagate_down, const vector<Blob<Dtype>*>& bottom) {
  const int count = top[]->count();
  const Dtype* top_diff = top[]->cpu_diff();
  if (propagate_down[]) {
    const Dtype* bottom_data = bottom[]->cpu_data();
    Dtype* bottom_diff = bottom[]->mutable_cpu_diff();
    caffe_powx(count, bottom_data, Dtype(power_ - ), bottom_diff);
    caffe_scal(count, Dtype(power_), bottom_diff);
    caffe_mul(count, bottom_diff, top_diff, bottom_diff);
  }
}

#ifdef CPU_ONLY
STUB_GPU(MySquareLayer);
#endif

INSTANTIATE_CLASS(MySquareLayer);
REGISTER_LAYER_CLASS(MySquare);

}  // namespace caffe
           

mysquare_layer.cu

#include <vector>

#include "caffe/layers/mysquare_layer.hpp"
#include "caffe/util/math_functions.hpp"
#include <iostream>
using namespace std;

namespace caffe {

template <typename Dtype>
void MySquareLayer<Dtype>::Forward_gpu(
    const vector<Blob<Dtype>*>& bottom, const vector<Blob<Dtype>*>& top) {
  const int count = top[]->count();
  Dtype* top_data = top[]->mutable_gpu_data();
  caffe_gpu_powx(count, bottom[]->gpu_data(), Dtype(power_), top_data);
}

template <typename Dtype>
void MySquareLayer<Dtype>::Backward_gpu(const vector<Blob<Dtype>*>& top,
    const vector<bool>& propagate_down, const vector<Blob<Dtype>*>& bottom) {
  const int count = top[]->count();
  const Dtype* top_diff = top[]->gpu_diff();
  if (propagate_down[]) {
    const Dtype* bottom_data = bottom[]->gpu_data();
    Dtype* bottom_diff = bottom[]->mutable_gpu_diff();
    const Dtype* bottom_data_w = bottom[]->cpu_data();
    const Dtype* bottom_diff_w = bottom[]->cpu_diff();

    cout << "bottom_data[0]: " << bottom_data_w[] << endl;
    cout << "bottom_diff[0]: " << bottom_diff_w[] << endl;

    caffe_gpu_powx(count, bottom_data, Dtype(power_ - ), bottom_diff);

    bottom_diff = bottom[]->mutable_gpu_diff();
    bottom_data_w = bottom[]->cpu_data();
    bottom_diff_w = bottom[]->cpu_diff();
    cout << "bottom_data[0]: " << bottom_data_w[] << endl;
    cout << "bottom_diff[0]: " << bottom_diff_w[] << endl;

    caffe_gpu_scal(count, Dtype(power_), bottom_diff);

    bottom_diff = bottom[]->mutable_gpu_diff();
    bottom_data_w = bottom[]->cpu_data();
    bottom_diff_w = bottom[]->cpu_diff();
    cout << "bottom_data[0]: " << bottom_data_w[] << endl;
    cout << "bottom_diff[0]: " << bottom_diff_w[] << endl;

    caffe_gpu_mul(count, bottom_diff, top_diff, bottom_diff);

    bottom_diff = bottom[]->mutable_gpu_diff();
    bottom_data_w = bottom[]->cpu_data();
    bottom_diff_w = bottom[]->cpu_diff();
    cout << "bottom_data[0]: " << bottom_data_w[] << endl;
    cout << "bottom_diff[0]: " << bottom_diff_w[] << endl;
  }
}

INSTANTIATE_LAYER_GPU_FUNCS(MySquareLayer);


}  // namespace caffe
           

caffe.proto

message LayerParameter{
...
++ optional MySquareParameter mysquare_param = ;
...
}
...
++ message MySquareParameter {
++  optional float power =  [default = ];
++ }
...
message V1LayerParameter{
...
++ MYSQUARE = ;
...
}
           

deploy.prototxt

name: "CaffeNet"
layer {
  name: "data"
  type: "Input"
  top: "data"
  input_param { shape: { dim:  dim:  dim:  dim:  } }
}

layer {
  name: "mysquare"
  type: "MySquare"
  bottom: "data"
  top: "mysquare_out"
  mysquare_param {
     power: 
  }
}
           

test.m

% 在caffe_root/matlab下
im = imread('../examples/images/cat.jpg');
input_data = {prepare_image(im)};
input_data = {pp(im)};
m = 'test/deploy.prototxt'
caffe.set_mode_gpu();
caffe.set_device();
net = caffe.Net(m,'test')
out = net.forward(input_data);
ii = input_data{,};
oo = out{,};
% 簡單觀察一下計算是否正确
ii(:,:,,)
oo(:,:,,)
(-)^
           

pp.m

% 在caffe_root/matlab下
% 下面是classification_demo.m中的一個函數。

function crops_data = pp(im)
% ------------------------------------------------------------------------
% caffe/matlab/+caffe/imagenet/ilsvrc_2012_mean.mat contains mean_data that
% is already in W x H x C with BGR channels
d = load('../+caffe/imagenet/ilsvrc_2012_mean.mat');
mean_data = d.mean_data;
IMAGE_DIM = ;
CROPPED_DIM = ;

% Convert an image returned by Matlab's imread to im_data in caffe's data
% format: W x H x C with BGR channels
im_data = im(:, :, [, , ]);  % permute channels from RGB to BGR
im_data = permute(im_data, [, , ]);  % flip width and height
im_data = single(im_data);  % convert from uint8 to single
im_data = imresize(im_data, [IMAGE_DIM IMAGE_DIM], 'bilinear');  % resize im_data
im_data = im_data - mean_data;  % subtract mean_data (already in W x H x C, BGR)

% oversample (4 corners, center, and their x-axis flips)
crops_data = zeros(CROPPED_DIM, CROPPED_DIM, , , 'single');
indices = [ IMAGE_DIM-CROPPED_DIM] + ;
n = ;
for i = indices
  for j = indices
    crops_data(:, :, :, n) = im_data(i:i+CROPPED_DIM-, j:j+CROPPED_DIM-, :);
    crops_data(:, :, :, n+) = crops_data(end:-:, :, :, n);
    n = n + ;
  end
end
center = floor(indices() / ) + ;
crops_data(:,:,:,) = ...
  im_data(center:center+CROPPED_DIM-,center:center+CROPPED_DIM-,:);
crops_data(:,:,:,) = crops_data(end:-:, :, :, );
           

2.總結

郁悶啊，馬上總結完了，不小心弄沒了，以後有時間再更新吧。。。