tfRecord寫入

本文為google inception-v3範例代碼中圖像預處理相關代碼的閱讀筆記

主要包括圖像讀取以及如何将圖檔寫入tfRecord

檔案讀寫相關

從txt_file_name中讀取txt檔案 傳回一個list，list的每個元素為txt中的行元素

tf.gfile.FastGFile(txt_file_name,'r').readlines()
           

從file_path代表的正規表達式中讀取檔案，傳回一個list，list中的每個元素代表一個檔案名路徑

使用tensorflow讀取圖檔

tf.gfile.Glob(file_path)
           

tensorflow簡單圖像處理

(1) 建立tensorflow圖像處理對象

ImageCoder()類的定義：

class ImageCoder(object):
  """Helper class that provides TensorFlow image coding utilities."""

  def __init__(self):
    # Create a single Session to run all image coding calls.
    self._sess = tf.Session()

    # Initializes function that converts PNG to JPEG data.
    self._png_data = tf.placeholder(dtype=tf.string)
    image = tf.image.decode_png(self._png_data, channels=)
    self._png_to_jpeg = tf.image.encode_jpeg(image, format='rgb', quality=)

    # Initializes function that decodes RGB JPEG data.
    self._decode_jpeg_data = tf.placeholder(dtype=tf.string)
    self._decode_jpeg = tf.image.decode_jpeg(self._decode_jpeg_data, channels=)

  def png_to_jpeg(self, image_data):
    return self._sess.run(self._png_to_jpeg,
                          feed_dict={self._png_data: image_data})

  def decode_jpeg(self, image_data):
    image = self._sess.run(self._decode_jpeg,
                           feed_dict={self._decode_jpeg_data: image_data})
    assert len(image.shape) == 
    assert image.shape[] == 
    return image
           

(2)讀取圖像

with tf.gfile.FastGFile(filename,'rb') as f:

    image_data = f.read()
           

tfrecord相關函數

（1）線程控制相關

線程監控器，用來監控是否所有線程已經結束

coord = tf.train.Coordinator()
           

usage:

–1建立線程

–2發起一系列線程，将coordinator傳遞給每個線程

....start thread1...(coord,...)

...start thread N...(coord,...)
           

–3等待線程執行完畢

coord.join(threads)
           

寫tfRecord的流程：

–1 建立一個寫tfRecord的對象：

writer = tf.python_io.TFRecordWriter(dst_file_name)
           

–2 讀取圖檔，并将其轉換成二進制

_process_image源碼

def _process_image(filename, coder):
  """Process a single image file.

  Args:
    filename: string, path to an image file e.g., '/path/to/example.JPG'.
    coder: instance of ImageCoder to provide TensorFlow image coding utils.
  Returns:
    image_buffer: string, JPEG encoding of RGB image.
    height: integer, image height in pixels.
    width: integer, image width in pixels.
  """
  # Read the image file.
  with tf.gfile.FastGFile(filename, 'rb') as f:
    image_data = f.read()

  # Convert any PNG to JPEG's for consistency.
  if _is_png(filename):
    print('Converting PNG to JPEG for %s' % filename)
    image_data = coder.png_to_jpeg(image_data)

  # Decode the RGB JPEG.
  image = coder.decode_jpeg(image_data)

  # Check that image converted to RGB
  assert len(image.shape) == 
  height = image.shape[]
  width = image.shape[]
  assert image.shape[] == 

  return image_data, height, width
           

–3将圖檔轉化為example。example為一張圖檔在tfRecord存儲的一個結構化單元，包含了圖檔内容和基本資訊

_convert_to_example定義：

def _convert_to_example(filename, image_buffer, label, text, height, width):
  """Build an Example proto for an example.
  Args:
    filename: string, path to an image file, e.g., '/path/to/example.JPG'
    image_buffer: string, JPEG encoding of RGB image
    label: integer, identifier for the ground truth for the network
    text: string, unique human-readable, e.g. 'dog'
    height: integer, image height in pixels
    width: integer, image width in pixels
  Returns:
    Example proto
  """

  colorspace = 'RGB'
  channels = 
  image_format = 'JPEG'

  example = tf.train.Example(features=tf.train.Features(feature={
      'image/height': _int64_feature(height),
      'image/width': _int64_feature(width),
      'image/colorspace': _bytes_feature(tf.compat.as_bytes(colorspace)),
      'image/channels': _int64_feature(channels),
      'image/class/label': _int64_feature(label),
      'image/class/text': _bytes_feature(tf.compat.as_bytes(text)),
      'image/format': _bytes_feature(tf.compat.as_bytes(image_format)),
      'image/filename': _bytes_feature(tf.compat.as_bytes(os.path.basename(filename))),
      'image/encoded': _bytes_feature(tf.compat.as_bytes(image_buffer))}))
  return example
           

其中子函數定義：

def _int64_feature(value):
  """Wrapper for inserting int64 features into Example proto."""
  if not isinstance(value, list):
    value = [value]
  return tf.train.Feature(int64_list=tf.train.Int64List(value=value))

def _bytes_feature(value):
  """Wrapper for inserting bytes features into Example proto."""
  return tf.train.Feature(bytes_list=tf.train.BytesList(value=[value]))
           

–4将一個example寫入tfRecord檔案，注意需要将example轉換為字元串流：

writer.write(example.SerializeToString())
           

–5寫完後不要忘記關閉writer對象～

writer.close()