分布式機器學習架構：MxNet

     MxNet官網： http://mxnet.readthedocs.io/en/latest/

前言：

caffe是很優秀的dl平台。影響了後面很多相關架構。

cxxnet借鑒了很多caffe的思想。相比之下，cxxnet在實作上更加幹淨，例如依賴很少，通過mshadow的模闆化使得gpu和cpu代碼隻用寫一份，分布式接口也很幹淨。

mxnet是cxxnet的下一代，目前實作了cxxnet所有功能，但借鑒了minerva/torch7/theano，加入更多新的功能。

1. ndarray程式設計接口，類似matlab/numpy.ndarray/torch.tensor。獨有優勢在于通過背後的engine可以在性能上和記憶體使用上更優
2. symbolic接口。這個可以使得快速建構一個神經網絡，和自動求導。
3. 更多binding 目前支援比較好的是python，馬上會有julia和R
4. 更加友善的多卡和多機運作
5. 性能上更優。目前mxnet比cxxnet快40%，而且gpu記憶體使用少了一半。

目前mxnet還在快速發展中。這個月的主要方向有三，更多的binding，更好的文檔，和更多的應用（language model、語音，機器翻譯，視訊）。位址在dmlc/mxnet · GitHub

官方簡介：           

       MXNet is a deep learning framework designed for both efficiency andflexibility.It allows you tomix theflavours of symbolicprogramming and imperative programming to maximize efficiency and productivity.In its core, a dynamic dependency scheduler that automatically parallelizes both symbolic and imperative operations on the fly.A graph optimization layer on top of that makes symbolic execution fast and memory efficient.The library is portable and lightweight, and it scales to multiple GPUs and multiple machines.

       MXNet is also more than a deep learning project. It is also a collection ofblue prints and guidelines for buildingdeep learning system, and interesting insights of DL systems for hackers.

      MxNet混合了符号式設計和指令式設計，來最大化效率和提高産出。其核心是一個動态排程器，不停的并行執行符号和指令操作。頂層的圖優化層使符号執行快速且有效。這個包輕量級可移植性好，并且可以擴充到多GPU和多個機器。

      MxNet不僅是一個深度學習工程，并且是一個為建構DL系統提供藍圖和指導的集合，并且為hackers 提供了一個有趣的視野。

最新發展

What's New

• MXNet Memory Monger, Training Deeper Nets with Sublinear Memory Cost
• Tutorial for NVidia GTC 2016
• Embedding Torch layers and functions in MXNet
• MXNet.js: Javascript Package for Deep Learning in Browser (without server)
• Design Note: Design Efficient Deep Learning Data Loading Module
• MXNet on Mobile Device
• Distributed Training
• Guide to Creating New Operators (Layers)
• Amalgamation and Go Binding for Predictors
• Training Deep Net on 14 Million Images on A Single Machine
• MxNet的記憶體管理：子線性的記憶體代價
• NVIDIA GTC2016上的 教程
• 嵌入 Torch網絡層和函數 到MxNet
• MxNet.js : 可運作到浏覽器中的javascript包
• 設計節點：設計有效的深度學習資料載入模型
• 移動裝置的上的 Mxnet
• 分布式訓練方法
• 網絡層 的運算符重載
• 使用一個深度網絡 訓練1400萬張圖檔

Contents

• Documentation and Tutorials
• Design Notes
• Code Examples
• Installation
• Pretrained Models
• Contribute to MXNet
• Frequent Asked Questions

Features

• Design notes providing useful insights that can re-used by other DL projects
• Flexible configuration for arbitrary computation graph
• Mix and match good flavours of programming to maximize flexibility and efficiency
• Lightweight, memory efficient and portable to smart devices
• Scales up to multi GPUs and distributed setting with auto parallelism
• Support for python, R, C++ and Julia
• Cloud-friendly and directly compatible with S3, HDFS, and Azure

Ask Questions

• Please use mxnet/issues for how to use mxnet and reporting bugs

License

© Contributors, 2015. Licensed under an Apache-2.0 license.

Reference Paper

Tianqi Chen, Mu Li, Yutian Li, Min Lin, Naiyan Wang, Minjie Wang, Tianjun Xiao,Bing Xu, Chiyuan Zhang, and Zheng Zhang.MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systems.In Neural Information Processing Systems, Workshop on Machine Learning Systems, 2015

History

MXNet is initiated and designed in collaboration by the authors of cxxnet, minerva andpurine2. The project reflects what we have learnt from the past projects. It combines important flavours of the existing projects for efficiency, flexibility and memory efficiency.