Broadcasting
■ expand
■ without copying data
■ VS tf.tile
■ tf. broadcast to
key idea
■ Insert 1 dim ahead if needed
■ Expand dims with size 1 to same size
Feature maps: [4, 32, 32, 3]
Bias: [3]→[1,1,1, 32]→[4, 32, 32, 3]
//即wx+b中b的偏置
注意*********
[1,3]
[1,4]
這兩者不可以進行broadcast！！！
從右到左最右的資料3,4不一緻，最右資料叫小次元，最左叫做大次元
Why broadcasting?
1、for real demanding
- [classes, students, scores]
-
Add bias for every student: + 5 score
. [4,32,8] + [4, 32, 8]
■[4, 32, 8] + [5.0]
2、memory consumption
- [4,32, 8]→1024
-
bias=[8]: [5.0,5.0,5.0…]→8
(節省記憶體，1024 * 4/8 * 4=2^7)
Situation 1:（√）
[4, 32, 14, 14]
[1, 32, 1,1]→[4, 32, 14, 14]
Situation 2（√）
[4, 32, 14, 14]
[14, 14]→[1, 1,14, 14]→[4, 32, 14, 14]
Situation 3（×）
[4, 32, 14, 14]
[2, 32, 14, 14]
- Dim 0 has dim, can NOT insert and expand to same
- Dim 0 has distinct dim, NOT size 1
- NOT broadcasting-able
eg：
In [25]: x=tf.random. normal([4,32,32,3])
In [27]: (x+tf.r andom. normal([3])). shape
Out[27]: Tensor Shape([4, 32， 32, 3])
In [28]: (x+tf.r andom. normal([32,32, 1])). shape
Out[28]: Tensor Shape([4, 32，32,
3])
In [29]: (x+tf.r andom. normal([4,1,1, 1])). shape
Out[29]: Tensor Shape([4, 32，32，3])
In [31]: (x+tf.random. normal([1,4,1, 1])). shape
InvalidAr gumentError: Incompatible shapes: [4,32,32,3] Vs. [1,4,1,1] [Op:Add]
name: add/
tf. broadcast_ to
In [37]: b=tf. broadcast_ to(tf. random. normal([4,1,1,1])，[4,32,32,3])
In [38]: b. shape
Out[38]: Tensor. Shape([4, 32, 32，3])
