【连载】深度学习笔记12：卷积神经网络的Tensorflow实现
在上一讲中，我们学习了如何利用 numpy 手动搭建卷积神经网络。但在实际的图像识别中，使用 numpy 去手写 cnn 未免有些吃力不讨好。在 dnn 的学习中，我们也是在手动搭建之后利用 tensorflow 去重新实现一遍，一来为了能够对神经网络的传播机制能够理解更加透彻，二来也是为了更加高效使用开源框架快速搭建起深度学习项目。本节就继续和大家一起学习如何利用 tensorflow 搭建一个卷积神经网络。
我们继续以 ng 课题组提供的 sign 手势数据集为例，学习如何通过 tensorflow 快速搭建起一个深度学习项目。数据集标签共有零到五总共 6 类标签，示例如下：
先对数据进行简单的预处理并查看训练集和测试集维度：
x_train = x_train_orig/255.
x_test = x_test_orig/255.
y_train = convert_to_one_hot(y_train_orig, 6).ty_test = convert_to_one_hot(y_test_orig, 6).t
print (number of training examples = + str(x_train.shape[0]))
print (number of test examples = + str(x_test.shape[0]))
print (x_train shape: + str(x_train.shape))
print (y_train shape: + str(y_train.shape))
print (x_test shape: + str(x_test.shape))
print (y_test shape: + str(y_test.shape))
可见我们总共有 1080 张 64643 训练集图像，120 张 64643 的测试集图像，共有 6 类标签。下面我们开始搭建过程。
创建 placeholder 首先需要为训练集预测变量和目标变量创建占位符变量 placeholder ，定义创建占位符变量函数：
def create_placeholders(n_h0, n_w0, n_c0, n_y):
creates the placeholders for the tensorflow session. arguments: n_h0 -- scalar, height of an input image n_w0 -- scalar, width of an input image n_c0 -- scalar, number of channels of the input n_y -- scalar, number of classes returns: x -- placeholder for the data input, of shape [none, n_h0, n_w0, n_c0] and dtype float y -- placeholder for the input labels, of shape [none, n_y] and dtype float x = tf.placeholder(tf.float32, shape=(none, n_h0, n_w0, n_c0), name='x') y = tf.placeholder(tf.float32, shape=(none, n_y), name='y')
return x, y参数初始化 然后需要对滤波器权值参数进行初始化：
def initialize_parameters():
initializes weight parameters to build a neural network with tensorflow. returns: parameters -- a dictionary of tensors containing w1, w2 tf.set_random_seed(1) w1 = tf.get_variable(w1, [4,4,3,8], initializer = tf.contrib.layers.xavier_initializer(seed = 0)) w2 = tf.get_variable(w2, [2,2,8,16], initializer = tf.contrib.layers.xavier_initializer(seed = 0)) parameters = {w1: w1,
w2: w2}
return parameters执行卷积网络的前向传播过程
前向传播过程如下所示：
conv2d -> relu -> maxpool -> conv2d -> relu -> maxpool -> flatten -> fullyconnected
可见我们要搭建的是一个典型的 cnn 过程，经过两次的卷积-relu激活-最大池化，然后展开接上一个全连接层。利用 tensorflow 搭建上述传播过程如下：
def forward_propagation(x, parameters):
implements the forward propagation for the model arguments: x -- input dataset placeholder, of shape (input size, number of examples) parameters -- python dictionary containing your parameters w1, w2 the shapes are given in initialize_parameters returns: z3 -- the output of the last linear unit # retrieve the parameters from the dictionary parameters w1 = parameters['w1'] w2 = parameters['w2']
# conv2d: stride of 1, padding 'same' z1 = tf.nn.conv2d(x,w1, strides = [1,1,1,1], padding = 'same')
# relu a1 = tf.nn.relu(z1)
# maxpool: window 8x8, sride 8, padding 'same' p1 = tf.nn.max_pool(a1, ksize = [1,8,8,1], strides = [1,8,8,1], padding = 'same')
# conv2d: filters w2, stride 1, padding 'same' z2 = tf.nn.conv2d(p1,w2, strides = [1,1,1,1], padding = 'same')
# relu a2 = tf.nn.relu(z2)
# maxpool: window 4x4, stride 4, padding 'same' p2 = tf.nn.max_pool(a2, ksize = [1,4,4,1], strides = [1,4,4,1], padding = 'same')
# flatten p2 = tf.contrib.layers.flatten(p2) z3 = tf.contrib.layers.fully_connected(p2, 6, activation_fn = none)
return z3计算当前损失 在 tensorflow 中计算损失函数非常简单，一行代码即可：
def compute_cost(z3, y):
computes the cost arguments: z3 -- output of forward propagation (output of the last linear unit), of shape (6, number of examples) y -- true labels vector placeholder, same shape as z3 returns: cost - tensor of the cost function cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits=z3, labels=y))
return cost 定义好上述过程之后，就可以封装整体的训练过程模型。可能你会问为什么没有反向传播，这里需要注意的是 tensorflow 帮助我们自动封装好了反向传播过程，无需我们再次定义，在实际搭建过程中我们只需将前向传播的网络结构定义清楚即可。
封装模型def model(x_train, y_train, x_test, y_test, learning_rate = 0.009, num_epochs = 100, minibatch_size = 64, print_cost = true):
implements a three-layer convnet in tensorflow: conv2d -> relu -> maxpool -> conv2d -> relu -> maxpool -> flatten -> fullyconnected arguments: x_train -- training set, of shape (none, 64, 64, 3) y_train -- test set, of shape (none, n_y = 6) x_test -- training set, of shape (none, 64, 64, 3) y_test -- test set, of shape (none, n_y = 6) learning_rate -- learning rate of the optimization num_epochs -- number of epochs of the optimization loop minibatch_size -- size of a minibatch print_cost -- true to print the cost every 100 epochs returns: train_accuracy -- real number, accuracy on the train set (x_train) test_accuracy -- real number, testing accuracy on the test set (x_test) parameters -- parameters learnt by the model. they can then be used to predict. ops.reset_default_graph() tf.set_random_seed(1) seed = 3 (m, n_h0, n_w0, n_c0) = x_train.shape n_y = y_train.shape[1] costs = [] # create placeholders of the correct shape x, y = create_placeholders(n_h0, n_w0, n_c0, n_y)
# initialize parameters parameters = initialize_parameters()
# forward propagation z3 = forward_propagation(x, parameters)
# cost function cost = compute_cost(z3, y)
# backpropagation optimizer = tf.train.adamoptimizer(learning_rate = learning_rate).minimize(cost) # initialize all the variables globally init = tf.global_variables_initializer()
# start the session to compute the tensorflow graph with tf.session() as sess:
# run the initialization sess.run(init)
# do the training loop for epoch in range(num_epochs): minibatch_cost = 0. num_minibatches = int(m / minibatch_size) seed = seed + 1 minibatches = random_mini_batches(x_train, y_train, minibatch_size, seed)
for minibatch in minibatches:
# select a minibatch (minibatch_x, minibatch_y) = minibatch _ , temp_cost = sess.run([optimizer, cost], feed_dict={x: minibatch_x, y: minibatch_y}) minibatch_cost += temp_cost / num_minibatches
# print the cost every epoch if print_cost == true and epoch % 5 == 0:
print (cost after epoch %i: %f % (epoch, minibatch_cost))
if print_cost == true and epoch % 1 == 0: costs.append(minibatch_cost)
# plot the cost plt.plot(np.squeeze(costs)) plt.ylabel('cost') plt.xlabel('iterations (per tens)') plt.title(learning rate = + str(learning_rate)) plt.show() # calculate the correct predictions predict_op = tf.argmax(z3, 1) correct_prediction = tf.equal(predict_op, tf.argmax(y, 1))
# calculate accuracy on the test set accuracy = tf.reduce_mean(tf.cast(correct_prediction, float)) print(accuracy) train_accuracy = accuracy.eval({x: x_train, y: y_train}) test_accuracy = accuracy.eval({x: x_test, y: y_test}) print(train accuracy:, train_accuracy) print(test accuracy:, test_accuracy)
return train_accuracy, test_accuracy, parameters 对训练集执行模型训练：
_, _, parameters = model(x_train, y_train, x_test, y_test) 训练迭代过程如下：
我们在训练集上取得了 0.67 的准确率，在测试集上的预测准确率为 0.58 ，虽然效果并不显著，模型也有待深度调优，但我们已经学会了如何用 tensorflow 快速搭建起一个深度学习系统了。
武汉凯迪正大高压漏电起痕试验仪技术规范书
2020年“古镇杯”中国国际照明灯具设计大赛获奖候选名单公示
半导体交易增加，日本超中国成韩国第二大贸易逆差国
无人机电子速度控制器的设计注意事项
意法半导体推出STM32微控制器专用先进电机控制算法，扩充性
【连载】深度学习笔记12：卷积神经网络的Tensorflow实现
关于无人机的性能分析和介绍
iQOO Neo 855版将于10月24日发布配备了1080P屏幕和8GB+256GB内存组合
罗氏线圈电流探头该如何选型
萨顿科普了强化学习、深度强化学习，并谈到了这项技术的潜力和发展方向
传感器未来的发展方向将会是工业控制领域
G.723标准数字录音系统设计
Nikola Motor Company的汽车公司起诉特斯拉，赔偿20亿美元
一线大牌激光电视的横评对比，且看有你喜欢的那一款吗？
人形机器人落地的解题方法
AcrelCloud-1000变电所运维云平台系统研究与应用
iOS 11重磅加持！FaceTime终于迎来大更新
小芯片如何为定制半导体设计提供前进的道路?
北京InfoComm China 2019隆重开幕，各种LED屏鲜丽夺目
大疆在美国被控侵权，疑是美方对中国科技公司的制裁
我们继续以 ng 课题组提供的 sign 手势数据集为例，学习如何通过 tensorflow 快速搭建起一个深度学习项目。数据集标签共有零到五总共 6 类标签，示例如下：
先对数据进行简单的预处理并查看训练集和测试集维度：
x_train = x_train_orig/255.
x_test = x_test_orig/255.
y_train = convert_to_one_hot(y_train_orig, 6).ty_test = convert_to_one_hot(y_test_orig, 6).t
print (number of training examples = + str(x_train.shape[0]))
print (number of test examples = + str(x_test.shape[0]))
print (x_train shape: + str(x_train.shape))
print (y_train shape: + str(y_train.shape))
print (x_test shape: + str(x_test.shape))
print (y_test shape: + str(y_test.shape))
可见我们总共有 1080 张 64643 训练集图像，120 张 64643 的测试集图像，共有 6 类标签。下面我们开始搭建过程。
创建 placeholder 首先需要为训练集预测变量和目标变量创建占位符变量 placeholder ，定义创建占位符变量函数：
def create_placeholders(n_h0, n_w0, n_c0, n_y):
creates the placeholders for the tensorflow session. arguments: n_h0 -- scalar, height of an input image n_w0 -- scalar, width of an input image n_c0 -- scalar, number of channels of the input n_y -- scalar, number of classes returns: x -- placeholder for the data input, of shape [none, n_h0, n_w0, n_c0] and dtype float y -- placeholder for the input labels, of shape [none, n_y] and dtype float x = tf.placeholder(tf.float32, shape=(none, n_h0, n_w0, n_c0), name='x') y = tf.placeholder(tf.float32, shape=(none, n_y), name='y')
return x, y参数初始化 然后需要对滤波器权值参数进行初始化：
def initialize_parameters():
initializes weight parameters to build a neural network with tensorflow. returns: parameters -- a dictionary of tensors containing w1, w2 tf.set_random_seed(1) w1 = tf.get_variable(w1, [4,4,3,8], initializer = tf.contrib.layers.xavier_initializer(seed = 0)) w2 = tf.get_variable(w2, [2,2,8,16], initializer = tf.contrib.layers.xavier_initializer(seed = 0)) parameters = {w1: w1,
w2: w2}
return parameters执行卷积网络的前向传播过程
前向传播过程如下所示：
conv2d -> relu -> maxpool -> conv2d -> relu -> maxpool -> flatten -> fullyconnected
可见我们要搭建的是一个典型的 cnn 过程，经过两次的卷积-relu激活-最大池化，然后展开接上一个全连接层。利用 tensorflow 搭建上述传播过程如下：
def forward_propagation(x, parameters):
implements the forward propagation for the model arguments: x -- input dataset placeholder, of shape (input size, number of examples) parameters -- python dictionary containing your parameters w1, w2 the shapes are given in initialize_parameters returns: z3 -- the output of the last linear unit # retrieve the parameters from the dictionary parameters w1 = parameters['w1'] w2 = parameters['w2']
# conv2d: stride of 1, padding 'same' z1 = tf.nn.conv2d(x,w1, strides = [1,1,1,1], padding = 'same')
# relu a1 = tf.nn.relu(z1)
# maxpool: window 8x8, sride 8, padding 'same' p1 = tf.nn.max_pool(a1, ksize = [1,8,8,1], strides = [1,8,8,1], padding = 'same')
# conv2d: filters w2, stride 1, padding 'same' z2 = tf.nn.conv2d(p1,w2, strides = [1,1,1,1], padding = 'same')
# relu a2 = tf.nn.relu(z2)
# maxpool: window 4x4, stride 4, padding 'same' p2 = tf.nn.max_pool(a2, ksize = [1,4,4,1], strides = [1,4,4,1], padding = 'same')
# flatten p2 = tf.contrib.layers.flatten(p2) z3 = tf.contrib.layers.fully_connected(p2, 6, activation_fn = none)
return z3计算当前损失 在 tensorflow 中计算损失函数非常简单，一行代码即可：
def compute_cost(z3, y):
computes the cost arguments: z3 -- output of forward propagation (output of the last linear unit), of shape (6, number of examples) y -- true labels vector placeholder, same shape as z3 returns: cost - tensor of the cost function cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits=z3, labels=y))
return cost 定义好上述过程之后，就可以封装整体的训练过程模型。可能你会问为什么没有反向传播，这里需要注意的是 tensorflow 帮助我们自动封装好了反向传播过程，无需我们再次定义，在实际搭建过程中我们只需将前向传播的网络结构定义清楚即可。
封装模型def model(x_train, y_train, x_test, y_test, learning_rate = 0.009, num_epochs = 100, minibatch_size = 64, print_cost = true):
implements a three-layer convnet in tensorflow: conv2d -> relu -> maxpool -> conv2d -> relu -> maxpool -> flatten -> fullyconnected arguments: x_train -- training set, of shape (none, 64, 64, 3) y_train -- test set, of shape (none, n_y = 6) x_test -- training set, of shape (none, 64, 64, 3) y_test -- test set, of shape (none, n_y = 6) learning_rate -- learning rate of the optimization num_epochs -- number of epochs of the optimization loop minibatch_size -- size of a minibatch print_cost -- true to print the cost every 100 epochs returns: train_accuracy -- real number, accuracy on the train set (x_train) test_accuracy -- real number, testing accuracy on the test set (x_test) parameters -- parameters learnt by the model. they can then be used to predict. ops.reset_default_graph() tf.set_random_seed(1) seed = 3 (m, n_h0, n_w0, n_c0) = x_train.shape n_y = y_train.shape[1] costs = [] # create placeholders of the correct shape x, y = create_placeholders(n_h0, n_w0, n_c0, n_y)
# initialize parameters parameters = initialize_parameters()
# forward propagation z3 = forward_propagation(x, parameters)
# cost function cost = compute_cost(z3, y)
# backpropagation optimizer = tf.train.adamoptimizer(learning_rate = learning_rate).minimize(cost) # initialize all the variables globally init = tf.global_variables_initializer()
# start the session to compute the tensorflow graph with tf.session() as sess:
# run the initialization sess.run(init)
# do the training loop for epoch in range(num_epochs): minibatch_cost = 0. num_minibatches = int(m / minibatch_size) seed = seed + 1 minibatches = random_mini_batches(x_train, y_train, minibatch_size, seed)
for minibatch in minibatches:
# select a minibatch (minibatch_x, minibatch_y) = minibatch _ , temp_cost = sess.run([optimizer, cost], feed_dict={x: minibatch_x, y: minibatch_y}) minibatch_cost += temp_cost / num_minibatches
# print the cost every epoch if print_cost == true and epoch % 5 == 0:
print (cost after epoch %i: %f % (epoch, minibatch_cost))
if print_cost == true and epoch % 1 == 0: costs.append(minibatch_cost)
# plot the cost plt.plot(np.squeeze(costs)) plt.ylabel('cost') plt.xlabel('iterations (per tens)') plt.title(learning rate = + str(learning_rate)) plt.show() # calculate the correct predictions predict_op = tf.argmax(z3, 1) correct_prediction = tf.equal(predict_op, tf.argmax(y, 1))
# calculate accuracy on the test set accuracy = tf.reduce_mean(tf.cast(correct_prediction, float)) print(accuracy) train_accuracy = accuracy.eval({x: x_train, y: y_train}) test_accuracy = accuracy.eval({x: x_test, y: y_test}) print(train accuracy:, train_accuracy) print(test accuracy:, test_accuracy)
return train_accuracy, test_accuracy, parameters 对训练集执行模型训练：
_, _, parameters = model(x_train, y_train, x_test, y_test) 训练迭代过程如下：
我们在训练集上取得了 0.67 的准确率，在测试集上的预测准确率为 0.58 ，虽然效果并不显著，模型也有待深度调优，但我们已经学会了如何用 tensorflow 快速搭建起一个深度学习系统了。
武汉凯迪正大高压漏电起痕试验仪技术规范书
2020年“古镇杯”中国国际照明灯具设计大赛获奖候选名单公示
半导体交易增加，日本超中国成韩国第二大贸易逆差国
无人机电子速度控制器的设计注意事项
意法半导体推出STM32微控制器专用先进电机控制算法，扩充性
【连载】深度学习笔记12：卷积神经网络的Tensorflow实现
关于无人机的性能分析和介绍
iQOO Neo 855版将于10月24日发布配备了1080P屏幕和8GB+256GB内存组合
罗氏线圈电流探头该如何选型
萨顿科普了强化学习、深度强化学习，并谈到了这项技术的潜力和发展方向
传感器未来的发展方向将会是工业控制领域
G.723标准数字录音系统设计
Nikola Motor Company的汽车公司起诉特斯拉，赔偿20亿美元
一线大牌激光电视的横评对比，且看有你喜欢的那一款吗？
人形机器人落地的解题方法
AcrelCloud-1000变电所运维云平台系统研究与应用
iOS 11重磅加持！FaceTime终于迎来大更新
小芯片如何为定制半导体设计提供前进的道路?
北京InfoComm China 2019隆重开幕，各种LED屏鲜丽夺目
大疆在美国被控侵权，疑是美方对中国科技公司的制裁