Welcome to Part 4 of our tutorial where we will be focused on training the neural network we built in the previous section
Before we get started training, we need to understand the overall flow of developing neural networks and how this relates to our hardware. Neural networks have two distinct stages:
In fact, the reduction in memory usage is one of the main reasons why FPGAs are so good at the inference phase. FPGAs have always suffered from memory bandwidth issues, but if we can minimise the amount of memory transfers then we can overcome this weakness. We can split the two phases by hardware:
Now that we have our CNN in place, we need to train it. main.py handles both the building and training of our neural network. As before launch the docker image and run main.py:
This will run our training. We use the following settings to compile our network:
LEARN_RATE=0.0001 DECAY_RATE=1e-6 ...
model.compile(loss='categorical_crossentropy', optimizer=optimizers.Adam(lr=LEARN_RATE, decay=DECAY_RATE), metrics=['accuracy'])
BATCHSIZE=32 EPOCHS=3 ...
model.fit(train_data, train_label, batch_size=BATCHSIZE, shuffle=True, epochs=EPOCHS, validation_data=(val_data, val_label) )
For each training phase we do, we need to set some parameters:
Once we are done training, we need to evaluate the accuracy on the testing data:
scores = model.evaluate(testing_data, testing_label, batch_size=BATCHSIZE)
Train on 23455 samples, validate on 4000 samples Epoch 1/3 23455/23455 [==============================] - 15s 631us/step - loss: 1.3695 - acc: 0.6072 - val_loss: 0.1170 - val_acc: 0.9862 Epoch 2/3 23455/23455 [==============================] - 14s 599us/step - loss: 0.2543 - acc: 0.9164 - val_loss: 0.0170 - val_acc: 0.9998 Epoch 3/3 23455/23455 [==============================] - 14s 590us/step - loss: 0.0929 - acc: 0.9720 - val_loss: 0.0034 - val_acc: 1.0000 7172/7172 [==============================] - 1s 99us/step Loss: 0.238 Accuracy: 0.922