Evaluation of Deep Neural Networks Training Assessment 2

Assignment Task

Introduction

In this assessment, you will experiment with densely connected deep neural network classifiers using a standard data set, Fashion MNIST. This is a dataset of small images of ten different categories of clothing items. You will build a network, fit this to data, optimise hyper- parameters and evaluate the results obtained by the best classifier. These are the same processes that you would go through in practice whenever you use deep learning.

Questions about this assessment can be posted to the Assessment 2 Discussion Board and you should consult the assessment rubric when preparing your submission.

Purpose

This assessment will provide you with the opportunity to:

• apply densely connected deep neural networks to an image classification problem
• develop practical skills in Keras/python for building and training deep learning networks
• apply knowledge of hyper-parameter optimisation to a practical problem
• demonstrate informed analysis of performance and

Your tasks

This assessment is divided into six parts:

1. Load and inspect the data

2. Build a network

3. Train the network

4. Network architecture comparison

5. Hyper-parameter optimisation

6. Final

Directions

1. Load and inspect the data

We will use the Fashion MNIST dataset (built into Keras/TF), which are all small images of different clothing items. They can be found in keras.datasets.fashion_mnist

Description

• Load the dataset and display the size of the various parts.
• Display, graphically, one example image of each class.
• Display summary stats/info for class labels.
• Split the non-test labels and images into training and validation datasets according to an 80/20 split.
• Do appropriate pre-processing of the images and/or labels.

2. Build a network

Write a function that creates a Keras model of a densely connected neural network and returns it, in a state where it is ready to have fit() run on it.

Description

• The API should take in values for the number of neurons in hidden layers 1, 2, 3 (as a list or array – where the length of the list/array indicates fewer hidden layers).
• The API should take in a string for the loss function.
• The API should take in the learning rate.
• Fix the optimiser to Adam.
• Set the activation functions in the hidden layers to be ReLU.
• Include the additional metric of ‘accuracy’.
• Style and clarity of code aligns with good coding practice.

3. Train the network

Description

• Call the function written in the previous section to generate a model with the following arguments: 2 hidden layers only, first with 128 nodes and the second with 64 nodes; loss function of categorical cross-entropy (you should determine which type is suitable for your data); and a learning rate of 0.001.
• Print a summary of the model that is generated.
• Train the model using a batch size of 2000 and 5 epochs, saving the training history.
• Display the learning curves.
• Answer the following questions on fitting (give reasons in each case):
  • Does this show overfitting?
  • Does this show underfitting?
  • Has the model converged?
  • Is it stable or unstable?
  • Code outputs are clearly displayed.

4. Network architecture comparison

Description

• Build and train a network using a sigmoid activation function in the hidden layers, plotting the learning curves.
• Build and train a network using LeakyReLU activation function in the hidden layers, plotting the learning curves.
• Compare the performance appropriately to select the best activation function from these two and ReLU above (state your reasons).
• Repeat the model building and training for 1. and 2. above and compare the new results with the previous ones. How different are the results?
• And how important is this difference?
• Code outputs are clearly displayed.

5. Hyper-parameter optimisation

Description

• Implement early stopping using an appropriate callback, included in the call to fit() – see example in Module 4.
• Run the network with a range of learning rates (0.01, 0.1, 10 and 100 times the base value used above) and compare learning curves to choose the best rate.
• After having chosen the best learning rate, do a simple parameter sweep for the number of nodes in the first hidden layer (using 16, 32, 64, 128, 256, 512, 1024) and display both individual learning curves as well as a summary plot of the results showing performance vs number of nodes.
• Note that it you want you can make the output more concise by suppressing text output with the verbose=0 option to fit()
• From all of the above results, choose the best hyper-parameters and clearly state your reasons. Include a comment on how reliable and repeatable you think this choice is.

6. Dropout

Description

• Using the best hyper-parameters from above, implement this model but include a dropout layer (with rate 0.4) between the last dense layer and the output layer.
• Train this and compare the result with the one from the equivalent network without dropout and decide whether the network performs better with dropout or not. Show learning curves and a brief summary of the results of your decision.

7. Final evaluation

Description

• Evaluate final performance using the best network (chosen from all the options you have explored above).
• Calculate the confusion matrix from these results and show the matrix graphically.
• Identify which class is most often incorrectly classified, explaining your reasoning.
• If the correct class is number 6 (shirt), which is the most likely error that your network will make? Explain your reasoning.

This IT Computer Science has been solved by our PhD Experts at My Uni Paper.

Be it a used or new solution, the quality of the work submitted by our assignment experts remains unhampered. You may continue to expect the same or even better quality with the used and new assignment solution files respectively. There’s one thing to be noticed that you could choose one between the two and acquire an HD either way. You could choose a new assignment solution file to get yourself an exclusive, plagiarism (with free Turnitin file), expert quality assignment or order an old solution file that was considered worthy of the highest distinction