you will use Python to implement a simple artificial neural network and be able to train a simple convolution neural network on a hand written dataset called MNIST.

INSTRUCTIONS TO CANDIDATES

ANSWER ALL QUESTIONS

 Introduction to Artificial Intelligence

 Homework 3

Instructions

In this homework you will use Python to implement a simple artificial neural network and be able to train a simple convolution neural network on a hand written dataset called MNIST. We have posted the numpy based implementation code on Blackboard. To run it, you need to install some dependency libraries like numpy, scikit-image, matplotlib. To setup the environment, you should use pip with the following command.

pip install numpy scikit-image matplotlib scikit-learn

 

 

The homework should be done in Python (Python 3.X is recommended). Include a documentation (in .doc or .pdf format) that describes the complete steps of your solutions for each problem including: (1) The answer to any questions posed, (2) any results (include print outs or terminal outputs), and (3) the name of any Python module(s) and function(s) (Numpy, Matplotlib, etc.) you used. Discussion is allowed, but you must submit your own write-up and list your collaborators.

 

Zip all your documents with the source code (.py files, if there are self-implemented modules, organize them in different folders), documentation, and data (image files, text, etc.) into a file called [Lastname_FirstInitial_Homework3.zip] and upload it to the Blackboard prior to the due date.

 

 Make sure your code works when you send it out.  Please use as less third-party libraries as possible cause it is easy for instructors to reproduce your results. If you used the libraries that requires installation, please list the dependencies and its version in your README file or using pip to generate the dependency list like pip freeze > requirements.txt.

For late assignments, you will receive 10% off per day on any assignment handed in late up to a week. However, after a week on any given homework you will receive no credit for the assignment. So please start your assignment ASAP.

Homework Description

The source code ANN_LeNet_MNIST_demo.py gives you three choices to build a neural network: ‘TwoLayerNet’, ‘ThreeLayerNet’, ‘LeNet5’. You should be able to run the program by giving specific parameter to the program like:

python ANN_LeNet_MNIST_demo.py -model TwoLayerNet -iter 10000 -opti SGD

Data preparation

You should first download the MNIST dataset (http://yann.lecun.com/exdb/mnist/) using this piece of code in MNIST_util.py

    download_mnist()

save_mnist()

After this step, you should see mnist.pkl in your folder which is the training and testing set you will use in this homework. You can use get_MNIST_train_viz_pngs() and get_MNIST_test_viz_pngs() to visualize these hand written characters.

Artificial Neural Networks

In ANN.py, we have provided the code for model TwoLayerNet. Which is a two layer artificial neural network with two layers of fully connected layers and one ReLU activation layer. The __init__ function defines all the layers that would be used in the model and initialize them if the weights are given. The forward function takes in input X (which is the hand written image in our task), and return the output of the neural network (which is a ten digits vector representing the probability of 10 digits given the input sample). The get_params and set_params return or set parameters of the neural network. Note that every FC layer is followed by an activation layer (ReLU in TwoLayerNet model) except the final FC layer.

In this homework, you should try to implement a three layer artificial neural network -ThreeLayerNet based one TwoLayerNet. Note that since you have three layers in the network, there should be three FC (fully connected) layers and two activation layers (use ReLU). The H2 in  __init__ function is the output dimension of the second FC layer and the input dimension of the third FC layer.

• Train the TwoLayerNet with default parameter setting, set iteration=25000, try both optimizers ‘SGD’ and ‘SGDMomentum‘, report the training and testing accuracy. Plot the training loss over iterations like the below image. (5 points)

Training loss over iterations. X-axis represents the training iterations/100, y is the training loss.

• Implement the ThreeLayerNet by yourself. Set H1=300, H2=100, and train the ThreeLayerNet model 25000 iterations. Report accuracy on both training and testing dataset and draw the loss figure. (10 points)
• Train LeNet5 model 2000 iterations, report the accuracy on testing dataset. (5 points)

Related Questions

. The fundamental operations of create, read, update, and delete (CRUD) in either Python or Java

CS 340 Milestone One Guidelines and Rubric  Overview: For this assignment, you will implement the fundamental operations of create, read, update,

. Develop a program to emulate a purchase transaction at a retail store. This  program will have two classes, a LineItem class and a Transaction class

Retail Transaction Programming Project  Project Requirements:  Develop a program to emulate a purchase transaction at a retail store. This

. The following program contains five errors. Identify the errors and fix them

7COM1028   Secure Systems Programming   Referral Coursework: Secure

. Accepts the following from a user: Item Name Item Quantity Item Price Allows the user to create a file to store the sales receipt contents

Create a GUI program that:Accepts the following from a user:Item NameItem QuantityItem PriceAllows the user to create a file to store the sales receip

. The final project will encompass developing a web service using a software stack and implementing an industry-standard interface. Regardless of whether you choose to pursue application development goals as a pure developer or as a software engineer

CS 340 Final Project Guidelines and Rubric  Overview The final project will encompass developing a web service using a software stack and impleme