This repository contains code and results for COVID-19 classification assignment by Deep Learning Spring 2020 course offered at Information Technology University, Lahore, Pakistan. This assignment is only for learning purposes and is not intended to be used for clinical purposes.
I. VGG16 FCLayer530, learning rate 0.001 2
II. VGG16 FCLayer530, learning rate 0.00001 4
III. VGG16 FCLayer 2000, learning rate 0.001 6
IV. VGG16 FCLayer: 2000 , learning rate 0.00001 8
V. RESNET FCLayer530, learning rate 0.001 10
VI. RESNET FCLayer530, learning rate 0.00001 12
VII. RESNET FCLayer 2000, learning rate 0.00001 14
I. VGG Net, learning rate 0.001 16
II. VGG Net 16, convolutional layers FREEZED, learning rate 0.001 18
III. VGG Net 16, convolutional layers freezed [0,2,5,10,24,28], learning rate 0.001 20
IV. Res Net 18, No Freezing learning rate 0.001 22
V. Res Net 18, FC layers freeze, learning rate 0.001 24
VI. Res Net 18,convolution 1 and 2 layers freeze, learning rate 0.001 26
Dataset:
https://drive.google.com/drive/u/1/folders/1-FzZhQO9oHIT9SNOWYoKsuz7fe447vtR
GitHub link:
https://github.com/Basir-mahmood/msds19043_COVID19_DLSpring2020
The assignment includes the classification of X-Ray Images and the classes are for the COVID-19 infected or normal. VGG16 and RESNET-18 are used for the detection and classification of the images. There are several analyses which are given in the following. The most significant problem faced was the time constraint, time to complete and submit within deadline is too near, whereas, each epoch takes around 6 minutes approximately. So, in the whole a lot of time is required for analysis.
In the first task, the vgg16 network's FC layer was changed, where the first layer's neuron was changed to 530, which is equal to = (roll number*10 )+100, and there is a second layer with two neurons i.e., number of classes . The accuracy over the epochs can be seen in the following image.
Following graphs, describe the losses and training curves over epochs.
Accuracy, F1 measure and confusion matrix is as follows,
| | Predicted Infected | Predicted Normal | | --- | --- | --- | | Actual Infected | 534 | 81 | | --- | --- | --- | | Actual Normal | 25 | 860 |
Following are the two sample images from each class results.
In this part, learning rate is changed to 0.00001, and the other parameters are same.
Following are the Accuracy, F1 Measure and Confusion Matrices.
| | Predicted Infected | Predicted Normal | | --- | --- | --- | | Actual Infected | 544 | 71 | | --- | --- | --- | | Actual Normal | 25 | 860 |
Above given are the 2 images from each result class predictions
Following are the loss and accuracy curves obtained over epochs.
F1- measure, Accuracy and Confusion Matrices are as follows,
Above are the 2 case examples for each resultant of class.
FC Layers neurons are changed from 530 to 2000. Following results are obtained.
Accuracy, F1- Measure and Confusion Matrices are following
| | Predicted Infected | Predicted Normal | | --- | --- | --- | | Actual Infected | 517 | 98 | | --- | --- | --- | | Actual Normal | 37 | 848 |
Above given are the 2 pictures for each case.
In this task, the ResNet network's FC layer was changed, where the first layer's neuron was changed to 530, which is equal to = (roll number*10 )+100, and there is a second layer with two neurons i.e., number of classes . The accuracy over the epochs can be seen in the following image.
Following graphs, describe the losses and training curves over epochs.
Accuracy, F1 measure and confusion matrix is as follows,
| | Predicted Infected | Predicted Normal | | --- | --- | --- | | Actual Infected | 562 | 53 | | --- | --- | --- | | Actual Normal | 171 | 714 |
Following are the two sample images from each class results.
In this task, the resNet network's FC layer was changed, where the first layer's neuron was changed to 530, which is equal to = (roll number*10 )+100, and there is a second layer with two neurons i.e., number of classes . The accuracy over the epochs can be seen in the following image.
Following graphs, describe the losses and training curves over epochs.
Accuracy, F1 measure and confusion matrix is as follows,
| | Predicted Infected | Predicted Normal | | --- | --- | --- | | Actual Infected | 539 | 76 | | --- | --- | --- | | Actual Normal | 121 | 764 |
Following are the two sample images from each class results.
In this task, the resNet network's FC layer was changed, where the first layer's neuron was changed to 200, and there is a second layer with two neurons i.e., number of classes . The accuracy over the epochs can be seen in the following image.
Following graphs, describe the losses and training curves over epochs.
Accuracy, F1 measure and confusion matrix is as follows,
| | Predicted Infected | Predicted Normal | | --- | --- | --- | | Actual Infected | 483 | 132 | | --- | --- | --- | | Actual Normal | 89 | 796 |
Following are the two sample images from each class results.
In this task, the VGG network's is trained on the data,
Following graphs, describe the losses and training curves over epochs.
Accuracy, F1 measure and confusion matrix is as follows,
| | Predicted Infected | Predicted Normal | | --- | --- | --- | | Actual Infected | 584 | 31 | | --- | --- | --- | | Actual Normal | 29 | 856 |
Following are the two sample images from each class results.
In this task, the VGG network's is trained on the data,
Following graphs, describe the losses and training curves over epochs.
Accuracy, F1 measure and confusion matrix is as follows,
| | Predicted Infected | Predicted Normal | | --- | --- | --- | | Actual Infected | 573 | 42 | | --- | --- | --- | | Actual Normal | 9 | 876 |
Following are the two sample images from each class results.
In this task, the VGG network's is trained on the data,
Following graphs, describe the losses and training curves over epochs.
Accuracy, F1 measure and confusion matrix is as follows,
| | Predicted Infected | Predicted Normal | | --- | --- | --- | | Actual Infected | 589 | 26 | | --- | --- | --- | | Actual Normal | 17 | 868 |
Following are the two sample images from each class results.
In this task, the VGG network's is trained on the data,
Following graphs, describe the losses and training curves over epochs.
Accuracy, F1 measure and confusion matrix is as follows,
| | Predicted Infected | Predicted Normal | | --- | --- | --- | | Actual Infected | 595 | 20 | | --- | --- | --- | | Actual Normal | 30 | 855 |
Following are the two sample images from each class results.
In this task, the VGG network's is trained on the data,
Following graphs, describe the losses and training curves over epochs.
Accuracy, F1 measure and confusion matrix is as follows,
| | Predicted Infected | Predicted Normal | | --- | --- | --- | | Actual Infected | 581 | 34 | | --- | --- | --- | | Actual Normal | 14 | 871 |
Following are the two sample images from each class results.
In this task, the VGG network's is trained on the data,
Following graphs, describe the losses and training curves over epochs.
Accuracy, F1 measure and confusion matrix is as follows,
| | Predicted Infected | Predicted Normal | | --- | --- | --- | | Actual Infected | 582 | 33 | | --- | --- | --- | | Actual Normal | 38 | 847 |
Following are the two sample images from each class results.
Comparison Table
| | Test Accuracy (%) | Test F-1 Score | | --- | --- | --- | | TASK 1 | | VGG16 FCLayer530, learning rate 0.001 | 90.06 | 0.921 | | VGG16 FCLayer530, learning rate 0.00001 | 93.60 | 0.947 | | VGG16 FCLayer 2000, learning rate 0.001 | 93.46 | 0.944 | | VGG16 FCLayer : 2000 , learning rate 0.00001 | 91.0 | 0.926 | | RESNET FCLayer530, learning rate 0.001 | 85.06 | 0.864 | | RESNET FCLayer530, learning rate 0.00001 | 86.86 | 0.885 | | RESNET FCLayer 2000, learning rate 0.00001 | 85.26 | 0.870 | | TASK 2 | | VGG Net, learning rate 0.001 | 96.0 | 0.966 | | VGG Net 16, convolutional layers FREEZED, learning rate 0.001 | 96.6 | 0.971 | | VGG Net 16, convolutional layers freezed [0,2,5,10,24,28], learning rate 0.001 | 97.13 | 0.975 | | Res Net 18, No Freezing learning rate 0.001 | 96.66 | 0.971 | | Res Net 18, FC layers freeze, learning rate 0.001 | 96.8 | 0.973 | | Res Net 18,convolution 1 and 2 layers freeze, learning rate 0.001 | 95.26 | 0.976 |
It could be compared from the graphs over epochs that decreasing the learning rate, smooth the curve and the curve goes towards better accuracy and loss with smoothness, whereas, high learning rate tends to make abrupt changes, but it is also evident from the graphs that decreasing the learning rate makes the learning slow.
From Task 1, It could be illustrated that increasing the number of neurons also increase the accuracy and also the F1-score. Therefore, as the number of neurons is very less i.e., 530 and corresponding accuracy is 90%, whereas, increasing the number of neurons to 2000, the accuracy increased to 93.4. Similarly, in task 2 it is increased to 96%.
From Task 2, for some convolutional layers freeze, we observe that the accuracy is increased to 97.13 %. Whereas, whole unfreeze network gives 96.6% accuracy. Therefore, it could be said that the intermediate layers frozen, which could learn several parameters can give better results.
From the images of wrong identification, there are images which have contrast problem. The network is predicting them wrong; it is supposed that such networks are learning brightness at particular places rather than texture or behavior.
Some of the X-Rays also have tubes or medical apparatus, it could be seen that such images are also predicted wrong.
Overall, the ResNet predict CoVID19 cases better than VGG, but with the fine tuning it is find that VGG16 is slightly better performing on the dataset, this behavior is mainly due to the higher number of convolution layers and the dataset properties.