AI-and-ML

Model trained on the the detection of forgery in artworks via the means of ML.

ABSTRACT: This project aims to accurately determine the authenticity of famous art paintings around the globe by using machine learning. In today's world, the exponentially accelerated expansion of computational notions and their endless potential, alongside which an increasing number of unexplored machine learning applications are unravelled, we have identified this problem statement as relevant and challenging. Currently, the detection of forgery or replicas in museums is done by examining the painting in fine-detail by an expert curator. Ongoing research on automated art-piece identification is limited. Through machine learning, we aim to identify if two paintings are painted by the same person, and thus verify their genuineness. We believe that our proposed solution for detecting forgery in art paintings holds interesting applications for curators, art historians, connoisseurs and art lovers. Through our project, not only can forgery be detected, but also the subtle similarities and variations between different artists can be found out, through the characteristics and style of their paintings identified via machine learning.

TIMELINE OF OUR PROCEDURES

1. Data Collection: Gather a large and diverse dataset of images of authentic and fraudulent artworks. This dataset should be well-labeled, with each image assigned to its respective class. NOTE: We may need to collect some images for training from various sources, such as online galleries, museums, and private collections, and also need to manually label the images if they are not already labeled. However, for the test-data we will simply rely on the WikiArt dataset.
2. Data Preprocessing: Preprocess the images to ensure consistency and improve the model's performance, including resizing images, normalizing pixel values, and augmenting the dataset through rotations, flips, or other transformations to increase the model's robustness.
3. Model Architecture: Utilize a deep learning architecture that has proven effective in fine-grained image classification tasks. One such architecture is the Vision Transformer (ViT) or its variants, which have demonstrated strong performance in various computer vision tasks. ViT models consist of a series of transformer encoder layers that process image patches, followed by a classification head.
4. Training: Train the model on the preprocessed dataset using a suitable optimization algorithm, such as Adam or Stochastic Gradient Descent (SGD), and a suitable loss function, such as Cross-Entropy loss. Implement techniques like learning rate scheduling and gradient clipping to improve model convergence and stability.
5. Fine-tuning: Perform transfer learning by initializing the model with pre-trained weights from a large-scale image classification task, like ImageNet. This helps the model learn more general features before focusing on the specific task of detecting fraud.
6. Regularization: Implement techniques like dropout, weight decay, or early stopping to prevent overfitting and improve the model's generalization performance.
7. Evaluation: Evaluate the model's performance on a separate test dataset, measuring metrics like accuracy, precision, recall, and F1-score, in order to provide insights into the model's strengths and weaknesses, allowing for further refinement.
8. Feature Visualization: Visualize the features learned by the model to ensure that it is focusing on the desired fine-features that distinguish authentic and fraudulent artworks. Techniques like GradCAM or saliency maps can be used for this purpose.
9. Deployment: Once the model has been trained and evaluated, deploy it in a production environment to detect fraud in real-world scenarios. This may involve integrating the model into a larger system, such as a web application or mobile app, and setting up monitoring and alerting mechanisms to notify users or administrators of potential fraud.
10. Monitoring and Maintenance: Continuously monitor the model's performance in production and retrain it as needed with new data to maintain its accuracy and robustness. This may involve retraining the model periodically, or implementing online learning techniques to update the model in real-time.