Skip to content

Commit

Permalink
[add] GSoC page
Browse files Browse the repository at this point in the history
  • Loading branch information
bassamadnan committed Aug 21, 2024
1 parent c0d8f40 commit d573085
Show file tree
Hide file tree
Showing 5 changed files with 116 additions and 5 deletions.
16 changes: 16 additions & 0 deletions docs/GSoC/GSoC.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# PictoPy

## Project Overview

PictoPy is an open-source desktop application designed for managing and organizing digital photos. It leverages advanced AI and machine learning techniques to provide smart, privacy-focused photo management solutions.
<br>


PictoPy has been a part of GSoC 2024 under AOSSIE organization.
## Key Features

- **Privacy-Focused**: Desgined to work offline with locally running models.
- **Smart Organization**: Uses YOLO models for automatic tagging and categorization of photos.
- **Face Recognition**: Groups photos based on detected faces.
- **Cross-Platform**: Built to work seamlessly on various desktop operating systems.
- **Open Source**: Developed transparently with community involvement.
89 changes: 89 additions & 0 deletions docs/GSoC/GSoC2024/BassamAdnan.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
# GSoC 2024 Report - PictoPy

## Project Overview

PictoPy is a project developed during Google Summer of Code 2024 for AOSSIE. The goal was to create a backend system for image processing, object detection, and face recognition.

## Phase 1 (Mid-Phase)

### Setup and Initial Development

1. **Backend Setup**
- Initialized a FastAPI backend with a standard directory structure
- Faced challenges with large library sizes for object detection models
- Switched to ONNX format, reducing environment size from 5GB (with PyTorch GPU) to ~400MB

!!! note "Related PRs"
- [PR-25](https://github.com/AOSSIE-Org/PictoPy/pull/25) (Merged)
- [PR-26](https://github.com/AOSSIE-Org/PictoPy/pull/26) (Closed)

2. **Routing Logic with Parallel Processing**
- Implemented non-blocking requests using FastAPI
- Explored various options for parallel processing (threading, multiprocessing)
- Settled on asyncio for concurrent processing of multiple images
- Switched from uvicorn to hypercorn for better cross-platform compatibility

3. **Database Design**
- Developed schemas for storing image and album information
- Image schema includes file path, object detection results, metadata, and potential face embeddings
- Album schema contains multiple images and album-specific information
- Plans to refine schemas for more concrete mappings and handle edge cases

!!! note "Related PRs"
- [PR-29](https://github.com/AOSSIE-Org/PictoPy/pull/29) (Merged)
- [PR-30](https://github.com/AOSSIE-Org/PictoPy/pull/30) (Merged)


## Phase 2 (Final Phase)

### Feature Implementation

1. **Face Embeddings**
- Integrated object detection model
- Implemented face detection and embedding generation
- Tested various models (ArcFace, VGG, etc.)
- Selected FaceNet for optimal size and performance constraints
- Used ultralytics model for object and face detection
- Generated face embeddings using FaceNet
- All models converted to ONNX format for efficiency

!!! note "Related PR"
[PR-34](https://github.com/AOSSIE-Org/PictoPy/pull/34) (Merged)

2. **Face Recognition and Schema Updates**
- Implemented face clustering using DBSCAN algorithm
- Updated schemas to accommodate clustering results
- Ensured proper handling of image operations (add/delete) affecting clusters
- Completed core project requirements (image database, album database, object detection, face detection, and recognition)
- Optimized DBSCAN parameters for ONNX model performance

!!! note "Related PR"
[PR-36](https://github.com/AOSSIE-Org/PictoPy/pull/36) (Merged)

3. **Documentation and API Collection**
- Created comprehensive documentation on setup, directory structure, and model architecture
- Developed a [Postman Collection](https://www.postman.com/cryosat-explorer-62744145/workspace/pictopy/overview) for API testing and development
- Utilized mkdocs-material for documentation, hosted [here](https://aossie-org.github.io/PictoPy/)
- Provided a Dockerfile for backend containerization

!!! note "Related PRs"
- [PR-37](https://github.com/AOSSIE-Org/PictoPy/pull/37) (Closed)
- [PR-39](https://github.com/AOSSIE-Org/PictoPy/pull/39) (Merged)
- [PR-41](https://github.com/AOSSIE-Org/PictoPy/pull/41) (Merged)

### Project Completion Status

- **Status**: Completed
- **Significant Changes**: None, adhered to original plan with minor adjustments (e.g., using ONNX runtime instead of exploring OpenVINO)

## Future Plans

- Potential development of an Electron-based frontend
- Continued contribution to the project
- Promotion of PictoPy to leverage its potential and encourage further development

### Blog Posts

- [My GSoC Journey — The Beginning](https://medium.com/@mailbassam/my-gsoc-journey-phase-1-56664a0d6ba0)
- [My GSoC Journey — Phase 1](https://medium.com/@mailbassam/my-gsoc-journey-phase-1-14a6bf654799)
- [My GSoC Journey — Phase 2](https://medium.com/@mailbassam/my-gsoc-journey-phase-2-4eca00dfc8fc)
6 changes: 5 additions & 1 deletion docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,10 @@ This project was announced by [AOSSIE](https://aossie.org/), an umbrella organiz
Architecture
</a>
</li>
<li><a href='./GSoC/GSoC'>
GSoC
</a>
</li>
</ul>
</div>
<div style="width:25%">
Expand Down Expand Up @@ -93,4 +97,4 @@ This project was announced by [AOSSIE](https://aossie.org/), an umbrella organiz
</li>
</ul>
</div>
</div>
</div>
6 changes: 2 additions & 4 deletions docs/overview/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,13 +27,11 @@ We use DBSCAN algorithm to perform clustering for face embeddings generated. All
on queries from the backend.

!!! note "Note"
We discuss all of the features and configuration of our application in further sections of the documentation. They can be used for both developers
as well as users who want to use the app. A postman collection has also been added which can be found in our API section.
We discuss all of the features and configuration of our application in further sections of the documentation. They can be used for both developers
as well as users who want to use the app. A postman collection has also been added which can be found in our API section.
<br>
<br>

## Backend rust (via Tauri)

The Rust backend, integrated through Tauri, is a core component of our application. It leverages Rust's performance and safety features to handle file system operations, provide a secure bridge between the frontend and the local system, and manage OS-level interactions. This backend efficiently manages tasks such as reading and writing image files, extracting metadata, and ensuring secure access to system resources. It communicates with the React frontend through an IPC mechanism, allowing for seamless integration of low-level functionalities with the user interface. This architecture enables high-performance, secure operations on the local system while maintaining a smooth user experience.


4 changes: 4 additions & 0 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,10 @@ nav:
- UI Components: frontend/ui-components.md
- State Management: frontend/state-management.md
- Gallery View: frontend/gallery-view.md
- GSoC:
- GSoC: GSoC/GSoC.md
- GSoC2024:
- Bassam Adnan: GSoC/GSoC2024/BassamAdnan.md

extra:
social:
Expand Down

0 comments on commit d573085

Please sign in to comment.