Skip to content

Latest commit

 

History

History
87 lines (61 loc) · 4.51 KB

README_ENG.md

File metadata and controls

87 lines (61 loc) · 4.51 KB
Raw

🚀 GreeDot

GreeDot Project Image

GreeDot is a project brings life to children's drawings by creating "Meta Characters". Enjoy various interaction with your characters!!

1. SKT Fly AI Challengers 4th(1st Prize 🏆)

SKT Fly AI Challengers Award

2. Demonstration Video

Demonstration Video

3. Project Overview

3-1. Background

  • A service that creates a personal AI friend that talks and moves, based on drawings made by children themselves.
  • With the age of smartphone use by children getting lower and the increase in dual-income families, we planned a service that creates friends for children.

3-2. Goals

  • Entertainment: Provide joy through engaging user experiences.
  • Social Skills Improvement: Serve as an AI buddy capable of communication and emotional exchange.
  • Emotional Monitoring: Identify the child's interests and concerns through conversation-based analysis reports.

4. System Architecture

System Architecture

4-1. Implementation Tools

Category Technologies
Frontend Flutter
Backend FastAPI
Cloud AWS RDS (MySQL), Azure VM, Azure Container (Blob Storage)
CI/CD GitHub Actions, Docker

4-2. AI, API

Category Technologies/Services
Emotion AI KoBert emotion predict
Vision AI & Rendering Animated drawings
Chat AI GPT 3.5-turbo + finetuning
APIs NAVER CLOVA voice, NAVER speech, Midjourney API

5. Technologies

5-1. Rendering (rigging + retargeting)

image
  • Tech flow
    • Animation (.bvh) creation algorithm using MediaPipe
    • Rigging + Retargeting
    • Applying animations to characters through Animated Drawings Renderer
image
  • Result

5-2. Emotion Classification

  • Trained KoBERT on about 80,000 sentences for six emotional situations
  • Improved accuracy through fine-tuning, outlier data removal, and model ensembling (test validation: 77%)

5-3. GPT Fine Tuning

  • Utilized GPT 3.5 model for fine-tuning about 70 conversations and applying personas
  • Provides various styles of voices based on personality and gender

6. Team Members

Name Role Implementation
Byeon Ji-hyeop 🤨 Team Leader, Backend, Cloud CI/CD Pipeline Construction
Lee Seong-won 😊 Backend, Frontend Drawing Upgrade Feature Implementation
Kim Dong-wook 😁 Vision AI Character Rigging Feature Implementation
Kim Dong-jae 😘 Frontend, Emotion AI STT, GPT Fine-tuning
Moon Ji-woo 😯 Frontend, Language AI Emotion Analysis Feature Implementation

7. Links