Skip to content

Commit

Permalink
feat: html website init
Browse files Browse the repository at this point in the history
  • Loading branch information
@tunglev
tunglev committed Dec 15, 2023
1 parent a0ec925 commit 0943627
Show file tree
Hide file tree
Showing 4 changed files with 608 additions and 0 deletions.
173 changes: 173 additions & 0 deletions index.html
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,173 @@
<!DOCTYPE html>
<html lang="en-US">
<head>
<meta charset="UTF-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">

<!-- Begin Jekyll SEO tag v2.8.0 -->
<title>Notice of possible occurrence: | HandsOn</title>
<meta name="generator" content="Jekyll v3.9.3" />
<meta property="og:title" content="Notice of possible occurrence:" />
<meta property="og:locale" content="en_US" />
<meta name="description" content="Educational augmented reality AI lab assistant and simulation with multi-user in the same environment support. Hands On offers a 3D virtual lab sandbox, allowing them to conduct scientific experiments using realistic virtual tools. One of its standout features is AI assistance, which provides real-time guidance, feedback, and explanations ." />
<meta property="og:description" content="Educational augmented reality AI lab assistant and simulation with multi-user in the same environment support. Hands On offers a 3D virtual lab sandbox, allowing them to conduct scientific experiments using realistic virtual tools. One of its standout features is AI assistance, which provides real-time guidance, feedback, and explanations ." />
<link rel="canonical" href="https://project-slot.github.io/HandsOn/" />
<meta property="og:url" content="https://project-slot.github.io/HandsOn/" />
<meta property="og:site_name" content="HandsOn" />
<meta property="og:type" content="website" />
<meta name="twitter:card" content="summary" />
<meta property="twitter:title" content="Notice of possible occurrence:" />
<link rel="stylesheet" href="www/index.css">
<script type="application/ld+json">
{"@context":"https://schema.org","@type":"WebSite","description":"Educational augmented reality AI lab assistant and simulation with multi-user in the same environment support. Hands On offers a 3D virtual lab sandbox, allowing them to conduct scientific experiments using realistic virtual tools. One of its standout features is AI assistance, which provides real-time guidance, feedback, and explanations .","headline":"Notice of possible occurrence:","name":"HandsOn","url":"https://project-slot.github.io/HandsOn/"}</script>
<!-- End Jekyll SEO tag -->

<link rel="stylesheet" href="/HandsOn/assets/css/style.css?v=a0ec9251d8d0dc39384f376c35327a9ddfdde23c">
<!-- start custom head snippets, customize with your own _includes/head-custom.html file -->

<!-- Setup Google Analytics -->



<!-- You can set your favicon here -->
<!-- link rel="shortcut icon" type="image/x-icon" href="/HandsOn/favicon.ico" -->

<!-- end custom head snippets -->

</head>
<body>
<div class="container-lg px-3 my-5 markdown-body">

<h1><a href="https://github.com/project-slot/HandsOn/">Hands On: Open Source AI Argumented Lab</a></h1>


<h3 id="notice-of-possible-occurrence">Notice of possible occurrence:</h3>
<p>After publishing this Github repo, I saw that my Azure credit was significantly dropped, resulting in Speech API deny. This might be due to bad people taking advantage of my Azure keys and endpoints, which are hard coded (although a bad practice but due to time constrain). I’m trying to fix it so that the speech-to-text will work again.</p>

<h1 id="docs">Overview</h1>

<p><img src="https://github.com/project-slot/HandsOn/assets/99946449/c31ee4f9-ec34-4025-8472-711add9ca54d" alt="0 2" />
<em>Development side architecture. Server architecture is underdevelopment.</em></p>

<ul>
<li>
<p>Official Docs <em>(coming soon)</em></p>
</li>
<li>
<p><a href="https://github.com/TungVietLe/HandsOn/blob/main/ADT.md">class ADT</a></p>
</li>
</ul>

<p>Laboratory equipment are expensive. Learning on 2D blackboard is unengaging. Aiming to make laboratories accessible to students around the world; Helping teachers to make lesson more engaging. Hands On is AR simulation equipped with AI for both teachers &amp; students to interact, experiment, and make mistakes freely.</p>

<p><img src="https://github.com/TungVietLe/HandsOn/assets/99946449/417254ae-a0df-4f6d-afbb-72be4eff66b2" alt="Screenshot 2023-10-30 094619" /></p>

<h1 id="video">Videos</h1>
<h3 id="demo-use-httpsyoutube0zqbgqmgolcsi09hiryc_1vlpyp-l">Demo Use: https://youtu.be/0ZqBgqMGOlc?si=09HIryc_1vLpYP-l</h3>
<h3 id="demo-notes-drawing-httpsyoutube-qrmq-fztkgsiwsmf8ovqxwj4frcs">Demo Notes Drawing: https://youtu.be/-qrMq-FZTkg?si=WSMF8oVQxwj4frcs</h3>
<video width="100%" height="100%" controls autoplay muted>
<source src="www/main.mp4" type="video/mp4">
Your browser does not support the video tag.
</video>
<em>Note: experiment virtual environment + force analysis.<em></em></em>
<video width="100%" height="100%" controls autoplay muted>
<source src="www/note.mp4" type="video/mp4">
Your browser does not support the video tag.
</video>
<em>Note: infinite canvas note taking.<em></em></em>



<h1 id="interacting-with-the-environment-using-natural-language">Interacting with the environment using natural language</h1>
<p>a. Speak to Hands On:</p>
<ul>
<li>“I need a weight[object] made of iron[material] to conduct a buoyancy
force experiment”.</li>
<li>“Change the simulation gravity[field] to the moon’s gravity[value]”.</li>
<li>“Set the height[field] of liquid[target] to 57cm[value]”.</li>
<li>“Spawn something to measure length[description]”.</li>
</ul>

<p>b. Best Practices:
i. It’s good to keep it concise about what your intentions are.
ii. You can ask Hands On to assist you with details. Example: “Show me
the forces analysis of objects.”
iii. Beside speech interaction, you can also interact with the simulation
using real-world physical movement. Example: “grab object, move
object around.”
iv. Users are also able to take notes anywhere in the environment, so the
world is your infinite, limitless canvas.</p>

<h1 id="whats-next-for-hands-on">What’s next for Hands On</h1>
<ul>
<li>Real-time AI feedback</li>
<li>Multi User Collaboration on the same AR environment</li>
<li>Tool recognition using computer vision</li>
<li>IOS ARKit development</li>
<li>Classroom / lab safety practice</li>
</ul>
<p><img src="https://github.com/TungVietLe/HandsOn/assets/99946449/73a44b9e-21be-490c-972d-8c47ef39c58c"
alt="Group 3" />
<em>Note: Suggested UI.<em></em></em>
</p>

<h1 id="inspiration">Inspiration</h1>
<p>I remember the first time I went to my physics lab during my high school years to experiment with electronic circuits. The equipment looks nothing familiar to me even though I have been studying it for a year. Turn out I was working with 2D symbol drawings the whole time, without touching or experiencing myself.</p>

<h1 id="what-it-does">What it does</h1>
<p>Laboratory equipment are expensive. Learning on the 2D blackboard is unengaging. Making Physic Experiment accessible anywhere, Hands On is an AR simulation for both teachers and students to interact, experiment, and make mistakes freely. It makes classroom experience more engaging for everyone. Better yet, Hands On is equipped with AI to instruct, explain to students in real-time: like “You should turn off the power source before plugging in”, or perform actions like “Give me the set of equipment for Archimedes experiment”, which can enhance creativity, safety, and familiarity.</p>

<p><img src="https://github.com/TungVietLe/HandsOn/assets/99946449/7bd3e593-048c-4d37-b5f2-1783798e9a94" alt="Screenshot 2023-10-30 094740" /></p>

<h1 id="how-we-built-it">How we built it</h1>
<p>Input utterances, label data, train, test, improve, and deploy AI model using Azure CLU.
Connected Cognitive Search and SQL database.
Code 3D physic simulations from scratch using Unity C#
Develop for Android AR Core</p>

<h1 id="many-challenges-still-to-come">Many challenges still to come</h1>
<p><img src="https://github.com/TungVietLe/HandsOn/assets/99946449/d3d547d4-d10c-4dba-bd61-728abf012ea6" alt="Screenshot 2023-10-30 140037" />
It’s just a super early state. The model is still improving and hasn’t generalized things well enough. Currently, the project is just a prototype with some simple requests.</p>

<h1 id="future-goals">Future goals</h1>
<ol>
<li>AI Real-time feedback (Anticipated example use cases):
<ul>
<li>“Why is my spectrometer not giving accurate readings?”</li>
<li>Real-Time Feedback: Hands On can suggest checking the alignment,
the quality of the light source, or the calibration of the spectrometer.</li>
</ul>
</li>
</ol>

<ul>
<li>“Is it safe to heat this substance with the Bunsen burner?”</li>
<li>Real-Time Feedback: Hands On can provide safety guidelines,
including the substance’s properties and suitable temperature ranges
for heating.</li>
</ul>

<ol>
<li>AR Multi-user in the same environment / Tools recognition
using computer vision
<ul>
<li>Enhance collaboration that allows students to work together and share
experiences.</li>
<li>Allow teachers to demonstrate how the experiment should be conducted.</li>
<li>Import tools into the simulation.</li>
</ul>
</li>
</ol>



<div class="footer border-top border-gray-light mt-5 pt-3 text-right text-gray">
This site is open source. <a href="https://github.com/project-slot/HandsOn">Improve this page</a>.
</div>

</div>
<script src="https://cdnjs.cloudflare.com/ajax/libs/anchor-js/4.1.0/anchor.min.js" integrity="sha256-lZaRhKri35AyJSypXXs4o6OPFTbTmUoltBbDCbdzegg=" crossorigin="anonymous"></script>
<script>anchors.add();</script>
</body>
</html>
Loading

0 comments on commit 0943627

Please sign in to comment.