Skip to content

Commit

Permalink
chatscene
Browse files Browse the repository at this point in the history
  • Loading branch information
@javyduck
javyduck committed Jun 20, 2024
1 parent 10b3279 commit 91ea6ec
Show file tree
Hide file tree
Showing 31 changed files with 9,261 additions and 0 deletions.
Binary file added chatscene/.DS_Store
View file
Binary file not shown.
1 change: 1 addition & 0 deletions chatscene/.nojekyll
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@

255 changes: 255 additions & 0 deletions chatscene/index.html
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,255 @@
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<!-- Meta tags for social media banners, these should be filled in appropriatly as they are your "business card" -->
<!-- Replace the content tag with appropriate information -->
<meta name="description" content="DESCRIPTION META TAG">
<meta property="og:title" content="SOCIAL MEDIA TITLE TAG"/>
<meta property="og:description" content="SOCIAL MEDIA DESCRIPTION TAG TAG"/>
<meta property="og:url" content="URL OF THE WEBSITE"/>
<!-- Path to banner image, should be in the path listed below. Optimal dimenssions are 1200X630-->
<meta property="og:image" content="static/image/your_banner_image.png" />
<meta property="og:image:width" content="1200"/>
<meta property="og:image:height" content="630"/>


<meta name="twitter:title" content="TWITTER BANNER TITLE META TAG">
<meta name="twitter:description" content="TWITTER BANNER DESCRIPTION META TAG">
<!-- Path to banner image, should be in the path listed below. Optimal dimenssions are 1200X600-->
<meta name="twitter:image" content="static/images/your_twitter_banner_image.png">
<meta name="twitter:card" content="summary_large_image">
<!-- Keywords for your paper to be indexed by-->
<meta name="keywords" content="KEYWORDS SHOULD BE PLACED HERE">
<meta name="viewport" content="width=device-width, initial-scale=1">


<title>ChatScene</title>
<link rel="icon" type="image/x-icon" href="static/images/favicon.ico">
<link href="https://fonts.googleapis.com/css?family=Google+Sans|Noto+Sans|Castoro"
rel="stylesheet">

<link rel="stylesheet" href="static/css/bulma.min.css">
<link rel="stylesheet" href="static/css/bulma-carousel.min.css">
<link rel="stylesheet" href="static/css/bulma-slider.min.css">
<link rel="stylesheet" href="static/css/fontawesome.all.min.css">
<link rel="stylesheet"
href="https://cdn.jsdelivr.net/gh/jpswalsh/academicons@1/css/academicons.min.css">
<link rel="stylesheet" href="static/css/index.css">

<script src="https://ajax.googleapis.com/ajax/libs/jquery/3.5.1/jquery.min.js"></script>
<script src="https://documentcloud.adobe.com/view-sdk/main.js"></script>
<script defer src="static/js/fontawesome.all.min.js"></script>
<script src="static/js/bulma-carousel.min.js"></script>
<script src="static/js/bulma-slider.min.js"></script>
<script src="static/js/index.js"></script>
</head>
<body>


<section class="hero">
<div class="hero-body">
<div class="container is-max-desktop">
<div class="columns is-centered">
<div class="column has-text-centered">
<h1 class="title is-2 publication-title">[CVPR 2024] ChatScene: Knowledge-Enabled <br> Safety-Critical Scenario Generation for Autonomous Vehicles</h1>
<div class="is-size-4 publication-authors">
<!-- Paper authors -->
<span class="author-block">
<a href="https://javyduck.github.io/" target="_blank">Jiawei Zhang<sup>1</sup></a>, </span>
<span class="author-block">
<a href="https://xuchejian.com/" target="_blank">Chejian Xu<sup>1</sup></a>,</span>
<span class="author-block">
<a href="https://aisecure.github.io/" target="_blank">Bo Li<sup>1, 2</sup></a>
</span>
</div>

<div class="is-size-5 publication-authors">
<span class="author-block"><sup>1</sup>UIUC, <sup>2</sup>UChicago</span>
</div>

<div class="column has-text-centered">
<div class="publication-links">
<!-- ArXiv abstract Link -->
<span class="link-block">
<a href="https://arxiv.org/abs/2405.14062" target="_blank"
class="external-link button is-normal is-rounded is-dark">
<span class="icon">
<i class="ai ai-arxiv"></i>
</span>
<span>arXiv</span>
</a>
</span>

<!-- Github link -->
<span class="link-block">
<a href="https://github.com/javyduck/ChatScene" target="_blank"
class="external-link button is-normal is-rounded is-dark">
<span class="icon">
<i class="fab fa-github"></i>
</span>
<span>Code</span>
</a>
</span>
</div>
</div>
</div>
</div>
</div>
</div>
</section>

<!-- Paper abstract -->
<section class="section hero is-light">
<div class="container is-max-desktop">
<div class="columns is-centered has-text-centered">
<div class="column is-four-fifths">
<h2 class="title is-3">Abstract</h2>
<div class="content has-text-justified">
<p>
We present ChatScene, a Large Language Model (LLM)-based agent that leverages the capabilities of LLMs to generate safety-critical scenarios for autonomous vehicles. Given unstructured language instructions, the agent first generates textually described traffic scenarios using LLMs. These scenario descriptions are subsequently broken down into several sub-descriptions for specified details such as behaviors and locations of vehicles. The agent then distinctively transforms the textually described sub-scenarios into domain-specific languages, which then generate actual code for prediction and control in simulators, facilitating the creation of diverse and complex scenarios within the CARLA simulation environment. A key part of our agent is a comprehensive knowledge retrieval component, which efficiently translates specific textual descriptions into corresponding domain-specific code snippets by training a knowledge database containing the scenario description and code pairs. Extensive experimental results underscore the efficacy of ChatScene in improving the safety of autonomous vehicles. For instance, the scenarios generated by ChatScene show a 15% increase in collision rates compared to state-of-the-art baselines when tested against different reinforcement learning-based ego vehicles. Furthermore, we show that by using our generated safety-critical scenarios to fine-tune different RL-based autonomous driving models, they can achieve a 9% reduction in collision rates, surpassing current SOTA methods. ChatScene effectively bridges the gap between textual descriptions of traffic scenarios and practical CARLA simulations, providing a unified way to conveniently generate safety-critical scenarios for safety testing and improvement for AVs.
</p>
</div>
</div>
</div>
</div>
</section>
<!-- End paper abstract -->


<!-- Image carousel -->
<section class="hero is-small is-light">
<div class="hero-body">
<div class="container has-text-centered">
<h2 class="title">Pipeline</h2>

<img src="static/images/chatscene.svg" width="100%" height="550">
</img>

</div>
</div>
</section>
<!--End paper poster -->

<!-- End image carousel -->


<!-- Video carousel -->
<section class="hero is-small is-light">
<div class="hero-body">
<div class="container has-text-centered">
<h2 class="title is-3">Text2Simulation On Eight Scenarios</h2>
<div id="results-carousel" class="carousel results-carousel">
<div class="item item-video1">
<video poster="" id="video1" autoplay controls muted loop height="100%">
<!-- Your video file here -->
<source src="static/videos/scenario_1.mp4"
type="video/mp4">
</video>
</div>
<div class="item item-video2">
<video poster="" id="video2" autoplay controls muted loop height="100%">
<!-- Your video file here -->
<source src="static/videos/scenario_2.mp4"
type="video/mp4">
</video>
</div>
<div class="item item-video3">
<video poster="" id="video3" autoplay controls muted loop height="100%">\
<!-- Your video file here -->
<source src="static/videos/scenario_3.mp4"
type="video/mp4">
</video>
</div>
<div class="item item-video4">
<video poster="" id="video4" autoplay controls muted loop height="100%">\
<!-- Your video file here -->
<source src="static/videos/scenario_4.mp4"
type="video/mp4">
</video>
</div>
<div class="item item-video5">
<video poster="" id="video5" autoplay controls muted loop height="100%">\
<!-- Your video file here -->
<source src="static/videos/scenario_5.mp4"
type="video/mp4">
</video>
</div>
<div class="item item-video6">
<video poster="" id="video6" autoplay controls muted loop height="100%">\
<!-- Your video file here -->
<source src="static/videos/scenario_6.mp4"
type="video/mp4">
</video>
</div>
<div class="item item-video7">
<video poster="" id="video7" autoplay controls muted loop height="100%">\
<!-- Your video file here -->
<source src="static/videos/scenario_7.mp4"
type="video/mp4">
</video>
</div>
<div class="item item-video8">
<video poster="" id="video8" autoplay controls muted loop height="100%">\
<!-- Your video file here -->
<source src="static/videos/scenario_8.mp4"
type="video/mp4">
</video>
</div>
</div>
</div>
</div>
</section>
<!-- End video carousel -->

<!-- Image carousel -->
<section class="hero is-small is-light">
<div class="hero-body">
<div class="container has-text-centered">
<h2 class="title">Poster</h2>

<img src="static/images/ChatScene_poster.svg" width="100%" height="550">
</img>

</div>
</div>
</section>
<!--End paper poster -->

<!--BibTex citation -->
<section class="section" id="BibTeX">
<div class="container is-max-desktop content">
<h2 class="title">Citation</h2>
<pre><code>@inproceedings{zhang2024chatscene,
title={ChatScene: Knowledge-Enabled Safety-Critical Scenario Generation for Autonomous Vehicles},
author={Zhang, Jiawei and Xu, Chejian and Li, Bo},
booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
pages={15459--15469},
year={2024}
}</code></pre>
</div>
</section>
<!--End BibTex citation -->
<footer class="footer">
<div class="container">
<div class="columns is-centered">
<div class="column is-4">
<div class="content">

<p>
The web was built using the <a href="https://github.com/eliahuhorwitz/Academic-project-page-template" target="_blank">academic template</a>.
</p>

</div>
</div>
</div>
</div>
</footer>
<!-- Statcounter tracking code -->

<!-- You can add a tracker to track page visits by creating an account at statcounter.com -->

<!-- End of Statcounter Code -->

</body>
</html>
Binary file added chatscene/static/.DS_Store
View file
Binary file not shown.
1 change: 1 addition & 0 deletions chatscene/static/css/bulma-carousel.min.css
View file

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading

0 comments on commit 91ea6ec

Please sign in to comment.