Skip to content

Commit

Permalink
update figures and menu
Browse files Browse the repository at this point in the history
  • Loading branch information
@ypwang61
ypwang61 committed Oct 4, 2024
1 parent 7e59329 commit 684f886
Show file tree
Hide file tree
Showing 21 changed files with 537 additions and 2 deletions.
5 changes: 4 additions & 1 deletion MENU
View file
Original file line number Diff line number Diff line change
@@ -1,3 +1,6 @@
Yiping Wang
Home[index.html]
Home[index2.html]
Publications[pub.html]
Miscellaneous[miscellaneous.html]
Fun[fun.html]
CV[CV_YipingWang_phd.pdf]
2 changes: 2 additions & 0 deletions creed.jemdoc
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
# jemdoc: menu{MENU}{creed.html}, nofooter

47 changes: 47 additions & 0 deletions fun.html
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
"http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en">
<head>
<meta name="generator" content="jemdoc, see http://jemdoc.jaboc.net/" />
<meta http-equiv="Content-Type" content="text/html;charset=utf-8" />
<link rel="stylesheet" href="jemdoc.css" type="text/css" />
<title>Fun</title>
</head>
<body>
<table summary="Table for page layout." id="tlayout">
<tr valign="top">
<td id="layout-menu">
<div class="menu-category">Yiping Wang</div>
<div class="menu-item"><a href="index2.html">Home</a></div>
<div class="menu-item"><a href="pub.html">Publications</a></div>
<div class="menu-item"><a href="miscellaneous.html">Miscellaneous</a></div>
<div class="menu-item"><a href="fun.html" class="current">Fun</a></div>
<div class="menu-item"><a href="CV_YipingWang_phd.pdf">CV</a></div>
</td>
<td id="layout-content">
<div id="toptitle">
<h1>Fun</h1>
</div>
<h2>Hobbies</h2>
<p>I like playing pingpong very much :)</p>
<h2>Some of my favorite thoughts</h2>
<ul>
<li><p>Deep learning is based on the audacious conjecture that biological neurons and artificial neurons are not that different (from <a href="https://twitter.com/ilyasut/status/1587478598809591808">Ilya</a>).</p>
</li>
<li><p><a href="http://www.incompleteideas.net/IncIdeas/BitterLesson.html">The Bitter Lesson</a></p>
</li>
<li><p>Useful or elegant, at least one.</p>
</li>
<li><p>若不披上这件衣裳，众生又怎知我尘缘已断、金海尽干</p>
</li>
</ul>
<h2>Some of my thoughts</h2>
<ul>
<li><p>09/2024: Artificial intelligence can be very smart, but it still needs to follow the truth. If intelligence continues to evolve in the future, seeking the truth to survive in the universe may serve as a creed for a peaceful transition between generations.</p>
</li>
</ul>
</td>
</tr>
</table>
</body>
</html>
Expand Down
18 changes: 18 additions & 0 deletions fun.jemdoc
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
# jemdoc: menu{MENU}{fun.html}, nofooter
==Fun

== Hobbies

I like playing pingpong very much :)

== Some of my favorite thoughts
- Deep learning is based on the audacious conjecture that biological neurons and artificial neurons are not that different (from [https://twitter.com/ilyasut/status/1587478598809591808 Ilya]).
- [http://www.incompleteideas.net/IncIdeas/BitterLesson.html The Bitter Lesson]
- Useful or elegant, at least one.
- 若不披上这件衣裳，众生又怎知我尘缘已断、金海尽干

== Some of my thoughts
- 09\/2024: Artificial intelligence can be very smart, but it still needs to follow the truth. If intelligence continues to evolve in the future, seeking the truth to survive in the universe may serve as a creed for a peaceful transition between generations.



1 change: 0 additions & 1 deletion index.jemdoc
View file
Original file line number Diff line number Diff line change
Expand Up @@ -50,4 +50,3 @@ During my undergraduate, I was very fortunate to work closely with [http://yuand




97 changes: 97 additions & 0 deletions index2.html
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
"http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en">
<head>
<meta name="generator" content="jemdoc, see http://jemdoc.jaboc.net/" />
<meta http-equiv="Content-Type" content="text/html;charset=utf-8" />
<link rel="stylesheet" href="jemdoc.css" type="text/css" />
<title>Yiping Wang 王宜平</title>
</head>
<body>
<table summary="Table for page layout." id="tlayout">
<tr valign="top">
<td id="layout-menu">
<div class="menu-category">Yiping Wang</div>
<div class="menu-item"><a href="index2.html">Home</a></div>
<div class="menu-item"><a href="pub.html">Publications</a></div>
<div class="menu-item"><a href="miscellaneous.html">Miscellaneous</a></div>
<div class="menu-item"><a href="fun.html">Fun</a></div>
<div class="menu-item"><a href="CV_YipingWang_phd.pdf">CV</a></div>
</td>
<td id="layout-content">
<div id="toptitle">
<h1>Yiping Wang 王宜平</h1>
</div>
<table class="imgtable"><tr><td>
<img src="photos/bio.jpg" alt="alt text" width="146px" height="200px" />&nbsp;</td>
<td align="left"><p>Yiping Wang<br />
Ph.D student<br /> <a href="https://www.cs.washington.edu/">Paul G. Allen School of Computer Science &amp; Engineering</a>, <br />
<a href="https://www.washington.edu/">University of Washington</a><br />
Email: ypwang61 at cs dot washington dot edu <br />
<a href="https://scholar.google.com/citations?user=IuMFxFUAAAAJ&amp;hl=en&amp;oi=ao">Google Scholar</a><br />
<a href="https://twitter.com/ypwang61">Twitter</a> </p>
</td></tr></table>
<h2>About me</h2>
<p>I'm a second-year Ph.D. student in Paul G. Allen School of Computer Science &amp; Engineering from University of Washington.
I feel very fortunate to have worked under the guidance of <a href="https://simonshaoleidu.com/index.html">Prof. Simon Shaolei Du</a> since 2022 summer.</p>
<p>My main research interest broadly spread across <b>machine learning theory</b> and <b>foundation models</b>.
For the theortical part, I care about understanding the foundations of deep learning and representation learning, especially the <b>training dynamics of</b> the basic components like <b>Transformer</b>.
For the empirical part, I am keen on developing efficient algorithms with strong theoretical guarantees or insightful observations. Currently, in this aspect, I'm working on <b>data selection/scheduling for multi-modal pretraining</b> and improving inference efficiency of LLM. I'm also working on some projects related to video generation.
In addition, I have always held a strong enthusiasm for understanding the essence of intelligence and exploring the cross-cutting areas of mathematics, physics, and AGI, such as using LLMs for mathematical proof and seeking scientific truth.</p>
<p>I'm grateful to all my collaborators and mentors along the way.
I'm priviledged to be working closely with <a href="http://yuandong-tian.com/">Dr. Yuandong Tian</a> since 2023 spring.
Besides, I'm also having intern at Microsoft started from June 2024, fortunate to be advised by <a href="https://scholar.google.com/citations?user=S6OFEFEAAAAJ">Yelong Shen</a> and <a href="https://sites.google.com/site/shuohangsite/">Shuohang Wang</a>.
During my undergraduate, I was fortunate to work closely with <a href="https://www.huaxiuyao.io/">Prof. Huaxiu Yao</a> and <a href="https://linjunz.github.io/">Prof. Linjun Zhang</a>.</p>
<p>Previously, I studied Computer Science and Mathematics in <a href="https://www.zju.edu.cn/english/">Zhejiang University</a>, got an honors degree from <a href="http://ckc.zju.edu.cn/ckcen/_t1906/main.psp">Chu Kochen Honors College</a>.</p>
<h2>News</h2>
<ul>
<li><p>09/2024: Our <a href="https://arxiv.org/abs/2405.19547">negCLIPLoss</a> paper is accepted by NeurIPS 2024 as spotlight!</p>
</li>
<li><p>06/2024: Started my internship at Microsoft!</p>
</li>
<li><p>01/2024: One paper (<a href="https://arxiv.org/abs/2310.00535">JoMA</a>) is accepted by ICLR 2024!</p>
</li>
<li><p>12/2023: Attended NeurIPS 2023 at New Orleans.</p>
</li>
<li><p>09/2023: One paper (<a href="https://arxiv.org/abs/2305.16380">Scan&amp;Snap</a>) is accepted by NeurIPS 2024!</p>
</li>
<li><p>09/2023: Become a husky in UW!</p>
</li>
</ul>
<h2>My Favourite Papers</h2>
<p><span class="preserve-space">(* denotes equal contribution or alphabetic ordering.)</span> <br /><br /></p>
<p><span class="topic-head">Data Selection Algorithm</span></p>
<p><div class="boxed">
We studied how to efficiently select data for multimodal pretraining tasks, drawing inspiration from both empirical observations and theoretical insights.<br /></p>
<table class="imgtable"><tr><td>
<img src="photos/negcliploss.png" alt="alt text" width="400px" height="180px" />&nbsp;</td>
<td align="left"><p><b><a href="https://arxiv.org/abs/2405.19547">CLIPLoss and Norm-Based Data Selection Methods for Multimodal Contrastive Learning</a></b> <span class="preserve-space"> </span><a href="https://github.com/ypwang61/negCLIPLoss_NormSim">[Code]</a> <a href="./pdfs/Poster_negCLIPLoss_NormSim.pdf">[Poster]</a> <a href="https://twitter.com/ypwang61/status/1798396572516151612">[Twitter]</a> <a href="https://arxiv.org/abs/2402.02055">[Previous Versions]</a> <br />
<b>Yiping Wang</b>*, Yifang Chen*, Wendan Yan, Alex Fang, Wenjing Zhou, Kevin Jamieson, Simon S. Du <br />
<i>NeurIPS 2024 (<font color="red">Spotlight</font>)</i><br /><br />
tl;dr: We design universal data selection methods for CLIP pretraining and achieve near SOTA results with less than 10% of preprocessing resources. It can obtain a new SOTA in <a href="https://www.datacomp.ai/dcclip/leaderboard.html">DataComp benchmark</a> when combined with other approaches.</p>
</td></tr></table>
<p></div></p>
<p><span class="topic-head">Training Dynamics of Transformer</span></p>
<p><div class="boxed">
We attempted to analyze the training dynamics of transformers in a mathematical way.<br /></p>
<table class="imgtable"><tr><td>
<img src="photos/scan.png" alt="alt text" width="400px" height="130px" />&nbsp;</td>
<td align="left"><p><b><a href="https://arxiv.org/abs/2305.16380">Scan and Snap: Understanding Training Dynamics and Token Composition in 1-layer Transformer</a></b> <span class="preserve-space"> </span> <a href="./pdfs/poster_scan_snap.pdf">[Poster]</a> <a href="https://twitter.com/tydsh/status/1663611845603885056">[Twitter]</a><br />
Yuandong Tian, <b>Yiping Wang</b>, Beidi Chen, Simon S. Du <br />
<i>NeurIPS 2023</i><br />
<i><font color="red"> Oral </font> presentation at High-dimensional learning dynamics workshop @ ICML 2023</i> <br /><br />
tl;dr: We analyze the 1-layer transformer with next token prediction loss, and rigorously prove its training process and reveal how the token is combined via self-attention layer and the nature of its inductive bias.</p>
</td></tr></table>
<table class="imgtable"><tr><td>
<img src="photos/joma.png" alt="alt text" width="400px" height="130px" />&nbsp;</td>
<td align="left"><p><b><a href="https://arxiv.org/abs/2310.00535">JoMA: Demystifying Multilayer Transformers via JOint Dynamics of MLP and Attention</a></b> <span class="preserve-space"> </span><a href="https://twitter.com/tydsh/status/1709785496056930654">[Twitter]</a><br />
Yuandong Tian, <b>Yiping Wang</b>, Zhenyu Zhang, Beidi Chen, Simon S. Du <br />
<i>ICLR 2024</i> <br /><br />
tl;dr: We analyze the training dynamics of multilayer transformer, characterizing the role of self-attention, MLP nonlinearity, and the learning procedure of hierarchical structure, if the data follow hierarchical generative models.</p>
</td></tr></table>
<p></div></p>
</td>
</tr>
</table>
</body>
</html>
82 changes: 82 additions & 0 deletions index2.jemdoc
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
# jemdoc: menu{MENU}{index.html}, nofooter
==Yiping Wang 王宜平

~~~
{}{img_left}{photos/bio.jpg}{alt text}{146}{200}
Yiping Wang\n
Ph.D student\n [https://www.cs.washington.edu/ Paul G. Allen School of Computer Science & Engineering], \n
[https://www.washington.edu/ University of Washington]\n
Email: ypwang61 at cs dot washington dot edu \n
[https://scholar.google.com/citations?user=IuMFxFUAAAAJ&hl=en&oi=ao Google Scholar ]\n
[https://twitter.com/ypwang61 Twitter]
~~~

== About me
I'm a second-year Ph.D. student in Paul G. Allen School of Computer Science & Engineering from University of Washington.
I feel very fortunate to have worked under the guidance of [https://simonshaoleidu.com/index.html Prof. Simon Shaolei Du] since 2022 summer.

My main research interest broadly spread across *machine learning theory* and *foundation models*.
For the theortical part, I care about understanding the foundations of deep learning and representation learning, especially the *training dynamics of* the basic components like *Transformer*.
For the empirical part, I am keen on developing efficient algorithms with strong theoretical guarantees or insightful observations. Currently, in this aspect, I'm working on *data selection/scheduling for multi-modal pretraining* and improving inference efficiency of LLM. I'm also working on some projects related to video generation.
In addition, I have always held a strong enthusiasm for understanding the essence of intelligence and exploring the cross-cutting areas of mathematics, physics, and AGI, such as using LLMs for mathematical proof and seeking scientific truth.

I'm grateful to all my collaborators and mentors along the way.
I'm priviledged to be working closely with [http://yuandong-tian.com/ Dr. Yuandong Tian] since 2023 spring.
Besides, I'm also having intern at Microsoft started from June 2024, fortunate to be advised by [https://scholar.google.com/citations?user=S6OFEFEAAAAJ Yelong Shen] and [https://sites.google.com/site/shuohangsite/ Shuohang Wang].
During my undergraduate, I was fortunate to work closely with [https://www.huaxiuyao.io/ Prof. Huaxiu Yao] and [https://linjunz.github.io/ Prof. Linjun Zhang].

Previously, I studied Computer Science and Mathematics in [https://www.zju.edu.cn/english/ Zhejiang University], got an honors degree from [http://ckc.zju.edu.cn/ckcen/_t1906/main.psp Chu Kochen Honors College].


== News
- 09/2024: Our [https://arxiv.org/abs/2405.19547 negCLIPLoss] paper is accepted by NeurIPS 2024 as spotlight!
- 06/2024: Started my internship at Microsoft!
- 01/2024: One paper ([https://arxiv.org/abs/2310.00535 JoMA]) is accepted by ICLR 2024!
- 12/2023: Attended NeurIPS 2023 at New Orleans.
- 09/2023: One paper ([https://arxiv.org/abs/2305.16380 Scan&Snap]) is accepted by NeurIPS 2024!
- 09/2023: Become a husky in UW!


== My Favourite Papers
{{<span class="preserve-space">(* denotes equal contribution or alphabetic ordering.)</span>}} \n\n


{{<span class="topic-head">Data Selection Algorithm</span>}}
{{<div class="boxed">}}
We studied how to efficiently select data for multimodal pretraining tasks, drawing inspiration from both empirical observations and theoretical insights.\n

~~~
{}{img_left}{photos/negcliploss.png}{alt text}{400}{180}
*[https://arxiv.org/abs/2405.19547 CLIPLoss and Norm-Based Data Selection Methods for Multimodal Contrastive Learning]* {{<span class="preserve-space"> </span>}}[https://github.com/ypwang61/negCLIPLoss_NormSim \[Code\]] [./pdfs/Poster_negCLIPLoss_NormSim.pdf \[Poster\]] [https://twitter.com/ypwang61/status/1798396572516151612 \[Twitter\]] [https://arxiv.org/abs/2402.02055 \[Previous Versions\]] \n
*Yiping Wang*\*, Yifang Chen\*, Wendan Yan, Alex Fang, Wenjing Zhou, Kevin Jamieson, Simon S. Du \n
/NeurIPS 2024 ({{<font color="red">Spotlight</font>}})/\n\n
tl;dr: We design universal data selection methods for CLIP pretraining and achieve near SOTA results with less than 10% of preprocessing resources. It can obtain a new SOTA in [https://www.datacomp.ai/dcclip/leaderboard.html DataComp benchmark] when combined with other approaches.
~~~

{{</div>}}


{{<span class="topic-head">Training Dynamics of Transformer</span>}}
{{<div class="boxed">}}
We attempted to analyze the training dynamics of transformers in a mathematical way.\n

~~~
{}{img_left}{photos/scan.png}{alt text}{400}{130}
*[https://arxiv.org/abs/2305.16380 Scan and Snap: Understanding Training Dynamics and Token Composition in 1-layer Transformer]* {{<span class="preserve-space"> </span>}} [./pdfs/poster_scan_snap.pdf \[Poster\]] [https://twitter.com/tydsh/status/1663611845603885056 \[Twitter\]]\n
Yuandong Tian, *Yiping Wang*, Beidi Chen, Simon S. Du \n
/NeurIPS 2023/\n
/{{<font color="red"> Oral </font>}} presentation at High-dimensional learning dynamics workshop @ ICML 2023/ \n\n
tl;dr: We analyze the 1-layer transformer with next token prediction loss, and rigorously prove its training process and reveal how the token is combined via self-attention layer and the nature of its inductive bias.
~~~

~~~
{}{img_left}{photos/joma.png}{alt text}{400}{130}
*[https://arxiv.org/abs/2310.00535 JoMA: Demystifying Multilayer Transformers via JOint Dynamics of MLP and Attention]* {{<span class="preserve-space"> </span>}}[https://twitter.com/tydsh/status/1709785496056930654 \[Twitter\]]\n
Yuandong Tian, *Yiping Wang*, Zhenyu Zhang, Beidi Chen, Simon S. Du \n
/ICLR 2024/ \n\n
tl;dr: We analyze the training dynamics of multilayer transformer, characterizing the role of self-attention, MLP nonlinearity, and the learning procedure of hierarchical structure, if the data follow hierarchical generative models.
~~~

{{</div>}}


26 changes: 26 additions & 0 deletions jemdoc.css
View file
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ body {
margin: 0;
}


#layout-menu {
background: #f6f6f6;
border: 1px solid #dddddd;
Expand Down Expand Up @@ -365,3 +366,28 @@ table.imgtable, table.imgtable td {
border: none;
text-align: left;
}


.letter-spacing {
letter-spacing: 0.5em;
}

.preserve-space {
white-space: pre;
}


.boxed {
border: 1px solid #dddddd; /* 灰色边框 */
background-color: #ffffee; /* 浅黄色背景 */
padding: 12px 12px 0px 12px; /* 内边距 */
margin-bottom: 5px; /* 下方留一些空白 */
border-radius: 5px; /* 可选：让边框有一些圆角 */
}

.topic-head {
font-size: 1.05em;
font-weight: bold;
color:#cf5732;
margin-bottom: -2em;
}
53 changes: 53 additions & 0 deletions miscellaneous.html
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
"http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en">
<head>
<meta name="generator" content="jemdoc, see http://jemdoc.jaboc.net/" />
<meta http-equiv="Content-Type" content="text/html;charset=utf-8" />
<link rel="stylesheet" href="jemdoc.css" type="text/css" />
<title>Miscellaneous</title>
</head>
<body>
<table summary="Table for page layout." id="tlayout">
<tr valign="top">
<td id="layout-menu">
<div class="menu-category">Yiping Wang</div>
<div class="menu-item"><a href="index2.html">Home</a></div>
<div class="menu-item"><a href="pub.html">Publications</a></div>
<div class="menu-item"><a href="miscellaneous.html" class="current">Miscellaneous</a></div>
<div class="menu-item"><a href="fun.html">Fun</a></div>
<div class="menu-item"><a href="CV_YipingWang_phd.pdf">CV</a></div>
</td>
<td id="layout-content">
<div id="toptitle">
<h1>Miscellaneous</h1>
</div>
<h2>Internship</h2>
<ul>
<li><p>06/2024 - Present: Research Intern @ Microsoft, Weizhu Chen's Group<br />
Mentor: <a href="https://scholar.google.com/citations?user=S6OFEFEAAAAJ">Yelong Shen</a> and <a href="https://sites.google.com/site/shuohangsite/">Shuohang Wang</a><br />
Project: Video Generation</p>
</li>
</ul>
<h2>Teaching</h2>
<ul>
<li><p>TA in <a href="https://courses.cs.washington.edu/courses/cse543/24au/">CSE 543 Deep Learning (24Au)</a></p>
</li>
</ul>
<h2>Services</h2>
<ul>
<li><p>Paper Reviewer: NeurIPS 2023, ICLR 2024, ICML 2024, NeurIPS 2024</p>
</li>
<li><p>UW CSE Ph.D. Admission Reviewer: 2024</p>
</li>
</ul>
<h2>Honors and Awards</h2>
<ul>
<li><p>12/2022: <i>Chu Kochen Scholarship</i> in Zhejiang University</p>
</li>
</ul>
</td>
</tr>
</table>
</body>
</html>
Expand Down
18 changes: 18 additions & 0 deletions miscellaneous.jemdoc
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
# jemdoc: menu{MENU}{miscellaneous.html}, nofooter
==Miscellaneous

== Internship
- 06\/2024 - Present: Research Intern @ Microsoft, Weizhu Chen's Group\n
Mentor: [https://scholar.google.com/citations?user=S6OFEFEAAAAJ Yelong Shen] and [https://sites.google.com/site/shuohangsite/ Shuohang Wang]\n
Project: Video Generation

== Teaching
- TA in [https://courses.cs.washington.edu/courses/cse543/24au/ CSE 543 Deep Learning (24Au)]

== Services
- Paper Reviewer: NeurIPS 2023, ICLR 2024, ICML 2024, NeurIPS 2024
- UW CSE Ph.D. Admission Reviewer: 2024

== Honors and Awards
- 12\/2022: /Chu Kochen Scholarship/ in Zhejiang University

Loading

0 comments on commit 684f886

Please sign in to comment.