update page

index.html

@@ -31,35 +31,41 @@ <h1>Yiping Wang 王宜平</h1>
 <h2>About me</h2>
 <p>I'm a first-year Ph.D. student in Paul G. Allen School of Computer Science &amp; Engineering from University of Washington. 
 I feel very fortunate to have worked under the guidance of <a href="https://simonshaoleidu.com/index.html">Prof. Simon Shaolei Du</a> since 2022 summer.</p>
-<p>My main research interest lies in <b>machine learning theory</b>, especially the foundations of deep learning and representation learning. 
-I am also keen on developing practical machine learning algorithms with strong theoretical guarantees, and currently I'm working on designing <b>data selection methods for training foundational models</b>.
-Furthermore, I always hold a strong enthusiasm for understanding the essence of intelligence and exploring the cross-cutting areas of mathematics, physics, and AI.</p>
+<p>My main research interest broadly spread across <b>machine learning theory</b> and <b>foundation models</b>. 
+For the theortical part, I care about understanding the foundations of deep learning and representation learning, especially the <b>training dynamics of</b> the basic components like <b>Transformer</b>. 
+For the empirical part, I am keen on developing efficient algorithms with strong theoretical guarantees or insightful observations. In this aspect, currently I'm working on <b>data selection/scheduling for multi-modal pretraining</b> and improving model efficiency.
+In addition, I always hold a strong enthusiasm for understanding the essence of intelligence and exploring the cross-cutting areas of mathematics, physics, and AGI, such as using LLM for mathematical proof.</p>
 <p>Previously, I got my bachelor's degree in <a href="http://www.en.cs.zju.edu.cn/">Computer Science &amp; Technology</a> from <a href="https://www.zju.edu.cn/english/">Zhejiang University</a> in 2023, with an honor degree from <a href="http://ckc.zju.edu.cn/ckcen/_t1906/main.psp">Chu Kochen Honors College</a>.
 I also minored in <a href="http://www.math.zju.edu.cn/mathen/main.psp">Mathematics</a> at <a href="https://www.zju.edu.cn/english/">Zhejiang University</a>. 
-During my undergraduate, I was very fortunate to work closely with <a href="http://yuandong-tian.com/">Dr. Yuandong Tian</a>, <a href="https://www.huaxiuyao.io/">Prof. Huaxiu Yao</a>, and <a href="https://linjunz.github.io/">Prof. Linjun Zhang</a> on several research projects.</p>
-<h2>Publications and Preprint </h2>
+During my undergraduate, I was very fortunate to work closely with <a href="http://yuandong-tian.com/">Dr. Yuandong Tian</a>, <a href="https://www.huaxiuyao.io/">Prof. Huaxiu Yao</a>, and <a href="https://linjunz.github.io/">Prof. Linjun Zhang</a> on several exciting research projects and learned a lot.</p>
+<h2>Selected Research</h2>
 <p>*: indicating equal contribution or alphabetic ordering.</p>
 <ol>
-<li><p><a href="https://arxiv.org/abs/2402.02055">Variance Alignment Score: A Simple But Tough-to-Beat Data Selection Method for Multimodal Contrastive Learning</a> <a href="https://github.com/ypwang61/VAS">[Code]</a> <br />
-*<b>Yiping Wang</b>, *Yifang Chen, Wendan Yan, Kevin Jamieson, Simon Du. <br />
-Preprint. <br /></p>
+<li><p><a href="https://arxiv.org/abs/2405.19547">CLIPLoss and Norm-Based Data Selection Methods for Multimodal Contrastive Learning</a> <a href="https://github.com/ypwang61/negCLIPLoss_NormSim">[Code]</a> <br />
+*<b>Yiping Wang</b>, *Yifang Chen, Wendan Yan, Alex Fang, Wenjing Zhou, Kevin Jamieson, Simon Shaolei Du <br />
+Preprint. <br />
+tl;dr: We design universal data selection methods for CLIP pretraining and achieve near SOTA results with less than 10% of preprocessing resources. It can achieve a new SOTA in <a href="https://www.datacomp.ai/dcclip/leaderboard.html">DataComp benchmark</a> when combined with current best approaches.</p>
 </li>
 <li><p><a href="https://arxiv.org/abs/2310.00535">JoMA: Demystifying Multilayer Transformers via JOint Dynamics of MLP and Attention</a> <br />
 Yuandong Tian, <b>Yiping Wang</b>, Zhenyu Zhang, Beidi Chen, Simon Du. <br />
-International Conference on Learning Representations (ICLR) 2024 <br /></p>
+International Conference on Learning Representations (ICLR) 2024 <br />
+tl;dr: We analyze the training dynamics of multilayer transformer, characterizing the role of self-attention, MLP nonlinearity, and the learning procedure of hierarchical structure, if the data follow hierarchical generative models.</p>
 </li>
 <li><p><a href="https://arxiv.org/abs/2305.16380">Scan and Snap: Understanding Training Dynamics and Token Composition in 1-layer Transformer</a> <br />
 Yuandong Tian, <b>Yiping Wang</b>, Beidi Chen, Simon Du. <br />
 Conference on Neural Information Processing Systems (NeurIPS) 2023 <br />
-Selected as  <font color="red"> Oral </font> presentation at High-dimensional learning dynamics workshop at ICML 2023</p>
+Selected as  <font color="red"> Oral </font> presentation at High-dimensional learning dynamics workshop at ICML 2023 <br />
+tl;dr: We analyze the 1-layer transformer with next token prediction loss, and rigorously prove its training process and reveal how the token is combined via self-attention layer and the nature of its inductive bias.</p>
 </li>
 <li><p><a href="https://arxiv.org/abs/2306.02556">Improved Active Multi-Task Representation Learning via Lasso</a> <br />
 <b>Yiping Wang</b>, Yifang Chen, Kevin Jamieson, Simon Du. <br />
-International Conference on Machine Learning (ICML) 2023</p>
+International Conference on Machine Learning (ICML) 2023 <br />
+tl;dr: We improve the sample complexity of active multi-task representation learning by proposing a new LASSO-based strategy.</p>
 </li>
 <li><p><a href="https://arxiv.org/abs/2210.05775">C-Mixup: Improving Generalization in Regression</a> <a href="https://github.com/huaxiuyao/C-Mixup">[Code]</a> <br />
 *Huaxiu Yao, *<b>Yiping Wang</b>, Linjun Zhang, James Zou, Chelsea Finn. <br />
-Conference on Neural Information Processing Systems (NeurIPS) 2022</p>
+Conference on Neural Information Processing Systems (NeurIPS) 2022 <br />
+tl;dr: We propose a simple yet effective data augmentation method to improve generalization on regression tasks.</p>
 </li>
 </ol>
 </td>

index.jemdoc

@@ -15,32 +15,39 @@ Email: ypwang61 at cs dot washington dot edu \n
 I'm a first-year Ph.D. student in Paul G. Allen School of Computer Science & Engineering from University of Washington. 
 I feel very fortunate to have worked under the guidance of [https://simonshaoleidu.com/index.html Prof. Simon Shaolei Du] since 2022 summer.
 
-My main research interest lies in *machine learning theory*, especially the foundations of deep learning and representation learning. 
-I am also keen on developing practical machine learning algorithms with strong theoretical guarantees, and currently I'm working on designing *data selection methods for training foundational models*.
-Furthermore, I always hold a strong enthusiasm for understanding the essence of intelligence and exploring the cross-cutting areas of mathematics, physics, and AI.
+My main research interest broadly spread across *machine learning theory* and *foundation models*. 
+For the theortical part, I care about understanding the foundations of deep learning and representation learning, especially the *training dynamics of* the basic components like *Transformer*. 
+For the empirical part, I am keen on developing efficient algorithms with strong theoretical guarantees or insightful observations. In this aspect, currently I'm working on *data selection/scheduling for multi-modal pretraining* and improving model efficiency.
+In addition, I always hold a strong enthusiasm for understanding the essence of intelligence and exploring the cross-cutting areas of mathematics, physics, and AGI, such as using LLM for mathematical proof.
 
 Previously, I got my bachelor's degree in [http://www.en.cs.zju.edu.cn/ Computer Science & Technology] from [https://www.zju.edu.cn/english/ Zhejiang University] in 2023, with an honor degree from [http://ckc.zju.edu.cn/ckcen/_t1906/main.psp Chu Kochen Honors College].
 I also minored in [http://www.math.zju.edu.cn/mathen/main.psp Mathematics] at [https://www.zju.edu.cn/english/ Zhejiang University]. 
-During my undergraduate, I was very fortunate to work closely with [http://yuandong-tian.com/ Dr. Yuandong Tian], [https://www.huaxiuyao.io/ Prof. Huaxiu Yao], and [https://linjunz.github.io/ Prof. Linjun Zhang] on several research projects.
+During my undergraduate, I was very fortunate to work closely with [http://yuandong-tian.com/ Dr. Yuandong Tian], [https://www.huaxiuyao.io/ Prof. Huaxiu Yao], and [https://linjunz.github.io/ Prof. Linjun Zhang] on several exciting research projects and learned a lot.
 
-== Publications and Preprint 
+== Selected Research
 \*: indicating equal contribution or alphabetic ordering.
-. [https://arxiv.org/abs/2402.02055 Variance Alignment Score: A Simple But Tough-to-Beat Data Selection Method for Multimodal Contrastive Learning] [https://github.com/ypwang61/VAS \[Code\]] \n
-    \**Yiping Wang*, \*Yifang Chen, Wendan Yan, Kevin Jamieson, Simon Du. \n
+. [https://arxiv.org/abs/2405.19547 CLIPLoss and Norm-Based Data Selection Methods for Multimodal Contrastive Learning] [https://github.com/ypwang61/negCLIPLoss_NormSim \[Code\]] \n
+    \**Yiping Wang*, \*Yifang Chen, Wendan Yan, Alex Fang, Wenjing Zhou, Kevin Jamieson, Simon Shaolei Du \n
     Preprint. \n
+    tl;dr: We design universal data selection methods for CLIP pretraining and achieve near SOTA results with less than 10% of preprocessing resources. It can achieve a new SOTA in [https://www.datacomp.ai/dcclip/leaderboard.html DataComp benchmark] when combined with current best approaches.
 . [https://arxiv.org/abs/2310.00535 JoMA: Demystifying Multilayer Transformers via JOint Dynamics of MLP and Attention] \n
     Yuandong Tian, *Yiping Wang*, Zhenyu Zhang, Beidi Chen, Simon Du. \n
     International Conference on Learning Representations (ICLR) 2024 \n
+    tl;dr: We analyze the training dynamics of multilayer transformer, characterizing the role of self-attention, MLP nonlinearity, and the learning procedure of hierarchical structure, if the data follow hierarchical generative models.
 . [https://arxiv.org/abs/2305.16380 Scan and Snap: Understanding Training Dynamics and Token Composition in 1-layer Transformer] \n
     Yuandong Tian, *Yiping Wang*, Beidi Chen, Simon Du. \n
     Conference on Neural Information Processing Systems (NeurIPS) 2023 \n
-    Selected as  {{<font color="red"> Oral </font>}} presentation at High-dimensional learning dynamics workshop at ICML 2023
+    Selected as  {{<font color="red"> Oral </font>}} presentation at High-dimensional learning dynamics workshop at ICML 2023 \n
+    tl;dr: We analyze the 1-layer transformer with next token prediction loss, and rigorously prove its training process and reveal how the token is combined via self-attention layer and the nature of its inductive bias.
 . [https://arxiv.org/abs/2306.02556 Improved Active Multi-Task Representation Learning via Lasso] \n
     *Yiping Wang*, Yifang Chen, Kevin Jamieson, Simon Du. \n
-    International Conference on Machine Learning (ICML) 2023
+    International Conference on Machine Learning (ICML) 2023 \n
+    tl;dr: We improve the sample complexity of active multi-task representation learning by proposing a new LASSO-based strategy.
 . [https://arxiv.org/abs/2210.05775 C-Mixup: Improving Generalization in Regression] [https://github.com/huaxiuyao/C-Mixup \[Code\]] \n
     \*Huaxiu Yao, \**Yiping Wang*, Linjun Zhang, James Zou, Chelsea Finn. \n
-    Conference on Neural Information Processing Systems (NeurIPS) 2022
+    Conference on Neural Information Processing Systems (NeurIPS) 2022 \n
+    tl;dr: We propose a simple yet effective data augmentation method to improve generalization on regression tasks.
+