From 74341f8646233b00d838cc4ecdc2699644d1c8e1 Mon Sep 17 00:00:00 2001 From: Michael Xie Date: Tue, 2 Aug 2022 08:05:23 -0700 Subject: [PATCH] mobile friendliness --- _posts/2022-08-01-understanding-incontext.md | 26 +++++++++++++++++--- 1 file changed, 23 insertions(+), 3 deletions(-) diff --git a/_posts/2022-08-01-understanding-incontext.md b/_posts/2022-08-01-understanding-incontext.md index d21de7e..9533c97 100644 --- a/_posts/2022-08-01-understanding-incontext.md +++ b/_posts/2022-08-01-understanding-incontext.md @@ -13,6 +13,26 @@ draft: True + *In this post, we provide a Bayesian inference framework for in-context learning in large language models like GPT-3 and show empirical evidence for our framework, highlighting the differences from traditional supervised learning. This blog post primarily draws from the theoretical framework for in-context learning from [An Explanation of In-context Learning as Implicit Bayesian Inference](https://arxiv.org/abs/2111.02080) [^BI] and experiments from [Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?](https://arxiv.org/abs/2202.12837) [^RRD].* @@ -47,11 +67,11 @@ Two examples of in-context learning, where a language model (LM) is given a list **What can in-context learning do?** On many benchmark NLP benchmarks, in-context learning is competitive with models trained with much more labeled data and is state-of-the-art on LAMBADA (commonsense sentence completion) and TriviaQA (question answering). Perhaps even more exciting is the array of applications that in-context learning has enabled people to spin up in just a few hours, including writing code from natural language descriptions, helping with app design mockups, and generalizing spreadsheet functions: {% figure %} -
+
-
+
-
+
{% endfigure %} In-context learning allows users to quickly build models for a new use case without worrying about fine-tuning and storing new parameters for each task. It typically requires very few training examples to get a prototype working, and the natural language interface is intuitive even for non-experts.