Skip to content

Commit 577c981

Browse files
committed
add group advantage explorer to website
1 parent 45eda17 commit 577c981

File tree

4 files changed

+242
-25
lines changed

4 files changed

+242
-25
lines changed

index.Rmd

Lines changed: 27 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -51,13 +51,13 @@ We also provide documentation on how to annotate new tasks, including the annota
5151

5252
- The code used to generate the Task Space from raw annotations can be found here: [`generate_task_map_from_raw.Rmd`](https://github.com/Watts-Lab/task-mapping/blob/master/analysis/analysis_task_space/generate_task_map_from_raw.Rmd).
5353

54-
# Accessing our Full Reproduction package
54+
# Exploring Our Data
5555

56-
Our full reproduction package is hosted [on GitHub](https://github.com/Watts-Lab/task-mapping).
56+
Please use the interactive tools below to explore the data associated with our paper!
5757

58-
# Visualizing the Task Map
58+
## Task Cluster Explorer
5959

60-
The following interactive visualizer allows you to explore the Task Map in two dimensions using PCA, and to conduct k-means clustering on the underlying 24-dimensional space. You can choose different values of *k*, and mouse over each dot to see the name of the task.
60+
The following interactive visualizer allows you to explore the Task Map in two dimensions using PCA, and to conduct k-means clustering on the underlying 24-dimensional space. You can choose different values of *k*, and mouse over each dot to see the name of the task. You can use this tool to get an intuitive sense of the relationships between tasks.
6161

6262
```{r setup, include=FALSE}
6363
knitr::opts_chunk$set(echo = TRUE)
@@ -73,13 +73,35 @@ library(tidyverse)
7373
knitr::include_app("https://xehu.shinyapps.io/interactive-task-map/")
7474
```
7575

76+
## Group Advantage Explorer (20 Experimental Tasks)
77+
78+
In our paper, we demonstrate the use of the Task Space by conducting a large-scale integrative experiment measuring the phenomenon of *group advantage* --- in which an interacting group outperforms individuals working alone. In our experiment, we define two types of group advantage:
79+
80+
- **Strong Group Advantage** is the ratio between an interacting group's performance to that of the *best* individual in an equivalently-sized "nominal" team (a statistical aggregation of participants who worked alone, used to account for the resource advantage of having more people).
81+
- **Weak Group Advantage** is the ratio between an interacting group's performance to that of a *randomly-selected* individual in an equivalently sized "nominal" team.
82+
83+
Our experiment involved 20 tasks sampled from the task space, which were implemented at three levels of complexity (low, medium, and high) and completed by groups of two different sizes (3 and 6). In the following interactive panel, please explore our data to see how group advantage varies across these experimental conditions. You can toggle between Strong/Weak Advantage, filter by complexity and group size, and hover to see task names.
84+
85+
A key takeaway is that **group advantage is incredibly heterogeneous**; there's no one answer for whether groups outperform individuals. But importantly, these differences are explainable by task features. However, it turns out that the Task Space features can explain 43% of the variance in this phenomenon, demonstrating that our framework can systematically account for variations in group outcomes like this one.
86+
87+
If you are interested in learning more, our data, code, and materials are available in our [GitHub repository](https://github.com/Watts-Lab/task-mapping).
88+
89+
```{r echo=F, out.width="100%"}
90+
knitr::include_app("https://xehu.shinyapps.io/interactive-task-map-group-advantage/")
91+
```
92+
93+
# Accessing our Full Reproduction Package
94+
95+
Our full reproduction package is hosted [on GitHub (https://github.com/Watts-Lab/task-mapping)](https://github.com/Watts-Lab/task-mapping).
96+
7697
# Team
98+
The paper's authors are listed below. For feedback, questions, or suggestions for new tasks and dimensions, please reach out to the Corresponding Authors.
7799
- [Xinlan Emily Hu](https://xinlanemilyhu.com) (Corresponding Author)
78100
- [Mark Whiting](https://whiting.me/)
79101
- [Linnea Gandhi](https://www.linneagandhi.com/)
80102
- [Duncan J. Watts](https://duncanjwatts.com/)
81103
- [Abdullah Almaatouq](http://amaatouq.io/) (Corresponding Author)
82104

83-
This work was also created with the support from many other people, including many research assistants at the University of Pennsylvania, as well as the with labor of Amazon Mechanical Turk workers.
105+
We also acknowledge that this work was created with the support from many other people, including research assistants at the University of Pennsylvania and the labor of Amazon Mechanical Turk workers.
84106

85107
This project is part of the [group dynamics / integrative experiments research](https://css.seas.upenn.edu/project/integrative-experiments/) at the [Computational Social Science Lab at Penn](https://css.seas.upenn.edu). You can learn more about our lab [here](https://css.seas.upenn.edu/people/).

index.html

Lines changed: 62 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -447,33 +447,76 @@ <h1>Annotating New Tasks</h1>
447447
be found here: <a href="https://github.com/Watts-Lab/task-mapping/blob/master/analysis/analysis_task_space/generate_task_map_from_raw.Rmd"><code>generate_task_map_from_raw.Rmd</code></a>.</p></li>
448448
</ul>
449449
</div>
450-
<div id="accessing-our-full-reproduction-package" class="section level1">
451-
<h1>Accessing our Full Reproduction package</h1>
452-
<p>Our full reproduction package is hosted <a href="https://github.com/Watts-Lab/task-mapping">on GitHub</a>.</p>
453-
</div>
454-
<div id="visualizing-the-task-map" class="section level1">
455-
<h1>Visualizing the Task Map</h1>
450+
<div id="exploring-our-data" class="section level1">
451+
<h1>Exploring Our Data</h1>
452+
<p>Please use the interactive tools below to explore the data associated
453+
with our paper!</p>
454+
<div id="task-cluster-explorer" class="section level2">
455+
<h2>Task Cluster Explorer</h2>
456456
<p>The following interactive visualizer allows you to explore the Task
457457
Map in two dimensions using PCA, and to conduct k-means clustering on
458458
the underlying 24-dimensional space. You can choose different values of
459-
<em>k</em>, and mouse over each dot to see the name of the task.</p>
459+
<em>k</em>, and mouse over each dot to see the name of the task. You can
460+
use this tool to get an intuitive sense of the relationships between
461+
tasks.</p>
460462
<iframe src="https://xehu.shinyapps.io/interactive-task-map/?showcase=0" width="100%" height="400px" data-external="1">
461463
</iframe>
462464
</div>
463-
<div id="team" class="section level1">
464-
<h1>Team</h1>
465+
<div id="group-advantage-explorer-20-experimental-tasks" class="section level2">
466+
<h2>Group Advantage Explorer (20 Experimental Tasks)</h2>
467+
<p>In our paper, we demonstrate the use of the Task Space by conducting
468+
a large-scale integrative experiment measuring the phenomenon of
469+
<em>group advantage</em> — in which an interacting group outperforms
470+
individuals working alone. In our experiment, we define two types of
471+
group advantage:</p>
465472
<ul>
466-
<li><a href="https://xinlanemilyhu.com">Xinlan Emily Hu</a>
467-
(Corresponding Author)</li>
468-
<li><a href="https://whiting.me/">Mark Whiting</a></li>
469-
<li><a href="https://www.linneagandhi.com/">Linnea Gandhi</a></li>
470-
<li><a href="https://duncanjwatts.com/">Duncan J. Watts</a></li>
471-
<li><a href="http://amaatouq.io/">Abdullah Almaatouq</a> (Corresponding
472-
Author)</li>
473+
<li><strong>Strong Group Advantage</strong> is the ratio between an
474+
interacting group’s performance to that of the <em>best</em> individual
475+
in an equivalently-sized “nominal” team (a statistical aggregation of
476+
participants who worked alone, used to account for the resource
477+
advantage of having more people).</li>
478+
<li><strong>Weak Group Advantage</strong> is the ratio between an
479+
interacting group’s performance to that of a <em>randomly-selected</em>
480+
individual in an equivalently sized “nominal” team.</li>
473481
</ul>
474-
<p>This work was also created with the support from many other people,
475-
including many research assistants at the University of Pennsylvania, as
476-
well as the with labor of Amazon Mechanical Turk workers.</p>
482+
<p>Our experiment involved 20 tasks sampled from the task space, which
483+
were implemented at three levels of complexity (low, medium, and high)
484+
and completed by groups of two different sizes (3 and 6). In the
485+
following interactive panel, please explore our data to see how group
486+
advantage varies across these experimental conditions. You can toggle
487+
between Strong/Weak Advantage, filter by complexity and group size, and
488+
hover to see task names.</p>
489+
<p>A key takeaway is that <strong>group advantage is incredibly
490+
heterogeneous</strong>; there’s no one answer for whether groups
491+
outperform individuals. But importantly, these differences are
492+
explainable by task features. However, it turns out that the Task Space
493+
features can explain 43% of the variance in this phenomenon,
494+
demonstrating that our framework can systematically account for
495+
variations in group outcomes like this one.</p>
496+
<p>If you are interested in learning more, our data, code, and materials
497+
are available in our <a href="https://github.com/Watts-Lab/task-mapping">GitHub
498+
repository</a>.</p>
499+
<iframe src="https://xehu.shinyapps.io/interactive-task-map-group-advantage/?showcase=0" width="100%" height="400px" data-external="1">
500+
</iframe>
501+
</div>
502+
</div>
503+
<div id="accessing-our-full-reproduction-package" class="section level1">
504+
<h1>Accessing our Full Reproduction Package</h1>
505+
<p>Our full reproduction package is hosted <a href="https://github.com/Watts-Lab/task-mapping">on GitHub
506+
(https://github.com/Watts-Lab/task-mapping)</a>.</p>
507+
</div>
508+
<div id="team" class="section level1">
509+
<h1>Team</h1>
510+
<p>The paper’s authors are listed below. For feedback, questions, or
511+
suggestions for new tasks and dimensions, please reach out to the
512+
Corresponding Authors. - <a href="https://xinlanemilyhu.com">Xinlan
513+
Emily Hu</a> (Corresponding Author) - <a href="https://whiting.me/">Mark
514+
Whiting</a> - <a href="https://www.linneagandhi.com/">Linnea Gandhi</a>
515+
- <a href="https://duncanjwatts.com/">Duncan J. Watts</a> - <a href="http://amaatouq.io/">Abdullah Almaatouq</a> (Corresponding
516+
Author)</p>
517+
<p>We also acknowledge that this work was created with the support from
518+
many other people, including research assistants at the University of
519+
Pennsylvania and the labor of Amazon Mechanical Turk workers.</p>
477520
<p>This project is part of the <a href="https://css.seas.upenn.edu/project/integrative-experiments/">group
478521
dynamics / integrative experiments research</a> at the <a href="https://css.seas.upenn.edu">Computational Social Science Lab at
479522
Penn</a>. You can learn more about our lab <a href="https://css.seas.upenn.edu/people/">here</a>.</p>
Lines changed: 149 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,149 @@
1+
---
2+
output: html_document
3+
runtime: shiny
4+
---
5+
6+
```{r setup, include=FALSE}
7+
knitr::opts_chunk$set(echo = TRUE)
8+
9+
library(ggplot2)
10+
library(plotly)
11+
library(dplyr)
12+
library(tidyr)
13+
library(readr)
14+
library(shiny)
15+
library(rlang)
16+
```
17+
18+
```{r load-data, echo=FALSE, message=FALSE, warning=FALSE}
19+
# Load PCA base
20+
task_map <- read_csv('./outputs/processed_data/task_map.csv')
21+
# Load group advantage at condition level (has task, complexity, playerCount, strong, weak)
22+
ga_cond <- read_csv('./outputs/processed_data/condition_level_group_advantage.csv')
23+
24+
# PCA of task map
25+
pca <- prcomp(task_map[, -1], scale = TRUE)
26+
pca_df <- as.data.frame(pca$x[, 1:2]) %>% mutate(task_pca = task_map$task)
27+
28+
# Normalize task names between sources for robust joining
29+
norm_task <- function(x) {
30+
x <- tolower(x)
31+
x <- gsub('[^a-z0-9]+', ' ', x)
32+
trimws(x)
33+
}
34+
35+
pca_df <- pca_df %>% mutate(task_norm = norm_task(.data$task_pca))
36+
37+
# Map GA task names to task_map names when they differ
38+
# User-provided mapping (GA name -> output/task_map name)
39+
name_map <- c(
40+
'Sudoku' = 'Sudoku',
41+
'Moral Reasoning' = 'Moral Reasoning (Disciplinary Action Case)',
42+
'Wolf Goat Cabbage' = 'Wolf, goat and cabbage transfer',
43+
'Guess the Correlation' = 'Guessing the correlation',
44+
'Writing Story' = 'Writing story',
45+
'Room Assignment' = 'Room assignment task',
46+
'Allocating Resources' = 'Allocating resources to programs',
47+
'Divergent Association' = 'Divergent Association Task',
48+
'Word Construction' = 'Word construction from a subset of letters',
49+
'Whac a Mole' = 'Whac-A-Mole',
50+
'Random Dot Motion' = 'Random dot motion',
51+
'Recall Association' = 'Recall association',
52+
'Recall Word Lists' = 'Recall word lists',
53+
'Typing' = 'Typing game',
54+
'Unscramble Words' = 'Unscramble words (anagrams)',
55+
'WildCam' = 'Wildcam Gorongosa (Zooniverse)',
56+
'Advertisement Writing' = 'Advertisement writing',
57+
'Putting Food Into Categories' = 'Putting food into categories'
58+
)
59+
60+
ga_cond <- ga_cond %>% mutate(
61+
task_mapped = dplyr::recode(.data$task, !!!name_map, .default = .data$task),
62+
task_norm = norm_task(.data$task_mapped)
63+
)
64+
65+
# Ensure complexity is ordered factor
66+
ga_cond <- ga_cond %>% mutate(
67+
complexity = factor(.data$complexity, levels = c('Low', 'Medium', 'High'), ordered = TRUE),
68+
playerCount = as.factor(.data$playerCount)
69+
)
70+
71+
# For default view, compute task-level means (across complexity and group sizes)
72+
ga_task_means <- ga_cond %>%
73+
group_by(.data$task_norm) %>%
74+
summarise(
75+
task = dplyr::first(.data$task_mapped),
76+
strong = mean(.data$strong, na.rm = TRUE),
77+
weak = mean(.data$weak, na.rm = TRUE),
78+
.groups = 'drop'
79+
)
80+
81+
# Join PCA with GA info; keep the mapped task name
82+
pca_ga <- pca_df %>%
83+
inner_join(ga_task_means, by = 'task_norm') %>%
84+
transmute(PC1 = .data$PC1, PC2 = .data$PC2, task_norm = .data$task_norm,
85+
task = .data$task, strong = .data$strong, weak = .data$weak)
86+
87+
# Keep a version with all condition rows for filtering
88+
pca_ga_cond <- pca_df %>% inner_join(ga_cond, by = 'task_norm')
89+
```
90+
91+
```{r ui-server, echo=FALSE}
92+
ui <- fluidPage(
93+
titlePanel('Interactive Task Map — Group Advantage (20 tasks)'),
94+
sidebarLayout(
95+
sidebarPanel(
96+
radioButtons('dv', 'Color by:', choices = c('Strong' = 'strong', 'Weak' = 'weak'), selected = 'strong', inline = TRUE),
97+
selectInput('complexity', 'Complexity:', choices = c('All', 'Low', 'Medium', 'High'), selected = 'All'),
98+
checkboxGroupInput('groupSize', 'Group size:', choices = c('3', '6'), selected = c('3','6'), inline = TRUE),
99+
checkboxInput('showLabels', 'Show labels', value = FALSE),
100+
helpText('Note: Colors are centered at 1. Blue indicates advantage (>1), red indicates disadvantage (<1).')
101+
),
102+
mainPanel(
103+
plotlyOutput('map_plot')
104+
)
105+
)
106+
)
107+
108+
server <- function(input, output) {
109+
# reactive filtered data
110+
filtered_points <- reactive({
111+
dv_col <- input$dv
112+
113+
if (input$complexity == 'All') {
114+
df <- pca_ga %>% mutate(value = .data[[dv_col]])
115+
} else {
116+
df <- pca_ga_cond %>%
117+
filter(.data$complexity == input$complexity) %>%
118+
filter(as.character(.data$playerCount) %in% input$groupSize) %>%
119+
group_by(.data$task_norm, .data$task_mapped, .data$PC1, .data$PC2) %>%
120+
summarise(value = mean(.data[[dv_col]], na.rm = TRUE), .groups = 'drop') %>%
121+
rename(task = .data$task_mapped)
122+
}
123+
124+
df
125+
})
126+
127+
output$map_plot <- renderPlotly({
128+
dv_label <- ifelse(input$dv == 'strong', 'Strong Advantage', 'Weak Advantage')
129+
130+
base <- ggplot() +
131+
geom_point(data = pca_df, aes(x = .data$PC1, y = .data$PC2), color = 'grey70', alpha = 0.5, size = 2) +
132+
theme_minimal() + labs(x = 'PC1', y = 'PC2')
133+
134+
pts <- filtered_points()
135+
136+
g <- base +
137+
geom_point(data = pts, aes(x = .data$PC1, y = .data$PC2, color = .data$value, text = paste0(.data$task, '\n', dv_label, ': ', round(.data$value, 3))), size = 4) +
138+
scale_color_gradient2(name = dv_label, low = '#b2182b', mid = '#f7f7f7', high = '#2166ac', midpoint = 1)
139+
140+
if (isTRUE(input$showLabels)) {
141+
g <- g + geom_text(data = pts, aes(x = .data$PC1, y = .data$PC2, label = .data$task), vjust = -0.8, size = 3)
142+
}
143+
144+
ggplotly(g, tooltip = c('text'))
145+
})
146+
}
147+
148+
shinyApp(ui, server)
149+
```

interactive-task-map.Rmd

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,9 @@ library(e1071)
1111
library(plotly)
1212
library(dplyr)
1313
library(tidyverse)
14+
library(readr)
15+
library(shiny)
16+
library(rlang)
1417
```
1518

1619
```{r load-data, echo=FALSE, message=FALSE, warning=FALSE}
@@ -73,7 +76,7 @@ server <- function(input, output) {
7376
plot_data <- clustering_data$data
7477
7578
ggplotly(
76-
ggplot(plot_data, aes(x = PC1, y = PC2, color = cluster, text = TaskName)) +
79+
ggplot(plot_data, aes(x = .data$PC1, y = .data$PC2, color = .data$cluster, text = .data$TaskName)) +
7780
geom_point(size = 3) +
7881
theme_minimal()
7982
)

0 commit comments

Comments
 (0)