Skip to content

Commit

Permalink
Add table of contents in readme
Browse files Browse the repository at this point in the history
Remove extra text in overview in favour of this
  • Loading branch information
atreyasha committed Mar 15, 2023
1 parent 09d3552 commit c9e0e07
Showing 1 changed file with 11 additions and 2 deletions.
13 changes: 11 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,10 +8,19 @@ This repository functions as the official codebase for the *"PrivacyGLUE: A Benc

PrivacyGLUE is the first comprehensive privacy-oriented NLP benchmark comprising 7 relevant and high-quality privacy tasks for measuring general language understanding in the privacy language domain. We release performances from the BERT, RoBERTa, Legal-BERT, Legal-RoBERTa and PrivBERT pretrained language models and perform model-pair agreement analysis to detect examples where models benefited from domain specialization. Our findings show that PrivBERT, the only model pretrained on privacy policies, outperforms other models by an average of 2–3% over all PrivacyGLUE tasks, shedding light on the importance of in-domain pretraining for privacy policies.

In this readme, we describe the PrivacyGLUE benchmark [tasks](#Tasks-runner) and our current [leaderboard](#Leaderboard-checkered_flag). Following this, we provide instructions to install [dependencies](#Dependencies-mag), [initialize](#Initialization-fire) the repository and [reproduce](#Usage-snowflake) the benchmark results. We direct readers to our paper for further details on our methodologies and results, with citation information found [below](#Citation-classical_building).

*Note that a previous version of this paper was [submitted](https://openreview.net/forum?id=n3xGexO17SJ) to the ACL Rolling Review (ARR) on 16th December 2022 before resubmission to the MDPI Applied Sciences special issue on NLP and applications on 3rd February 2023.*

## Table of Contents

1. [Tasks](#Tasks-runner)
2. [Leaderboard](#Leaderboard-checkered_flag)
3. [Dependencies](#Dependencies-mag)
3. [Initialization](#Initialization-fire)
4. [Usage](#Usage-snowflake)
5. [Notebooks](#Notebooks-book)
5. [Test](#Test-microscope)
5. [Citation](#Citation-classical_building)

## Tasks :runner:

| Task | Study | Type | Train/Dev/Test Instances | Classes |
Expand Down

0 comments on commit c9e0e07

Please sign in to comment.