Repositories list

• cirrus-scripts

  Public
  Scripts for running bitextor/paracrawl/europat jobs on cirrus.ac.uk

  Shell

  •1•7•8•1•Updated Sep 26, 2024Sep 26, 2024

• giashard

  Public
  Sharding program for Paracrawl

  Go

  •1•2•0•0•Updated Sep 17, 2024Sep 17, 2024

• giawarc

  Public
  Processing utilities for Internet Archive

  C++

  •0•1•4•1•Updated Apr 19, 2024Apr 19, 2024

• corset

  Public
  Corset is a web-based data selection portal that helps you getting relevant data from massive amounts of parallel data.

  SCSS

  •

  GNU General Public License v3.0

  •3•17•1•0•Updated Nov 6, 2023Nov 6, 2023

• keops

  Public
  Tool for manual evaluation of parallel sentences.

  PHP

  •

  GNU General Public License v3.0

  •4•14•0•0•Updated Oct 19, 2023Oct 19, 2023

• europat-scripts

  Public
  Scripts for obtaining patent data

  Java

  •2•4•1•1•Updated Apr 14, 2023Apr 14, 2023

• tmxutil

  Public
  Tools to generate & filter Europat tmx files.

  Python

  •

  MIT License

  •1•4•1•0•Updated Jan 17, 2023Jan 17, 2023

• synthesis

  Public
  Data synthesis by contextualizing glossary translations

  Python

  •3•6•0•0•Updated Jul 1, 2021Jul 1, 2021

• opus-train

  Public
  Automate download and training with OPUS corpora

  Shell

  •

  MIT License

  •2•2•0•0•Updated Jan 28, 2021Jan 28, 2021

• human-evaluations

  Public
  Results of the human evaluation

  Rich Text Format

  •3•5•0•0•Updated Dec 9, 2020Dec 9, 2020

• corpus-issues

  Public
  Open here any Paracrawl corpus related issue

  0•0•0•0•Updated Nov 18, 2020Nov 18, 2020

• multilingual-ted

  Public
  Creative Commons Zero v1.0 Universal

  •0•0•0•0•Updated Nov 13, 2020Nov 13, 2020

• b64filter

  Public archive
  Program for operating on one document per Base 64 encoded line files

  Go

  •0•1•1•0•Updated Aug 4, 2020Aug 4, 2020

• Domain_Adaptation

  Public
  InDomain detection is a tool designed to extract in-domain data from a large collections of data.

  Python

  •

  GNU General Public License v3.0

  •1•1•0•0•Updated Jun 5, 2020Jun 5, 2020

• url_language_analysis

  Public
  Python

  •0•0•0•0•Updated Mar 6, 2020Mar 6, 2020

• targeted-crawling

  Public
  Python

  •0•1•0•0•Updated Mar 6, 2020Mar 6, 2020

• go-warc

  Public
  A golang library to work with WARC files from the common crawl

  Go

  •

  GNU General Public License v2.0

  •7•0•0•0•Updated Aug 4, 2019Aug 4, 2019

• extractor

  Public
  C++

  •

  Apache License 2.0

  •4•23•1•0•Updated Nov 29, 2017Nov 29, 2017

• embedding

  Public
  Mine parallel corpora with embeddings

  Perl

  •0•4•0•0•Updated Sep 2, 2017Sep 2, 2017

• DataCollection

  Public
  Data collection, alignment and TAUS repository

  Python

  •

  Apache License 2.0

  •8•8•3•0•Updated Aug 1, 2017Aug 1, 2017