0.10.1

Minor release with minor improvements and bug fixes.

What's Changed

• Bugfix: Delete baseline pvalue from correction method input by @JorgeGabin in #440
• Fix: fix msmarco location by @cmacdonald in #435
• Feature: added corpus_iter for Terrier index by @cmacdonald in #426
• remove sklearn as required dependency by @cmacdonald in #410
• Add troubleshoot for installation and certification error by @Krissy510 in #411
• fix parsing of trecxml topics by @lukaszett in #414
• paired t-tost by @seanmacavaney in #420
• read_results optimization by @seanmacavaney in #421
• pickling QE pipelines to parallelised QE gridsearch by @cmacdonald in #430
• Require Python 3.8 minimum by @cmacdonald in #431
• Bump logback from 1.2.0 to 1.2.13 in /terrier-python-helper by @dependabot
• improved error message pt.apply.query - from #433 by @cmacdonald in #434
• Improved testing of FeaturesBatchRetrieve by @cmacdonald in #437

New Contributors

• @Krissy510 made their first contribution in #411
• @JorgeGabin made their first contribution in #440

Full Changelog: 0.10.0...0.10.1

0.10.0

What's Changed

New Features

• Transformer.__call__ now supports both dataframe and iterdicts by @cmacdonald in #381
• Terrier: Custom stopwords by @cmacdonald in #372
• Terrier: Access the stemmer of Terrier from PyTerrier by @cmacdonald in #382
• Terrier: Improved API for loading Terrier indices into memory by @cmacdonald in #386

Improvements

• added tokenizer as arg for pt.text.sliding by @mihirs16 in #387
• addresses #367 - include qid in pt.apply Exception by @cmacdonald in #370
• addresses #377: pt.apply.query() raises exception if the query column does not exist by @cmacdonald in #380
• let pt.tqdm exist without pt.init() by @cmacdonald in #399
• deprecate pt.Utils by @cmacdonald in #384
• removes two warnings by @cmacdonald in #385
• work on test failure by @cmacdonald in #401
• Test pyterrier with newer Python versions by @cmacdonald in #400
• bump supported Anserini version by @cmacdonald in #406, addresses #404
• Terrier: allow to put term and LexiconEntry into a tuple by @cmacdonald in #369

Bugs:

• stringify properties and controls, addresses #357 by @cmacdonald in #358
• fix bug in metadata size warning by @seanmacavaney in #362

Documentation

• Update pipeline_examples.md by @gurcankavakci in #359
• Fixed typo by @hermlon in #364
• Update ltr.rst by @Hermi-Mire in #371
• Update transformer.rst by @albertoueda in #383
• clarify docstring for indexing with regards to metadata by @lukaszett in #394
• Query Rewriting & Expansion by @cakiki in #402, #403

New Contributors

• @gurcankavakci made their first contribution in #359
• @hermlon made their first contribution in #364
• @Hermi-Mire made their first contribution in #371
• @lukaszett made their first contribution in #394
• @cakiki made their first contribution in #402
• @mihirs16 made their first contribution in #387

Full Changelog: 0.9.2...0.10.0

0.9.2

Minor release with minor improvements and bug fixes.

What's Changed

• add sbert example notebook by @cmacdonald in #344
• Update scikit-learn requirement from the deprecated sklearn, which was causing build errors at some times.
• adding batching operations to apply.generic() and apply.by_query() by @cmacdonald in #351 - thanks to Xun Zhou, University of Michigan via #350
• improve error messages for invalid indexing configurations by @cmacdonald in #349 -- thanks to @maxhenze in #348
• Various empty dataframe fixes by @cmacdonald in #353 -- thanks to report by Prithvijit Dasgupta, University of Michigan in #352
• improved error message for add_ranks by @cmacdonald in #354

Full Changelog: 0.9.1...0.9.2

0.9.1

Bugfix release addressing a problem with pretokenised indices on Windows

What's Changed

• Nofifo pretok indexing fixes by @cmacdonald in #343

Full Changelog: 0.9.0...0.9.1

0.9.0

Significant update - refactoring of public API (e.g. pt.transformer.TransformerBase -> pt.Transformer) and support in the Terrier backend for making indices from pre-tokenised documents. Python 3.10 is now supported.

What's Changed

• fix error in IRDSDataset when a query field is named "query" by @seanmacavaney in #303
• Fix type annotation by @heinrichreimer in #313
• addresses #315 IRDS corpus_iter are not subscriptable by @cmacdonald in #316
• Missing comma in bm25_qe example by @JohnGiorgi in #319
• Argument meta should be supplied as dictionary by @JohnGiorgi in #320
• use Jnius 1.4 by @cmacdonald in #249
• Python 3.10 support by @cmacdonald in #322
• Lz4 support for pt.io.autoopen() by @cmacdonald in #323
• addresses #326 faster version of add_ranks for single queries by @cmacdonald in #327
• addresses #321 pt.apply.doc_score batching by @cmacdonald in #325
• IterDictIndexer can index pre-tokenised documents by @cmacdonald in #328
• Bump logback-core from 1.2.0 to 1.2.9 in /terrier-python-helper by @dependabot in #336
• documenting BM25F controls and tuning by @cmacdonald in #296, addresses #294
• 0.9refactor by @cmacdonald in #314, #339, addresses #271
• pt.Experiment() alters the input measures list to drop "mrt" #301
• Expose Termpipelines in Terrier index backend by @cmacdonald in #338
• pt.rewrite.tokenise() impl by @cmacdonald in #340 addresses #252 #253
• upgraded GitHub actions by @cmacdonald in #341, #342
• fix LTR groupby for xgboost & lightgbm by @cmacdonald in #284

New Contributors

• @heinrichreimer made their first contribution in #313
• @JohnGiorgi made their first contribution in #319

Full Changelog: 0.8.1...0.9.0

0.8.1

Minor release with minor improvements and bug fixes.

What's Changed

• fixed bug with is_transformer by @seanmacavaney in #274
• addresses #275 issue k in kmaxavg, improved testing by @cmacdonald in #276
• defer loading ir_datasets by @seanmacavaney in #280
• Set meta and meta_lengths in constructor by @MWschutte in #282
• Anserini fixes by @cmacdonald in #279, reported by @Azouu
• prevent use of nptyping v2 by @cmacdonald in #291, reported by @tabonnet
• SourceTransformer pass through extra columns, addresses #287 by @cmacdonald in #288, reported by @Xiao0728
• more transformers with repr by @cmacdonald in #289

New Contributors

• @MWschutte made their first contribution in #282

Full Changelog: 0.8.0...0.8.1

0.8.0

PyTerrier 0.8.0 Release Notes

Released on 18/01/2022

What's Changed - Major

• Require Python 3.7 by @cmacdonald in #255
• Deprecate automatic coercion of transformers by @cmacdonald in #258
• introduce pt.Transformer as public API; * pt.transformer.TransformerBase will be deprecated in 0.9; by @cmacdonald in #258
• introduce query biased summarisation - addresses #205 by @cmacdonald in #223, suggested by @adambaker
• provide re-ranking runs from datasets by @seanmacavaney in #262

What's Changed - Minor

• faster testing in Github Actions: focus on requested jnius, rather than changing jnius version 3 times by @cmacdonald in #256
• Faster tests by @cmacdonald in #257
• Use Flake to identify bugs, reduce imports etc by @cmacdonald in #259
• pyterrier loaded message to stderr by @seanmacavaney in #260
• Fix code block in ltr.rst in section Working with Features by @bart-kosmala in #261
• get_dataset() for non-existant irds dataset by @seanmacavaney in #263
• Filter out non-indexed/metadata fields when indexing by @seanmacavaney in #267, reported by @bjoernengelmann in #266
• mirroring of Vaswani dataset files by @seanmacavaney in #268
• pt.io.read_results() can merge topics by @seanmacavaney in #265
• addresses #264, text.scorer() will default to takes='docs' by @cmacdonald in #269
• change paths and exercise names by @cmacdonald in #270

New Contributors

• @bart-kosmala made their first contribution in #261

Full Changelog: 0.7.2...0.8.0

0.7.2

Minor release addressing some useful bug fixes and small features. This is the last release that will support Python 3.6.

What's Changed

• using chunked instead of ichunked - this fixes indexing speed/memory-consumption/crashes with indexing pipelines, by @seanmacavaney in #238
• remove deprecated code by @cmacdonald in #239
• combsum dropping documents not appearing on both sides of + by @cmacdonald in #240
• addresses #203, verbose in pt.Experiment by @cmacdonald in #245
• Py37 minimum warning, addresses #241 by @cmacdonald in #246
• use caching in GitHub Actions by @cmacdonald in #248
• save run files automatically in pt.Experiment #163 by @cmacdonald in #247
• Set dtype for qrels columns at read time in io method by @jjdelvalle in #254
• support meta config in IterDictIndexer constructor, addresses #250, by @cmacdonald in #251

New Contributors

• @jjdelvalle made their first contribution in #254

Full Changelog: 0.7.1...0.7.2

0.7.1

PyTerrier 0.7.1

Minor update to support activities for CIKM 2021 tutorial. In particular:

• pt.debug.print_num_rows() added
• Terrier Data Repository support for TREC Covid test collection.

0.7.0

PyTerrier 0.7.0

Notable update: containing a small number of new features, and numerous other minor updates. New features include new operators on Terrier indices (#196), as well as the ability to express a Terrier weighting model in a Python lambda (#208, #215). A number of upstream fixes in terrier-core supported multi-lingual settings, gridsearch etc. There was also enchanced support for ir_measures

• #227, #230 dataframe type coercion during evaluation
• #226, #228 pt.Experiment consistency in missing qids in topics/qrels/runs
• #224 Using PyTerrier when not connected to Internet
• #220, #221 pt.debug utility transformers
• #219 pt.apply.new_column fails on empty dataframes
• #216 Mirrors for datasets
• #208, #215 Terrier weighting models expressed in Python
• #211 only compute ranks for score column
• #209 pt.apply.by_query verbose support
• #202, #231 pandas 1.3.0 compatibility
• #201 ParallelIndexer should use indexer.setExternalParalllism()
• #200 BatchRetrieve.from_dataset() support for http://data.terrier.org/
• #197 meaningful error message when passing corpus_iter to TRECCollectionIndexer
• #196 add + and len() support on a Terrier index.
• #195 intersect operator returns extra columns
• #192 more documentation on how to tune weightmodels
• #191 Revised GridSearch documentation
• #188 Global properties for BatchRetrieve
• #187 dataset indices with different variants dont get different dirs
• #185 Addititional dataset.get_topicsqrels() to make pt.Experiment faster to write
• #178 rank accuracy under negative scalar multiple
• #177 add a dataset search feature
• #176 Evaluating empty resultset
• #136 more pre-built indices?
• #113 datasets to support mirrors
• #21, #222 Alias BatchRetrieve to TerrierRetrieve