phylociraptor v1.0.0

Changes since last release:
199abb3 bump version to 1.0.0
de123b3 Few more README updates
96621c3 Some updates to README
9a3d1c8 prepare for release
7fa8906 Small updates to small-example
9a7a130 Better determination of size of PDF. Display none when outgroup could not be inferred.
72d7d2e Update plot-heatmap to allow subsamling of trees via the treefile
b37c1fe Adjust for new layer of hashes in filter-align. Improve terminal output of script.
8a2c267 play with formatting of minimal tutorial
c3c4593 play with formatting of minimal tutorial
d811a07 play with formatting of minimal tutorial
74f6bb7 play with formatting of minimal tutorial
251f022 update minimal tutorial
a7df151 comment out devel parts of bayesian inference
dfce34e Check that alignment statistics file is there before trying to create plots
32f0d41 Small update to test_cases
30c13d1 Fix problem where values in similarity would get different colors despite being the same. Add option to plot-heatmap to change the number of displayed decimal places
06fc73a Fix rare case where nodes in tree do not have support value in plot-conflict.
d28aab3 Fix naming of trees in conflict plot without lineage information
c411c66 Update minimal test-case config file to new format
188e598 Give a hint that snakemake is missing in hashing library import.
c14d56b Update report for new file paths in the new hashing layer
9e41207 Slightly improve check to also show output locations and some cleanup
e6e3c48 Fix typo
f8ec7e0 introduce 2nd hash layer in filter-align to allow separate hashes for trimming and filtering
11b3def Revert some changes in preparation of more complete fix of filter-align hashing
95361ce Add rcv to filter-align and subsequent steps. Make sure filtering parameters are properly included in hash calculation.
052bd38 Fix header in plot-heatmap
ba414f2 Fix a problem with reproducibility of estimate-conflict by sorting the list of taxa before they are shuffled.
c75de1c Fix bug in how the partition_alignment rule determines the correct hash for the best models file. Update for concat with new rcv functionality
6238e90 Fix problem with orthology when no augustus training species is provided with busco 3.
d3b9dd5 Add possibility to calculate rcv (relative coomposition variability), filter alignments and display in report accordingly
b4236df Update how genome statistics file is written due to NCBI format change
64347ac Fix problem with hashing when only one or two aligners are specified
99fa819 Fix handling of exclude-orthlogy when empty string is provided
8e56b9b Clean up output and error display of different verbosity levels a bit
f1337cd add trimming,min_parsimony_sites to read_params_per filter-align
d1d99e2 add --config-file option to check usage
8d70707 fix none when file to read in during hashing
c64c2ab Reverse None and empty strings in check to make written yaml configs more compatible
9313e50 Attempt to fix hashing when empty strings or None is specified
c3aa812 Add small test case used in manuscript
f6e13ef Different parsing of NCBI genome overview file. Has the format changed?
22aedba Fix bug in compare plot for tree plotting. Add more sensitive size estimation for plot size
3c27f82 Add additional figures to filtered-alignments tab in report
02857e2 Add progress bar to report command
be5c4bf Fix error when no raxml trees are found in hashing dict
50921bc Reformat debug output for snakemake command
a3bb340 Catch missing python module error in lib
f17eda5 Add explicit calculation of random seed to plot-conflict and plot-tree
b444763 Determine and display random seed explicitly if user does not specify one in estimate-conflict
9336415 Move speciestree params hash file to phylogeny folder!

test-cases-release

This release includes the version of phylociraptor used for the fungal and vertebrate test-cases mentioned in the online methods of the phylociraptor manuscript. Use this release to recreate the results of these two examples.

For general use it is recommended to use the current version (version >1.0.0) since it includes additional features and bug fixes.

phylociraptor v0.9.13 hashes

Changes since last release:
ffdbaf5 bump version to 0.9.13
97206e1 Comment out debug related output in report
2a8fb86 Several small fixes for hash calculation
3bfc669 Remove and recalculate duplicated quartets before writting output files so that number specified by -n is correct
24a7aae Make both trees in conflict plot face to the right
ef81ee1 Display raxml tree with support values in the report
3c403c6 Add hashing to t2t pca. Small code cleanup
a5468b7 Add hashing support to heatmap plotting
1d360be Update gitignore
3e8e982 Fix naming of trees for conflict plots
9c54c67 Add hashing to plot-tree
a2dc1e4 Fix up get-lineage for hashing
e0cae81 Add hashing functionality to figure report
36f9b2d Remove more testing output from report
6a74f46 Fix two small problems with how trimmed alignments are represented in the report
8688b31 First set of changes to incorporate hashing into the HTML report. This should generally work now.
4a1a44d Reorganize how phylogeny results are stored in results
ca85109 parse-log.py can not run inside a container.
b2df5e0 Rename some rulefiles for clarity
ae5f42a Fix some additional small problems with check and make yaml parameter files have a uniform prefix (parameters)
facde3f Fix small error in check for njtree
6ef125c Modify phylociraptor check to take into account analysis hashes. Also support hashes in initial checks for different runmodes.
327b646 Move hash functionality functions into libphylociraptor to be able to reuse them in check and other modules
758327e Change modify-busco script so that it creates new directories for each modified busco set instead of overwriting the original directory.
b817be9 Fix parsing of full busco table for busco 3
9c098f7 Fix hash calculation if no mode column is specified
39bb602 Fix error when no exclude files is specified or the file does not exist
3e5224d move function to make more general, control ruleorder
a18f4fd modeltest - add checkpoints from previous stage as input and control rule order
80a31cc add checkpoints from previous stage as input and control rule order
3c34773 add checkpoints from previous stage as input
d6c298b add checkpoint from previous stage as input
c5c5f6f make sure file is only read if it's there; move get_modeltest_checkpoint function to here
e5bb77a use rsync instead
8651061 Treat nodes without support values as unsupported in plot tree. Fixes #27
be9dc46 Small fixes to phylociraptor script.
eb8927a update fix aa also in second create_sequence_files occasion
52f8a2a hash functionality for speciestree section
58e6e9e hash functionality for njtree section
5a33252 hash functionality for mltree section
781cc5d hash functionality for modeltest section
6927b18 hash functionality for filter-align section
740e609 hash functionality for align section
420e00f hash functionality for filter-orthology section
3ff4e2d hash functionality for orthology section
ad48b8f config template update to structure used in hash branch
b78142e new script for reading and writing of yaml paramter files
5c68f38 add hash functions to functions.smk
8f0f843 export name of config file as environmental variable
322c3d2 add functions for hash functionality
6da485e update setup to new config file structure
47c95bd updated structure of config file
63196cf remove unneccesary outputs from raxml rule
3e1f6ab replace the rare aminoacid 'U' with 'X' since raxml can't handle
85e9a4b Fix small issue in how util plot-heatmap is called
44e32ce Capture additional R console output in plot scripts
4a54043 Fix bug in check related to new --verbose and --debug
5681db1 Improve behavior of --verbose and --debug.
dd200d2 Change behavior of phylociraptor --verbose. Now only a reduced set of the whole output is given, use --debug to see all output
0145064 Run parse-log.py inside a container to prevent python problems
6d9f7a6 Fix small error in how finished raxml trees are identified in check.
d5ada65 Add possibility to plot only a single uncollapsed tree and plot title and additional information to util plot-tree.
c227474 Fix error when trees for plot-conflict are not provided in correct order
1681241 Change colors in plot-conflict plots with no lineage info to match plots with lineage info.
9dcc16b Much improved plot-conflict utility: More information, colors instead of line thickness, better script output.
5fd8af4 Add possibility to provide a list of tips to estimate-conflict
b2aaf6a Separate plot-tree and plot-conflict scripts

phylociraptor v0.9.12

Changes since last release:
6b64f0a bump version to 0.9.12
367c094 Improved plotting of quartet similarity heatmaps
b7bd491 Additional fix to plot-tree. Now missing and singletons and taxonomy bars should work. Be more verbose about paraphyletic groups
7dc193d Fix util get-lineage and take care of some edge cases

phylociraptor v0.9.11

Changes since last release:
5d0f36f bump version to 0.9.11
8f602a0 PCA and heatmap plotting are now separated. Improved naming of respective utils
6efba63 util plot-tree: Fix problem when sample is in lineage file but not in tree. Fix collapsing of groups of samples with missing taxonomy info
06b7f99 Use same colors for tips in plot-tree and plot-conflicts
73f3750 Fix some typos and a little bit of code cleanup
ce4c6d8 Fix error in phylociraptor snakemake version check
21bf18b Small fix for automated release github workflow

phylociraptor v0.9.10

Changes since last release:
1fb5f65 bump version to 0.9.10
441d665 Remove debugging code from plot-tree and other code cleanups
5a1675f Improved (faster, less NCBI queries) lineage retrieval for util get-lineage
39421b1 Display also higher lineage information for singleton taxa in plot-tree
5700c54 Better color selection and smart selection of plot height in plot-tree functionality
052e65a Fix missing root error and plot tips which have missing lineage information in util plot-tree
63d5003 Fix thread handling for raxmlng
0472aca Remove hardcoded email from get_lineage.py in utils
1c71eb6 Add extensive logging to rule download genome overview where it was still missing
4b6a6eb Further tinkering with import check for snakemake
7129b76 Improve snakemake import error handling further
5f526be Add impot check for snakemake package and snakemake version check similar to annocomba
f181858 Push raxmlng container version to 1.1
a4876d0 Fix some broken links in documentation and update some info
989a323 Update README and documentation
ae30095 Add verbose check functionality for mltree
535912d Add direct --unlock functionality for unlocking workdir
904da73 Second fix for util
916e16e Fix error in util usage info

phylociraptor v0.9.9

Changes since last release:
683f9cb bump version to 0.9.9
516648b Add new util function manage-jobs to list and cancel submitted hpc jobs
e14e15f Correct error in cluster-config for SGE file
5ca510e Remove several old config files
669781a Modify cluster log path for test case cluster config files
b016a4d Fix bug in serial-local transition. Better reporting of downloaded singularity containers: Now warning will only displayed if container is actually downloaded
58698c4 Rename serial to local for -t to omit confusion.
2ded668 Change default location of cluster logfiles for better organization of all logfiles
ee31e40 Add better logging and organization of log files
94e2b53 Add missing info to util modify-busco
3979ee2 Add initial logging functionality to phylociraptor script
0d96b6a fix unusual amino acid letters to prevent alicut errors
f87add5 apply exclude filter in filter-orthology
5d95fdc Several small fixes and tweaks
a8e8770 Add missing rulenames to cluster config file templates
7187ea0 make rename assemblies use relative paths or copy if files are 'outside'

phylociraptor v0.9.8

Changes since last release:
498bc98 Bump version to 0.9.8
2ec3c86 make symbolic links via relative paths
49fd666 Fix usageinfo for util
611495c Update documentation
043e897 Add utility module: modify-busco to reduce busco set
fe453f7 Fix parsing of aligners and settings in reports
947670f Help is now displayed correctly when no arguments are given for util modules
b6cc046 Fix reports display of two gene tree plots. Seperator lines are now displayed correctly for alignments, modeltest and gene trees
f3c1cc9 an attempt to generalize the script
3e8f7e0 fix small bug related to long help
8d1b935 Add bmge info to README
b1eb24e Small improvements to plot similarity script to accomodate additional trimmer
e0e8e3b Add new report
cf2c41a Add bmge parameters entry to config files
4467431 Addust report figure script to handle more alignment trimmers. This should work with all combinations now
6293dd0 Add bmge as alignment trimmer option. Basic functionality.

phylociraptor v0.9.7

Changes since last release:
0017e9e Bump version to 0.9.7
8aa7a12 phylociraptor: Fix raxml tree path searching for util
25e5490 plot-tree.R: Fix color generation for < 11 colors)
1f09208 estimate_conflict.py: Code cleanup, add warning when nquartets exceeds total number of possible quartets. Adjust similarity calculation accordingly
543da3e Add missing muscle_parameters in config files of test cases
c7629f9 Update README with util information.
1c57aae Rename flag to specify quartets
eaa8cb0 Code cleanup for plot-tree
b24b8af Improvements for usage information of phylociraptor util
e059c57 Update documentation to include information about phylociraptor util
36e20c2 Extend plot similarity functionality. Now tip2tip distances of trees can be calculated and plotted as PCA
604cb2d Update .gitignore
39d0af2 Refine script. Add possibility to have detailed tree information
16b3965 Add additional usage info
ffd7850 Add help for new plot-similarity util
390c53e Add new util plot-similarity to plot a similarity matrix of quartet differences and prepare for PCA plotting
40e63c1 Remove printing of docker command
607db73 Add R scripts used in phylociraptor util
4d4552c phylocriptor docker now by default updates its image as singularity images are pulled inside of it
f3fefb7 update gitignore
3f65b22 Create phylociraptor_modules to as part of cleaning up code in phylociraptor script. Rename compare_tree.py script
4c11e36 phylociraptor: add wd bindpoint to util calls to prevent singularity problems with phylociraptor-docker on Macs
fbb5fab util: Fix small bug compare_tree.py which lead to trees not being counted correctly
b4b8e14 phylociraptor: make compare-trees available in util
eb4e9f2 Add compare_trees script to util
4f5484c Add phylociraptor util and first module for downloading lineage information
b9b4ae4 report: If random seed is set, use this information also in the report-figure to get consistent colors
b87875a mltree: Separate iqtree from preparing iqtree input. This is now done in a new rule prepare_iqtree
16e79fa Update Documentation
6eb4f4e align: Add support for MUSCLE aligner
47e4295 report/filter-align: Fix small bug causing the filtered alignments table to be messed up in the report.
7a4d5e4 More informative rulenames and cleaned up cluster config files
7804d03 Make parsing of config file parameters with different options easier: Now strings (seperated by space and comma) as well as python lists are accepted
846a7aa report: Fix small error in commandline parsing of report figure
7767fda report: Make sure parsimony informative sites information is taken directly from config.yaml
c3f5d0f Correct typo also in config files for test cases following PR #21
56c9681 capture log / stderr during alignment
82daca2 change input function to return gene id instead of path; don't use 'bc'
963a73f make sure working directory for aliscore is removed if it happens to exist from previous aborted run
bca8039 change input files so that mltree mode only reruns individual combinations rather all in case modeltest is partly rerun
6b18418 fix typo

phylociraptor v0.9.6

Changes since last release:
4f22187 bump version to 0.9.6
b0502e0 phylociraptor: add -F and change the behavior of -f so that -F and -f correspond to the snakemake equivalents.
b54a62b change behavior of aggregate_best_models to take checkpoints as input
fd2e9e7 modeltest: Only create statistics and mean bootstrap for genes with filtered alignments. This should fix #18
2808272 speed up filtering alignments based on nparsimony and nseqs
0a54dca model: Remove maxmem parameter
9ae1507 setup/orthology: Change file extension to fasta instead of fna to avoid confusion if protein files are provided.
4d62154 report: Handle custom config.yaml files for --figure
b6d8604 orthology: Cosmetics for busco mode output
14b57f3 phylociraptor: Modifications to the script to accompany the last commit
1986f46 all modes: Fix problem with custom config.yaml file when data/config.yaml does not exist
19cf799 phylociraptor: Print error message when cluster config file does not exist.
0451e20 orthology: it is now possible to run orthology starting from a set of protein sequences. mode=protein in the samples csv file. This only works when using BUSCO5
95b047b setup: Additional small changes to genome_download.py. Better error messages and removed debug output
f26d8f5 setup: The genome download script should now download older assembly accessions correctly
a800a14 Fix typo in README
927acdb speciestree: Extend use of random seed also to astral
38451d8 phylociraptor: Quickfix for phylociraptor check. Now only the done files will be evaluated for speciestree, njtree and mltree. May have to be revisited. Temporarily fixes #15
29047af Add badges to readme
a5994ba orthology: Double check names in CSV file with names already in resulst/assemblies to identify better if samples have been removed and prevent orthology runs for samples not in CSV file.
cd1d65a Add license
2513ced tree: remove dedicated maxmem option from config file for iqtree. This can still be passed on to iqtree with additional_params
988c234 orthology: When BUSCO5 is specified the default behaviour is now to use metaeuk. This is also the deafult of BUSCO5. By specifying an augustus species this is overwritten and augustus is used instead. This does not alter the BUSCO3 behaviour where an augustus species is still mandatory
ecb5115 config: Add new seed option for reproducibility of iqtree and raxml. Remove -redo from iqtree as default option. Change iqtree threading to -nt threads instead of -nt AUTO and maxthreads
d06b88c create one central place to change all containers: data/containers.yaml
4dbe6b5 minimal-example: Fix typos in commands