Release v2.1.0
By downloading, installing or using the software, you accept and agree to be bound by all of the terms and conditions of the LICENSE and DISCLAIMER.
Major Features and Improvements
Arch
- Some bugs fixed for spark computing engine
Component
- Unified IO keys naming format for all components
- Add LLMLoader to support running FATE-LLM v2.0 with pipeline
OSX
- Compatible with eggroll-v2.x
- add 2.x api backport support
- bug fix
-
Improved the display issue of output data.
-
Enhanced the PyPI package: configuration files have been relocated to the user's home directory, and the relative paths for uploading data are based on the user's home directory.
-
Added support for running FATE algorithms with Spark + Hadoop.
-
Fixed an issue where failed tasks could not be retried.
-
Fixed an issue where the system couldn't run when the task cores exceeded the system total cores.
- Pipeline: add supports for fate-llm 2.0
- newly added LLMModelLoader, LLMDatasetLoader, LLMDataFuncLoader
- newly added configuration parsing of seq2seq_runner and ot_runner
- Pipeline: unified input interface of components
- Adapt to fate-v2.0 framework:
- Migrate parameter-efficient fine-tuning training methods and models.
- Migrate Standard Offsite-Tuning and Extended Offsite-Tuning(Federated Offsite-Tuning+)
- Newly trainer,dataset, data_processing function design
- New FedKSeed Federated Tuning Algorithm: train large language models in a federated learning setting with extremely low communication cost
- Add Support for Job Runtime Configuration