YiVal stands at the intersection of flexibility and adaptability in the AI development landscape. Crafted meticulously for those who seek a tailored experimentation experience, it effortlessly caters to both hands-on developers and those who lean into automation.
At its core, YiVal is more than just a framework—it's a philosophy. A philosophy that believes in offering tools that can be seamlessly molded to fit unique experimentation needs. With a robust architecture and versatile components, Yival ensures that every AI journey, regardless of its complexity, is smooth and efficient.
Web UI
Dive into the world of YiChing and discover your fortune on our index page. A fun and interactive way to get started with Yival.
- Experiment Result Analysis: Gain insights into aggregated outputs for each combination, evaluator outcomes, average latency, token usage, and sample test case results. The best combinations will be highlighted for ease of reference.
- Data Analysis Page: Delve deep into your experiment data, extracting meaningful insights and patterns that can guide further experimentation.
- Detailed Test Results: A granular look at each test case result for every combination, providing a comprehensive understanding of the experiment's outcomes.
- Improver Experiment Result Analysis: After the improvement phase, see the aggregated outputs for each combination. This includes evaluator outputs, average latency, token usage, and sample test cases, with the best combinations highlighted.
- Improver Detailed Test Results: Post-improvement, this page offers a detailed view of each test case result for every combination, showcasing the enhancement in results.
- Export Data: Securely store and export your experiment for future reference or to share with peers.
- Rating: Human touch matters. Add ratings to each test case on the experiment results page based on configurable criteria.
- Interactive Mode: Flexibility at its best. Enter new test cases for combinations, tailoring your experimentation in real-time.
The process starts with the generation of a dataset which can come from multiple sources:
- Specific Data Generator: A defined method or algorithm that automatically churns out data.
- Data Reader: A component that reads data from external places.
- Manual Input: As straightforward as it sounds, data can be added manually.
Once we have our dataset, we form combinations that are pivotal for the subsequent analysis:
- Formed using specific combination creators.
- Defined manually.
This is the heart of the Yival framework. A custom function provided by the user takes in the dataset and combination list to produce valuable insights.
After analysis, the results are subjected to evaluation. Several methodologies can be applied to grasp and gauge the data's behavior deeply.
From the evaluations, the most promising results are selected. This process ensures only the most vital insights are pushed forward.
The selected results are then fine-tuned in this phase. An "Improver" is applied to enhance these results. This stage can loop back to the analysis stage, indicating an ongoing, iterative process of refinement.
flowchart TD
%% Data Generation Stage
A[Start]
A --> |Data Generation| A1[Dataset]
A1 --> A2[DataGenerator]
A2 --> A3[Specific Generator]
A1 --> A4[DataReader]
A4 --> A5[Specific Reader]
A1 --> A6[Manual Input]
%% Create Combinations Stage
A1 --> |Create Combinations| B
B --> B1[Specific Combination Creator]
B --> B2[Set Combinations Manually]
B --> B3[List of Combinations]
%% Evaluate Stage
B3 --> |Analysis| C
C --> C1[User's Function]
C --> C2[Results from Function]
A1 --> C1
%% Evaluator Stage
C2 --> |Evaluation| D
D --> D1[Method 1]
D --> D2[Method 2]
D --> D3[Method 3]
%% Select Stage
D --> |Selection| E
%% Improver Stage
E --> |Improvement| F
F --> F1[Improver]
F1 --> C
%% Styling
style A fill:#f9d77e,stroke:#f96e5b
style B fill:#a1d4c6,stroke:#f96e5b
style C fill:#f6c3d5,stroke:#f96e5b
style D fill:#b2b1cf,stroke:#f96e5b
style E fill:#f9efaa,stroke:#f96e5b
style F fill:#f2a3b3,stroke:#f96e5b
pip install yival
To get started with a demo for basic interactive mode of YiVal, run the following command:
yival demo --basic_interactive
Once started, navigate to the following address in your web browser:
http://127.0.0.1:8073/interactive
Click to view the screenshot
For more details on this demo, check out the Basic Interactive Mode Demo.
yival demo --qa_expected_results
Once started, navigate to the following address in your web browser: http://127.0.0.1:8073/
Click to view the screenshot
For more details, check out the Question Answering with expected result evaluator.