Skip to content
This repository has been archived by the owner on Apr 17, 2024. It is now read-only.

Feature/ksd-280 #39

Merged
merged 2 commits into from
Sep 5, 2023
Merged

Feature/ksd-280 #39

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
c985cab
feat: update roadmap
dave-at-koor Aug 31, 2023
2a60f44
feat: add link to previous roadmap
dave-at-koor Aug 31, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 7 additions & 11 deletions docs/engineering/development/index.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,24 +24,23 @@ Development happens in cycles. A few key cycles drive our work.

Projects cut across cycles. Larger projects can have milestones that target specific outcomes on a shorter timeline.


## Dev Activities

Our weekly dev cycle includes the following activities.

1. **Unstructured time**
1. **Unstructured time**
- vast majority of the week
- for coding and creating related artifacts: requirements, designs, usage guides, demos, etc.
2. **Planning**
2. **Planning**
- Review projects in Linear.
- Update issue statuses, and make sure everyone has assignments for the next week.
- Avoid having too much or too little work in progress.
3. **[Show and tell](show-and-tell)**
3. **Progress review – [Show and tell](show-and-tell)**
- to show the results for the past week
- Follow the link and review the protocol for sharing your work with the team.
- Help your colleagues by reviewing their work, asking questions, and suggesting improvement ideas.
4. **Retrospective**
- Once a month or as needed (when people want to unpack a particular incident).
4. **Retrospective**
- Once a month or as needed (when people want to unpack a particular incident).
- Reflect on recent past. Acknowledge what is working, and call out what could be better.
5. **Release demos**
- For recording demonstrations of features to share with the world.
Expand All @@ -61,7 +60,6 @@ We use [Linear](https://linear.app/) to manage and record our work.
Our repositories are kept at [GitHub](https://github.com/koor-tech).
<Badge type="warning" text="ToDo" /> Let's build a complete list at some point.


## Development Stages

### Feature requirements and functional design
Expand All @@ -74,13 +72,12 @@ User stories help to define all of the angles of a use case. A story takes the f

> A \<class-of-user\> wants to \<action-to-take\> so that \<desired-effect-of-action\>.

For example:
For example:

> A *salesperson* wants to *keep a list of prospective customers* so that *he or she can send promotional coupons*.
> A _salesperson_ wants to _keep a list of prospective customers_ so that _he or she can send promotional coupons_.

Under that heading, describe all of the functional details the system should handle to make this happen. A functional detail defines what should happen, not how. Leave the question of how to those implementing the features.


### Working on stories

1. When a story is picked up for development, that is the time to ask all of the detailed questions.
Expand All @@ -93,7 +90,6 @@ Under that heading, describe all of the functional details the system should han
2. Do a PR on your story. Make adjustments based on review comments.
3. Stage completed work as you go. We should be able to use all of the completed stories at the end of each cycle.


### Implementation

<Badge type="warning" text="ToDo" /> Make this more complete
Expand Down
80 changes: 80 additions & 0 deletions docs/engineering/product/product-roadmap-previous.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
---
title: Product Roadmap
outline: [2, 3]
---

# Product Roadmap


**Last updated:** 14 August 2023

## Past to Now

Koor's primary product offering has been the Koor Storage Distribution (KSD). Strictly speaking, KSD is a fork of Rook with a handful of helpful scripts and documents added to the mix. More broadly, KSD includes a loosely organized set of features that enhance the experience of using Rook and Ceph for self-managed data storage.

### What we have called KSD

* Koor Storage Distribution (KSD)
* A fork of Rook that is kept at parity with the latest version.
* Koor Operator
* Installs KSD on any Kubernetes cluster, and sets up Rook Ceph storage
* Uses Helm chart or CRDs to define storage
* Checks the status of storage, flagging the user of resources that are configured below minimum thresholds
* Diagnostic scripts
* Gathers debugging information from a running cluster for troubleshooting issues
* Extended Ceph Exporter
* Extends metrics exported from Promethius for improved utilization charts
* Version service
* Checks for newer version of Ceph and Rook that could be adopted

### No adoption so far

The real problem with what we have offered so far is that have had zero adoption that we know about. Perhaps a few have used the Koor Operator without our knowledge. Maybe some have taken a look. Either KSD was not well understood or the benefits were not useful enough to be worth the trouble of adopting or even trying it.

Our customers to date are those who sought help with issues using Rook and Ceph. We are able to resolve issues and recommend improvements, so that has worked out. We need a much larger customer base, and we need customers to use our products.


## Next - Koor Data Control Center

Our new product strategy is to offer a single, multi-faceted product. For now as a working title, let's call it the Koor Data Control Center (KDCC). That's what we are building and offering, even though we may try different names in our marketing and sales packaging.

Simply put, the KDCC is everything you need to see what is happening with your data storage systems and to change the system as needed. It's a single-pane-of-glass view that provides a common user interfaces for many perspectives of data storage operation. It is also offers GUI-based controls for setting up data storage, for tuning a running system, and to assist with larger modifications, like upgrades, expansion, and migrations.

The components and features of the KDCC include the following.

### Koor Dashboard

* A top-level view that shows critical system metrics and active alerts.
* High-level operational charts with drill-downs to see the details of each subsystem.
* Versions of Ceph, Rook, and installed extensions, with information about available updates.

### Koor Controls

* Control panels with editable fields for properties that can be modified.
* Staging for changes that require restart.
* Scrubbing schedules to ensure deep scrubbing happens frequently enough.

### Data Storage Set-up

* Koor Operator and Helm chart for installing the Koor Storage Distribution.
* A selection of templates to match common use cases.
* Automated system upgrades.
* SSO integration.

### Troubleshooting Tools

* Diagnostic scripts to gather data to help pinpoint issues.


## MVP

We do not know what a minimum-viable product is. We have to keep adding features until rates of adoption start to soar.

Rather than worrying about predicting an MVP, let's focus on choosing the next best feature to add and making that as valuable as possible.


## Timeline

We have to hurry. It is critical to put together a functioning alpha version of the KDCC by the end of August, which is about 15 calendar days away. The alpha does not have to do much other than serve as a good basis to continue adding features and functionality for the coming months.

Once the architecture has been reviewed, and we select an initial feature set and define development tasks in Linear. This needs to be our top priority.
122 changes: 81 additions & 41 deletions docs/engineering/product/product-roadmap.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,76 +5,116 @@ outline: [2, 3]

# Product Roadmap

**Last updated:** 31 August 2023

**Last updated:** 14 August 2023
[Previous road map here](./product-roadmap-previous.md)

## Past to Now
## Koor Data Control Center

Koor's primary product offering has been the Koor Storage Distribution (KSD). Strictly speaking, KSD is a fork of Rook with a handful of helpful scripts and documents added to the mix. More broadly, KSD includes a loosely organized set of features that enhance the experience of using Rook and Ceph for self-managed data storage.
Koor's primary product offering is the Koor Data Control Center. This product includes a collection of software projects that work together to make Rook Ceph data storage as easy as possible.

### What we have called KSD
(Internally, we may use the acronyms, KDCC or DCC, but for our website and marketing efforts, we need to spell it out. Data Control Center is immediately understandable, as opposed to DCC or KDCC.)

* Koor Storage Distribution (KSD)
* A fork of Rook that is kept at parity with the latest version.
* Koor Operator
* Installs KSD on any Kubernetes cluster, and sets up Rook Ceph storage
* Uses Helm chart or CRDs to define storage
* Checks the status of storage, flagging the user of resources that are configured below minimum thresholds
* Diagnostic scripts
* Gathers debugging information from a running cluster for troubleshooting issues
* Extended Ceph Exporter
* Extends metrics exported from Promethius for improved utilization charts
* Version service
* Checks for newer version of Ceph and Rook that could be adopted
The Data Control Center is a materialization of the data control plane, which more of an abstract notion. The Data Control Center has everything you need to see what is happening with your data storage systems and to make changes as needed.

### No adoption so far
It's a single-pane-of-glass view that provides a common user interfaces for many perspectives of data storage operation. It is also offers GUI-based controls for setting up data storage, for tuning a running system, and to assist with larger modifications, like upgrades, expansion, and migrations.

The real problem with what we have offered so far is that have had zero adoption that we know about. Perhaps a few have used the Koor Operator without our knowledge. Maybe some have taken a look. Either KSD was not well understood or the benefits were not useful enough to be worth the trouble of adopting or even trying it.
The Data Control Center provides a browser-based user interface to view operating metrics of Rook and Ceph, and to assist with system changes. The UI is backed by APIs for microservices that do the heavy lifting. The services are hosted in the same environment as the data, giving them the access to available information.

Our customers to date are those who sought help with issues using Rook and Ceph. We are able to resolve issues and recommend improvements, so that has worked out. We need a much larger customer base, and we need customers to use our products.
Components of the control center include:

- Koor Storage Distribution (KSD)
- Koor's fork of Rook that is kept at parity with the latest version.
- Includes supplemental proprietary code for Koor users.
- Koor Operator
- Installs KSD on any Kubernetes cluster, and sets up Rook Ceph storage
- Uses Helm chart or CRDs to define storage
- Checks the status of storage, flagging the user when resources are below minimum thresholds
- Version service
- Compares installed and latest versions of software
- Flags the user when Ceph and Rook upgrades can be applied
- _(future)_ Enables scheduled and one-click system upgrades for non-breaking upgrade paths
- Diagnostic scripts
- Gathers debugging information from a running cluster for troubleshooting issues
- Extended Ceph Exporter
- Extends metrics exported from Promethius for improved utilization charts

## Next - Koor Data Control Center
## Control Center Feature Areas

Our new product strategy is to offer a single, multi-faceted product. For now as a working title, let's call it the Koor Data Control Center (KDCC). That's what we are building and offering, even though we may try different names in our marketing and sales packaging.
The Data Control Center UI is organized into the following feature areas.

Simply put, the KDCC is everything you need to see what is happening with your data storage systems and to change the system as needed. It's a single-pane-of-glass view that provides a common user interfaces for many perspectives of data storage operation. It is also offers GUI-based controls for setting up data storage, for tuning a running system, and to assist with larger modifications, like upgrades, expansion, and migrations.
### Dashboard

The components and features of the KDCC include the following.
- A top-level view that shows critical system metrics and active alerts.
- High-level operational charts with drill-downs to see the details of each subsystem.
- Versions of Ceph, Rook, and installed extensions, with information about available updates.

### Koor Dashboard
### Charts and Gauges

* A top-level view that shows critical system metrics and active alerts.
* High-level operational charts with drill-downs to see the details of each subsystem.
* Versions of Ceph, Rook, and installed extensions, with information about available updates.
- A comprehensive array of metrics displayed for quick visibility.
- Many of the metrics have drill-downs to see details.

### Koor Controls

* Control panels with editable fields for properties that can be modified.
* Staging for changes that require restart.
* Scrubbing schedules to ensure deep scrubbing happens frequently enough.
- Control panels with editable fields for properties that can be modified for system tuning.
- Staging for changes that require restart.
- Scrubbing schedules to ensure deep scrubbing happens frequently enough.

### Data Storage Set-up
### Data Storage Set-up and Expansion

* Koor Operator and Helm chart for installing the Koor Storage Distribution.
* A selection of templates to match common use cases.
* Automated system upgrades.
* SSO integration.
- Koor Operator and Helm chart for installing the Koor Storage Distribution.
- A selection of templates to match common use cases.
- Automated system upgrades.
- SSO integration.

### Troubleshooting Tools

* Diagnostic scripts to gather data to help pinpoint issues.
- Diagnostic scripts to gather data to help pinpoint issues.

## MVP and Milestones

## MVP
We do not know what a minimum-viable product is. Instead, we will focus on a series of milestones. As the rate of adoption starts to soar, we might look back point to where the Data Control Center reached MVP status.

We do not know what a minimum-viable product is. We have to keep adding features until rates of adoption start to soar.
As of August 31, we are working to go live with the Data Control Center as it stands.

Rather than worrying about predicting an MVP, let's focus on choosing the next best feature to add and making that as valuable as possible.
### First Alpha release - 0.1.0

Our first release will be an alpha with a few capabilities that work in a limited fashion. The first alpha includes:

- Authentication to gain access to the data cluster
- A dashboard view of some key metrics of the system (real data)
- A card view of the software versions obtained from our live version service

### Alpha versions

Getting alpha 0.1.0 live is just the beginning. That will allow early adopters to try the software and give us feedback. We also have more features to add to reach beta status.

For each alpha release, we will increment the minor version by 1. So 0.2.0 would follow 0.1.0.

### First Beta release - 0.N.0

Once we have all of the features defined here, that will be our first beta release. We will stay in beta until we resolve any blocking issues and any bugs that cause serious problems.

The features for beta include working features that establish a basis for future expansion:

- Authentication
- Dashboard with key metrics, including software versions
- At least one control panel that lets users change system parameters one-by-one
- Automated upgrades
- An interface for applying a named set of changes that suit a particular use case
- Choose a simple pattern

### 1.0 Release

We will reach this milestone when beta features are working.

## Timeline

We have to hurry. It is critical to put together a functioning alpha version of the KDCC by the end of August, which is about 15 calendar days away. The alpha does not have to do much other than serve as a good basis to continue adding features and functionality for the coming months.
Our timeline will be tight. We need to hurry without rushing. In other words, we should optimize by accomplishing fewer things and doing them reasonably well.

Once the architecture has been reviewed, and we select an initial feature set and define development tasks in Linear. This needs to be our top priority.
```mermaid
timeline
7-Sept : First alpha 0.1.0
28-Sept : First beta
12-Oct : 1.0 Release
```