Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions PLUGINS.md
Original file line number Diff line number Diff line change
Expand Up @@ -117,8 +117,10 @@ OpenShift development utilities and helpers
- **`/openshift:bump-deps` `<dependency> [version] [--create-jira] [--create-pr]`** - Bump dependencies in OpenShift projects with automated analysis and PR creation
- **`/openshift:create-cluster` `"[release-image] [platform] [options]"`** - Extract OpenShift installer from release image and create an OCP cluster
- **`/openshift:destroy-cluster` `"[install-dir]"`** - Destroy an OpenShift cluster created by create-cluster command
- **`/openshift:expand-test-case` `[test-idea-or-file-or-commands] [format]`** - Expand basic test ideas or existing oc commands into comprehensive test scenarios with edge cases in oc CLI or Ginkgo format
- **`/openshift:new-e2e-test` `[test-specification]`** - Write and validate new OpenShift E2E tests using Ginkgo framework
- **`/openshift:rebase` `<tag>`** - Rebase OpenShift fork of an upstream repository to a new upstream release.
- **`/openshift:review-test-cases` `[file-path-or-test-code-or-commands]`** - Review test cases for completeness, quality, and best practices - accepts file path or direct oc commands/test code

See [plugins/openshift/README.md](plugins/openshift/README.md) for detailed documentation.

Expand Down
12 changes: 12 additions & 0 deletions docs/data.json
Original file line number Diff line number Diff line change
Expand Up @@ -394,6 +394,12 @@
"synopsis": "/openshift:destroy-cluster [install-dir]",
"argument_hint": "\"[install-dir]\""
},
{
"name": "expand-test-case",
"description": "Expand basic test ideas or existing oc commands into comprehensive test scenarios with edge cases in oc CLI or Ginkgo format",
"synopsis": "/openshift:expand-test-case [test-idea-or-file-or-commands] [format]",
"argument_hint": "[test-idea-or-file-or-commands] [format]"
},
{
"name": "new-e2e-test",
"description": "Write and validate new OpenShift E2E tests using Ginkgo framework",
Expand All @@ -405,6 +411,12 @@
"description": "Rebase OpenShift fork of an upstream repository to a new upstream release.",
"synopsis": "/openshift:rebase [tag]",
"argument_hint": "<tag>"
},
{
"name": "review-test-cases",
"description": "Review test cases for completeness, quality, and best practices - accepts file path or direct oc commands/test code",
"synopsis": "/openshift:review-test-cases [file-path-or-test-code-or-commands]",
"argument_hint": "[file-path-or-test-code-or-commands]"
}
],
"skills": [],
Expand Down
79 changes: 79 additions & 0 deletions plugins/openshift/commands/expand-test-case.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
---
description: Expand basic test ideas or existing oc commands into comprehensive test scenarios with edge cases in oc CLI or Ginkgo format
argument-hint: [test-idea-or-file-or-commands] [format]
---

## Name
openshift:expand-test-case

## Synopsis
```
/openshift:expand-test-case [test-idea-or-file-or-commands] [format]
```

## Description

The `expand-test-case` command transforms basic test ideas or existing oc commands into comprehensive test scenarios. It accepts three types of input:

1. **Test idea**: Simple description of what to test (e.g., "verify pod deployment")
2. **File path**: Path to existing test file to expand (e.g., `/path/to/test.sh` or `/path/to/test.go`)
3. **oc commands**: Direct oc CLI commands to analyze and expand (e.g., `oc create pod nginx`)

The command expands the input to cover positive flows, negative scenarios, edge cases, and boundary conditions, helping QE engineers ensure thorough test coverage.

Supports two output formats:
- **oc CLI**: Shell scripts with oc commands for manual or automated execution
- **Ginkgo**: Go test code using Ginkgo/Gomega framework for E2E tests

## Implementation

The command analyzes the input and generates comprehensive scenarios:

1. **Parse Input**: Determine if input is a test idea, file path, or oc commands
- If file path: Read and analyze existing test code
- If oc commands: Parse commands to understand what's being tested
- If test idea: Understand the core feature or behavior
2. **Identify Test Dimensions**: Determine coverage aspects (functionality, security, performance, edge cases)
3. **Generate Positive Tests**: Happy path scenarios where everything works
4. **Generate Negative Tests**: Error handling, invalid inputs, permission issues
5. **Add Edge Cases**: Boundary values, race conditions, resource limits
6. **Define Validation**: Clear success criteria and assertions
7. **Format Output**: Generate in requested format (oc CLI or Ginkgo) - **MUST follow the standards in "Test Coverage Guidelines" section below**

**CRITICAL**: All generated test scenarios MUST adhere to the coverage dimensions, best practices, and standards defined in the **"Test Coverage Guidelines"** section below. Use the referenced examples and patterns from OpenShift origin repository.

## Test Coverage Guidelines

The command generates comprehensive test scenarios following industry best practices:

**Test Coverage Dimensions:**
- **Positive Tests**: Valid inputs and expected workflows
- **Negative Tests**: Invalid inputs, permission errors, missing dependencies
- **Edge Cases**: Boundary values (0, max values, empty inputs, special characters)
- **Security Tests**: RBAC validation, security context enforcement, privilege escalation
- **Resource Tests**: Low memory, disk pressure, network issues, rate limiting
- **Concurrency**: Multiple operations happening simultaneously
- **Failure Recovery**: Restart behavior, cleanup on failure

**References:**
- OpenShift Test Examples: https://github.com/openshift/origin/tree/master/test/extended
- Ginkgo BDD Framework: https://onsi.github.io/ginkgo/
- Test Pattern Catalog: https://github.com/openshift/origin/blob/master/test/extended/README.md
- oc CLI Reference: https://docs.openshift.com/container-platform/latest/cli_reference/openshift_cli/developer-cli-commands.html

**Best Practices Applied:**
- Use stable, descriptive test names (no dynamic IDs or timestamps)
- Ensure proper resource cleanup (prevent resource leaks)
- Include meaningful assertions with clear failure messages
- Isolate tests (each test creates its own resources)
- Add appropriate timeouts to prevent hanging tests
- Follow Ginkgo patterns: Describe/Context/It hierarchy
- Use framework helpers: e2epod, e2enode, e2enamespace

## Arguments

- **$1** (test-idea-or-file-or-commands): One of:
- **Test idea**: Description of what to test
- **File path**: Path to existing test file
- **oc commands**: Set of oc CLI commands to analyze and expand
- **$2** (format): Output format - "oc CLI" or "Ginkgo" (optional, will prompt if not provided)
69 changes: 69 additions & 0 deletions plugins/openshift/commands/review-test-cases.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
---
description: Review test cases for completeness, quality, and best practices - accepts file path or direct oc commands/test code
argument-hint: [file-path-or-test-code-or-commands]
---

## Name
openshift:review-test-cases

## Synopsis
```
/openshift:review-test-cases [file-path-or-test-code-or-commands]
```

## Description

The `review-test-cases` command provides comprehensive review of OpenShift test cases to ensure quality, completeness, and adherence to best practices. It accepts three types of input:

1. **File path**: Path to test file (e.g., `/path/to/test.sh` or `/path/to/test.go`)
2. **oc commands**: Direct oc CLI commands to review (e.g., paste a set of oc commands)
3. **Test code**: Pasted Ginkgo test code to analyze

The command analyzes test code in both oc CLI shell scripts and Ginkgo Go tests, helping QE engineers identify gaps in test coverage, improve test reliability, and ensure tests follow OpenShift testing standards.

## Implementation

The command analyzes test cases and provides structured feedback:

1. **Parse Test Input**: Determine if input is a file path, oc commands, or test code
- If file path: Read and analyze the test file
- If oc commands: Parse command sequence
- If test code: Analyze pasted Ginkgo/test code
2. **Identify Test Format**: Detect if it's oc CLI shell script or Ginkgo Go code
3. **Analyze Test Structure**: Review organization, naming, and patterns
4. **Check Coverage**: Verify positive, negative, and edge case coverage
5. **Review Assertions**: Ensure proper validation and error checking
6. **Evaluate Cleanup**: Verify resource cleanup and namespace management
7. **Assess Best Practices**: **MUST follow the standards defined in "Testing Guidelines and References" section below**
8. **Generate Recommendations**: Provide actionable improvement suggestions based on the guidelines

**CRITICAL**: All reviews MUST be evaluated against the specific standards, references, and best practices listed in the **"Testing Guidelines and References"** section below. Do not use generic testing advice - follow the OpenShift-specific guidelines provided.

## Testing Guidelines and References

The review follows established testing best practices from:

**For Ginkgo/E2E Tests:**
- OpenShift Origin Test Extended: https://github.com/openshift/origin/tree/master/test/extended
- Ginkgo Testing Framework: https://onsi.github.io/ginkgo/
- OpenShift Test Best Practices: https://github.com/openshift/origin/blob/master/test/extended/README.md

**For oc CLI Tests:**
- OpenShift CLI Documentation: https://docs.openshift.com/container-platform/latest/cli_reference/openshift_cli/developer-cli-commands.html
- Bash Best Practices: https://google.github.io/styleguide/shellguide.html

**Key Testing Standards:**
- Use descriptive, stable test names (no timestamps, random IDs)
- Proper resource cleanup (AfterEach, defer, trap)
- Meaningful assertions with clear failure messages
- Test isolation (each test creates own resources)
- Appropriate timeouts and waits
- No BeforeAll/AfterAll in Ginkgo tests
- Use framework helpers (e2epod, e2enode) when available

## Arguments

- **$1** (file-path-or-test-code-or-commands): One of:
- **File path**: Path to test file (shell script or Go test file)
- **oc commands**: Set of oc CLI commands to review
- **Test code**: Pasted test code (Ginkgo or shell script)