ScanRunner: A Modular CLI for File Validation, Compliance, and AI-Powered Insights

Overview

ScanRunner is a flexible and extensible command-line tool designed to simplify file scanning, compliance checks, and AI-driven insights. Built with the powerful Cobra CLI framework, it enables users to process YAML/JSON files, enforce compliance rules, and leverage lightweight AI models to generate actionable suggestions.

Key Features

✨ File Scanning and Validation

Scan directories for YAML/JSON files.
Validate files for structural correctness and required fields.

📋 Compliance Enforcement

Apply customizable rules for mock compliance checks.
Analyze Kubernetes manifests for missing labels, RBAC misconfigurations, and more.

🤖 AI-Powered Suggestions

Leverage pre-trained lightweight AI models (ONNX) for context-aware recommendations.
Prioritize findings and suggest resolutions dynamically.

📈 Reporting and Summarization

Aggregate results into clean JSON/Markdown reports.
Generate summary statistics for compliance results and AI insights.

⚙️ Concurrency and Scalability

Process files in parallel using a worker pool.
Efficient resource management for large-scale tasks.

Architecture

graph TD
    A[User] -->|Inputs commands| B[CLI Tool]
    B --> C{Command Handler}
    C -->|Scan Command| D[File Parser]
    C -->|Validate Command| E[Compliance Engine]
    C -->|Report Command| F[Report Generator]
    C -->|AI Integration| G[AI Engine]
    
    D --> H[File Scanner]
    D --> I[File Parser - YAML/JSON]
    D --> J[File Validator]

    E --> K[Rule Engine]
    E --> L[Compliance Evaluator]
    E --> M[Policy Checker]

    G --> N[Model Loader]
    G --> O[Inference Engine]
    G --> P[Suggestion Generator]

    F --> Q[Results Aggregator]
    F --> R[Summary Formatter]
    F --> S[Report Formatter - JSON/Markdown]

    subgraph Internal Components
        D
        E
        G
        F
    end

    subgraph Core CLI Logic
        C
    end

    subgraph Output
        Q
        R
        S
    end

Loading

Setup Instructions

Prerequisites

Go (1.20 or later) installed.
Familiarity with command-line tools and YAML/JSON files.

Install ScanRunner

Clone the repository:

git clone https://github.com/your_username/scanrunner.git
cd scanrunner

Run Setup script:
```
bash setup.sh
```
Build the tool:
```
go build -o scanrunner main.go
```

Basic Commands

Build the CLI Tool
- Compile the tool before running commands:
```
go build -o scanrunner main.go
```
Root Command
- Display help for the root command:
```
./scanrunner --help
```
Scan Command
- Run the scan command with default settings:
```
./scanrunner scan
```
- Specify a custom directory to scan:
```
./scanrunner scan --config=/path/to/config.yaml
```

Validate Command

Run the validate command with default compliance rules:
```
./scanrunner validate
```

Specify a custom rules file:

./scanrunner validate --rules=/path/to/custom-rules.yaml

Specify a custom configuration file:

./scanrunner validate --config=/path/to/config.yaml

Enable strict mode:
```
./scanrunner validate --strict
```

Report Command
- Generate a report in the default format (JSON):
```
./scanrunner report
```
- Specify a custom output format (e.g., Markdown):
```
./scanrunner report --format=markdown
```
- Save the report to a specific path:
```
./scanrunner report --output=/path/to/report.md
```
Version Command
- Display the version of the CLI tool:
```
./scanrunner version
```

Folder Structure

Yes, the folder structure you provided is a great choice for your project, and the descriptions can be slightly refined for better clarity. Here's the updated explanation for your scanRunner folder structure:

Folder Structure Description

scanrunner/
├── bin/                      # Compiled binaries (optional for pre-built releases)
├── cmd/                      # CLI commands
│   ├── root.go               # Base command setup
│   ├── scan.go               # Scan command implementation
│   ├── validate.go           # Validate command implementation
│   ├── report.go             # Report command implementation
│   ├── scan_secrets.go       # Secret scanning command (Phase 5)
│   ├── scan_dependencies.go  # Dependency scanning command (Phase 6)
│   ├── scan_iac.go           # IaC scanning command (Phase 6)
│   ├── scan_images.go        # Image scanning command (Phase 6)
│   ├── scan_code.go          # Static code analysis command (Post-MVP)
│   ├── validate_policy.go    # Kubernetes policy validation command (Post-MVP)
│   ├── scan_audit.go         # Audit log scanning command (Post-MVP)
│   ├── version.go            # Version command implementation
├── internal/
│   ├── cli/                  # CLI utilities for argument parsing and validation
│   │   ├── parser.go         # CLI argument and flag parser
│   │   ├── validator.go      # Input validation logic
│   ├── model/                # CLI utilities for argument parsing and validation
│   │   ├── rules.go          # Validation Rules Model
│   ├── compliance/           # Compliance logic and rule engine
│   │   ├── rules.go          # Compliance rules logic and handling
│   │   ├── evaluator.go      # Compliance evaluation engine
│   │   ├── report.go         # Report generation logic for compliance checks
│   ├── fileparser/           # File handling logic
│   │   ├── yaml_parser.go    # YAML file parsing and validation
│   │   ├── json_parser.go    # JSON file parsing and validation
│   │   ├── scanner.go        # File system scanner and directory walker
│   │   ├── writer.go         # File content writer for output or modifications
│   ├── concurrency/          # Concurrency utilities for parallel processing
│   │   ├── worker_pool.go    # Goroutines and worker pool utilities
│   │   ├── task_manager.go   # Task distribution and synchronization
│   │   ├── aggregator.go     # Result aggregation logic from multiple workers
│   ├── ai/                   # AI-related components
│   │   ├── inference.go      # AI model inference logic
│   │   ├── ai_model.onnx     # Pre-trained lightweight ONNX model
│   │   ├── model_loader.go   # ONNX model loading and initialization
│   │   ├── suggestion.go     # AI-based suggestion generation based on results
│   ├── security/             # Security scanning integrations
│   │   ├── trivy.go          # Trivy integration for secret and dependency scanning
│   │   ├── checkov.go        # Checkov integration for IaC scanning (Post-MVP)
│   ├── kubernetes/           # Kubernetes runtime compliance (Post-MVP)
│   │   ├── operator.go       # Operator implementation using Kubebuilder
│   │   ├── policy_validator.go # Policy validation logic
│   │   ├── audit_log.go      # Kubernetes audit log processing
│   │   ├── pod_security.go   # Pod security compliance checks
├── pkg/                      # Shared utilities and reusable components
│   ├── logger.go             # Logging utilities for consistent output
│   ├── config_loader.go      # Configuration loading and management
│   ├── error_handler.go      # Centralized error handling utilities
│   ├── rules_loader.go       # Centralized rules handling utilities
│   ├── report_formatter.go   # Formatting logic for reports (e.g., JSON/Markdown)
│   ├── stats_generator.go    # Summary statistics generator for results
├── config/                   # Configuration files
│   ├── custom-rules.yaml     # Default rules file for compliance checks
│   ├── default-config.yaml   # Default configuration shipped with the tool
├── assets/                   # Static assets and templates
│   ├── templates/
│   │   ├── report_template.txt  # Report template for terminal output
│   │   ├── report_template.json # Report template for JSON output
├── docs/                     # Documentation files
│   ├── usage.md              # Detailed CLI usage guide
│   ├── architecture.md       # High-level architecture design document
│   ├── faq.md                # Frequently asked questions
├── test/                     # Test suite for the project
│   ├── cli_test.go           # End-to-end CLI tests
│   ├── compliance_test.go    # Tests for compliance rules and engine
│   ├── fileparser_test.go    # Tests for file parsing and validation
│   ├── concurrency_test.go   # Tests for concurrency and parallel processing
│   ├── ai_test.go            # Tests for AI model inference and suggestions
│   ├── security_test.go      # Tests for security scanning integrations
│   ├── integration/          # Integration tests for workflow validation
├── CONTRIBUTING.md           # Contribution guidelines for open-source contributors
├── LICENSE                   # Licensing terms for the project
├── go.mod                    # Go module file for dependency management
├── Makefile                  # Build and automation commands for the project
├── main.go                   # Entry point for the CLI tool
└── README.md                 # Project overview and basic usage instructions

Folder Descriptions

bin/:
- Stores pre-built binaries for distribution (optional).
cmd/:
- Contains all CLI-related commands implemented with Cobra, including:
  - root.go: Base setup for the CLI and parent command.
  - scan.go, validate.go, report.go: Implementations of individual commands.
  - version.go: Outputs the tool’s version.
internal/:
- Core internal logic split into focused modules:
  - cli/: Argument parsing (parser.go) and input validation (validator.go).
  - compliance/: Compliance rules (rules.go), evaluation logic (evaluator.go), and report generation (report.go).
  - fileparser/: File handling logic for YAML/JSON parsing, scanning directories, and writing files.
  - concurrency/: Worker pool, task management, and result aggregation for parallel processing.
  - ai/: AI model loading (model_loader.go), inference logic (inference.go), and suggestion generation (suggestion.go).
pkg/:
- Shared utility functions used across modules, including logging, configuration, error handling, and report formatting.
config/:
- Default and example configuration files for users to customize.
assets/:
- Templates for report output and static assets like the AI model.
docs/:
- Comprehensive documentation for usage, architecture, and FAQs.
test/:
- Unit, integration, and end-to-end tests to ensure the tool’s functionality.
CONTRIBUTING.md:
- Guidelines for contributing to the open-source project.
LICENSE:
- Defines the licensing terms for the tool (e.g., MIT).
Makefile:
- Simplifies build, test, and run commands for developers.
README.md:
- Serves as the primary project overview and usage guide.

Contributing

We welcome contributions! To get started:

Fork this repository.
Clone your fork and create a feature branch.
Submit a pull request with your changes.

For detailed guidelines, see CONTRIBUTING.md.

Future Features

Advanced compliance rule definitions.
Real-time drift detection.
Integration with cloud storage providers (e.g., AWS S3).
Interactive CLI wizard for setup.

License

This project is licensed under the MIT License. See LICENSE for details.

Contact

Have questions or feedback? Reach out via GitHub Issues.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Readme.md

Readme.md

ScanRunner: A Modular CLI for File Validation, Compliance, and AI-Powered Insights

Overview

Key Features

Architecture

Setup Instructions

Prerequisites

Install ScanRunner

Basic Commands

Folder Structure

Folder Structure Description

Folder Descriptions

Contributing

Future Features

License

Contact

Files

Readme.md

Latest commit

History

Readme.md

File metadata and controls

ScanRunner: A Modular CLI for File Validation, Compliance, and AI-Powered Insights

Overview

Key Features

Architecture

Setup Instructions

Prerequisites

Install ScanRunner

Basic Commands

Folder Structure

Folder Structure Description

Folder Descriptions

Contributing

Future Features

License

Contact