Skip to content

Commit

Permalink
Initial implementation of external data sources
Browse files Browse the repository at this point in the history
  • Loading branch information
ilyaluk committed Dec 12, 2022
1 parent abb9ed4 commit 15c8330
Show file tree
Hide file tree
Showing 43 changed files with 667 additions and 1,190 deletions.
4 changes: 3 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
@@ -1,9 +1,11 @@
main
packer-plugin-scaffolding
packer-plugin-*
**/packer_log*.txt
**/git_*test.pkr.hcl
crash.log

dist/*
.docs/*
docs.zip

.idea
6 changes: 3 additions & 3 deletions .goreleaser.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ before:
- go test ./...
# As part of the release doc files are included as a separate deliverable for
# consumption by Packer.io. To include a separate docs.zip uncomment the following command.
#- make ci-release-docs
- make ci-release-docs
# Check plugin compatibility with required version of the Packer SDK
- make plugin-check
builds:
Expand Down Expand Up @@ -78,8 +78,8 @@ release:
# draft: true
# As part of the release doc files are included as a separate deliverable for consumption by Packer.io.
# To include a separate docs.zip uncomment the extra_files config and the docs.zip command hook above.
#extra_files:
#- glob: ./docs.zip
extra_files:
- glob: ./docs.zip

changelog:
skip: true
2 changes: 1 addition & 1 deletion GNUmakefile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
NAME=scaffolding
NAME=external
BINARY=packer-plugin-${NAME}

COUNT?=1
Expand Down
3 changes: 3 additions & 0 deletions LICENSE
Original file line number Diff line number Diff line change
@@ -1,3 +1,6 @@
Copyright (c) 2017 HashiCorp, Inc.
Copyright (c) 2022 Joom

Mozilla Public License Version 2.0
==================================

Expand Down
113 changes: 35 additions & 78 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,90 +1,47 @@
# Packer Plugin Scaffolding

This repository is a template for a Packer multi-component plugin. It is intended as a starting point for creating Packer plugins, containing:
- A builder ([builder/scaffolding](builder/scaffolding))
- A provisioner ([provisioner/scaffolding](provisioner/scaffolding))
- A post-processor ([post-processor/scaffolding](post-processor/scaffolding))
- A data source ([datasource/scaffolding](datasource/scaffolding))
- Docs ([docs](docs))
- A working example ([example](example))

These folders contain boilerplate code that you will need to edit to create your own Packer multi-component plugin.
A full guide to creating Packer plugins can be found at [Extending Packer](https://www.packer.io/docs/plugins/creation).

In this repository you will also find a pre-defined GitHub Action configuration for the release workflow
(`.goreleaser.yml` and `.github/workflows/release.yml`). The release workflow configuration makes sure the GitHub
release artifacts are created with the correct binaries and naming conventions.

Please see the [GitHub template repository documentation](https://docs.github.com/en/free-pro-team@latest/github/creating-cloning-and-archiving-repositories/creating-a-repository-from-a-template)
for how to create a new repository from this template on GitHub.

## Packer plugin projects

Here's a non exaustive list of Packer plugins that you can checkout:

* [github.com/hashicorp/packer-plugin-docker](https://github.com/hashicorp/packer-plugin-docker)
* [github.com/exoscale/packer-plugin-exoscale](https://github.com/exoscale/packer-plugin-exoscale)
* [github.com/sylviamoss/packer-plugin-comment](https://github.com/sylviamoss/packer-plugin-comment)
* [github.com/hashicorp/packer-plugin-hashicups](https://github.com/hashicorp/packer-plugin-hashicups)
# Packer external data source plugin

A plugin for [Packer](https://www.packer.io/) which provides access to external commands. Compatible with Packer >= 1.7.0

## Usage
```hcl
packer {
required_plugins {
git = {
version = ">= 0.0.1"
source = "github.com/joomcode/external"
}
}
}
data "external-json" "example" {
program = ["jq", "{ \"foo\": .key1 }"]
query = {
key1 = "val1"
}
}
data "external-raw" "example" {
program = ["rev"]
query = "hello"
}
locals {
json_result = data.external-json.example.result["foo"] # "val1"
raw_result = data.external-raw.example.result # "olleh\n"
}
```

Looking at their code will give you good examples.
See docs for more detailed information.

## Running Acceptance Tests

Make sure to install the plugin with `go build .` and to have Packer installed locally.
Then source the built binary to the plugin path with `cp packer-plugin-scaffolding ~/.packer.d/plugins/packer-plugin-scaffolding`
Once everything needed is set up, run:
```
PACKER_ACC=1 go test -count 1 -v ./... -timeout=120m
```bash
make testacc
```

This will run the acceptance tests for all plugins in this set.

## Test Plugin Example Action

This scaffolding configures a [manually triggered plugin test action](/.github/workflows/test-plugin-example.yml).
By default, the action will run Packer at the latest version to init, validate, and build the example configuration
within the [example](example) folder. This is useful to quickly test a basic template of your plugin against Packer.

The example must contain the `required_plugins` block and require your plugin at the latest or any other released version.
This will help test and validate plugin releases.

## Registering Documentation on Packer.io

Documentation for a plugin is maintained within the `docs` directory and served on GitHub.
To include plugin docs on Packer.io a global pre-hook has been added to the main scaffolding .goreleaser.yml file, that if uncommented will generate and include a docs.zip file as part of the plugin release.

The `docs.zip` file will contain all of the `.mdx` files under the plugins root `docs/` directory that can be consumed remotely by Packer.io.

Once the first `docs.zip` file has been included into a release you will need to open a one time pull-request against [hashicorp/packer](https://github.com/hashicorp/packer) to register the plugin docs.
This is done by adding the block below for the respective plugin to the file [website/data/docs-remote-navigation.js](https://github.com/hashicorp/packer/blob/master/website/data/docs-remote-plugins.json).

```json
{
"title": "Scaffolding",
"path": "scaffolding",
"repo": "hashicorp/packer-plugin-scaffolding",
"version": "latest",
"sourceBranch": "main"
}
```

If a plugin maintainer wishes to only include a specific version of released docs then the `"version"` key in the above configuration should be set to a released version of the plugin. Otherwise it should be set to `"latest"`.

The `"sourceBranch"` key in the above configuration ensures potential contributors can link back to source files in the plugin repository from the Packer docs site. If a `"sourceBranch"` value is not present, it will default to `"main"`.

The documentation structure needed for Packer.io can be generated manually, by creating a simple zip file called `docs.zip` of the docs directory and included in the plugin release.

```/bin/bash
[[ -d docs/ ]] && zip -r docs.zip docs/
```

Once the first `docs.zip` file has been included into a release you will need to open a one time pull-request against [hashicorp/packer](https://github.com/hashicorp/packer) to register the plugin docs.

# Requirements

- [packer-plugin-sdk](https://github.com/hashicorp/packer-plugin-sdk) >= v0.2.9
- [Go](https://golang.org/doc/install) >= 1.17

## Packer Compatibility
This scaffolding template is compatible with Packer >= v1.7.0
32 changes: 0 additions & 32 deletions builder/scaffolding/artifact.go

This file was deleted.

80 changes: 0 additions & 80 deletions builder/scaffolding/builder.go

This file was deleted.

47 changes: 0 additions & 47 deletions builder/scaffolding/builder.hcl2spec.go

This file was deleted.

38 changes: 0 additions & 38 deletions builder/scaffolding/step_say_config.go

This file was deleted.

15 changes: 0 additions & 15 deletions builder/scaffolding/test-fixtures/template.pkr.hcl

This file was deleted.

Loading

0 comments on commit 15c8330

Please sign in to comment.