Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: change plugin system to use gRPC #3529

Merged
merged 91 commits into from
Nov 10, 2023
Merged
Show file tree
Hide file tree
Changes from 89 commits
Commits
Show all changes
91 commits
Select commit Hold shift + click to select a range
3c5f815
chore: update plugin template Go version to 1.20
jeronimoalbi May 16, 2023
1a83cb2
docs: add documentation to clarify the plugin cache purpose
jeronimoalbi May 16, 2023
67836fc
chore: improve plugins code spacing for better redability
jeronimoalbi May 16, 2023
9d1a2ee
chore: add context support to plugin scaffold
jeronimoalbi May 16, 2023
e197e0d
refactor: move plugin RPC implementation to an `rpc.go` file
jeronimoalbi May 26, 2023
ed7ffff
refactor: simplify RPC plugin implementation
jeronimoalbi May 26, 2023
f267f91
feat: add proto buffer files for the gRPC plugins
jeronimoalbi May 26, 2023
5e8eb30
chore: generate plugin files from proto
jeronimoalbi May 26, 2023
d73c16b
chore: remove extra blank line
jeronimoalbi Jun 2, 2023
3aacaaa
chore: add Flags to ExecutedCommand proto definition
jeronimoalbi Jun 2, 2023
3031c87
refactor: remove rpc plugin implementation
jeronimoalbi Jun 2, 2023
0f68caa
refactor: move interface types methods to the generated types
jeronimoalbi Jun 2, 2023
bb1747b
refactor: move plugin interface into grpc package
jeronimoalbi Jun 2, 2023
cf29bee
feat: add gRPC plugin implementation
jeronimoalbi Jun 2, 2023
ec03661
refactor: change plugin to use gRPC for communication
jeronimoalbi Jun 2, 2023
ea02278
refactor: change plugin command to use the new gRPC interface
jeronimoalbi Jun 2, 2023
03134aa
refactor: add context to gRPC plugin client communication
jeronimoalbi Jun 2, 2023
6e84d5b
refactor: move plugin interface and gRPC implementation
jeronimoalbi Jun 2, 2023
2554254
chore: add plugin flag type aliases for v1
jeronimoalbi Jun 2, 2023
99f5afb
refactor: add flag contructor methods to executed command
jeronimoalbi Jun 5, 2023
72bbab5
refactor: update plugin template to use gRPC
jeronimoalbi Jun 5, 2023
67c9e0b
test: fixed tests
jeronimoalbi Jun 5, 2023
73acd29
chore: move proto flag type enum definition into flag message
jeronimoalbi Jun 6, 2023
bd48cee
chore: add changelog entry
jeronimoalbi Jun 6, 2023
db76443
chore: correct flag type reference in plugin service tests
jeronimoalbi Jun 8, 2023
b56d445
test: improve plugin scaffold test
jeronimoalbi Jun 8, 2023
a6125ce
test: change flag types
jeronimoalbi Jun 8, 2023
4a23707
chore: lower plugin template Go version to 1.19
jeronimoalbi Jun 9, 2023
ecbf14c
chore: use `hplugin` alias go hashicorp's go-plugin package
jeronimoalbi Jun 9, 2023
00962e2
test: fix issue that killed plugins when testing
jeronimoalbi Jun 9, 2023
777d036
Merge branch 'main' into feat/plugin-system-improvements-versioning
jeronimoalbi Jun 9, 2023
685e37f
test: fix plugin command tests
jeronimoalbi Jun 9, 2023
e74cebe
Merge branch 'main' into feat/plugin-system-improvements-versioning
jeronimoalbi Jun 9, 2023
db37c4c
fix: correct issue in plugin load
jeronimoalbi Jun 14, 2023
9dbc006
chore: update plugin documentation
jeronimoalbi Jun 15, 2023
4a5193e
Merge branch 'main' into feat/plugin-system-improvements-versioning
jeronimoalbi Jun 28, 2023
86fdf15
chore: fix changelog
jeronimoalbi Jun 28, 2023
d194e8c
chore: add buf to tool dependencies
jeronimoalbi Jun 28, 2023
e75a58b
feat: add make file targets to generate code from proto
jeronimoalbi Jun 28, 2023
d7d9d0c
fix: correct proto file linting issue
jeronimoalbi Jun 28, 2023
095d992
chore: proto file format fix
jeronimoalbi Jun 28, 2023
25ad77a
ci: add GitHub workflow to check and lint proto files
jeronimoalbi Jun 28, 2023
eef0459
ci: disable proto breaking change
jeronimoalbi Jun 28, 2023
18a73f0
Merge branch 'main' into feat/plugin-system-improvements-versioning
jeronimoalbi Jul 4, 2023
eae8ffa
fix: correct Buf repository name
jeronimoalbi Jul 4, 2023
003b84d
Merge branch 'main' into feat/plugin-system-improvements-versioning
jeronimoalbi Jul 19, 2023
7482630
test: change plugin integration test to use a local example folder
jeronimoalbi Jul 19, 2023
20791be
Merge branch 'main' into feat/plugin-system-improvements-versioning
Pantani Jul 19, 2023
c700f88
Merge branch 'main' into feat/plugin-system-improvements-versioning
Pantani Jul 19, 2023
800cecb
Merge branch 'main' into feat/plugin-system-improvements-versioning
jeronimoalbi Jul 21, 2023
7ae7648
chore: update Go and CLI versions
jeronimoalbi Jul 21, 2023
e25ec18
ci: update GitHub workflows to use Go version 1.20
jeronimoalbi Jul 21, 2023
28e8bda
fix: beam me up Scotty
jeronimoalbi Jul 21, 2023
22fc230
test: move command and manifest test to the right location
jeronimoalbi Jul 21, 2023
ca52e3e
test: add plugin gRPC types tests
jeronimoalbi Jul 24, 2023
eaf1726
ci: correct CI issue with naked return
jeronimoalbi Jul 24, 2023
26c3a57
fix: correct plugin command path issue
jeronimoalbi Jul 24, 2023
c0d667e
Merge branch 'main' into feat/plugin-system-improvements-versioning
Pantani Jul 25, 2023
7f9a664
Merge branch 'main' into feat/plugin-system-improvements-versioning
jeronimoalbi Jul 26, 2023
1c812f8
Merge branch 'main' into feat/plugin-system-improvements-versioning
jeronimoalbi Jul 26, 2023
6304305
Merge branch 'main' into feat/plugin-system-improvements-versioning
jeronimoalbi Jul 27, 2023
1e97691
Merge branch 'main' into feat/plugin-system-improvements-versioning
jeronimoalbi Aug 7, 2023
d5e7765
Merge branch 'main' into feat/plugin-system-improvements-versioning
jeronimoalbi Aug 8, 2023
4a49f5d
Merge branch 'main' into feat/plugin-system-improvements-versioning
Pantani Aug 9, 2023
e085ea8
Merge branch 'main' into feat/plugin-system-improvements-versioning
jeronimoalbi Aug 10, 2023
daf8f23
Merge branch 'main' into feat/plugin-system-improvements-versioning
jeronimoalbi Aug 22, 2023
87a6635
Merge branch 'main' into feat/plugin-system-improvements-versioning
Pantani Aug 28, 2023
521305d
Merge branch 'main' into feat/plugin-system-improvements-versioning
jeronimoalbi Sep 25, 2023
813702f
chore: remove deprecated replace from Go mod
jeronimoalbi Oct 5, 2023
bb09bd0
Merge branch 'main' into feat/plugin-system-improvements-versioning
jeronimoalbi Oct 5, 2023
8524cc2
Merge branch 'main' into feat/plugin-system-improvements-versioning
jeronimoalbi Oct 10, 2023
a9e35c9
Merge branch 'main' into feat/plugin-system-improvements-versioning
jeronimoalbi Oct 13, 2023
5d7b6a6
Merge branch 'main' into feat/plugin-system-improvements-versioning
Pantani Oct 16, 2023
a5e268f
Merge branch 'main' into feat/plugin-system-improvements-versioning
Pantani Oct 16, 2023
367a939
Merge branch 'main' into feat/plugin-system-improvements-versioning
Pantani Oct 16, 2023
9c3336e
chore: change Ignite App handshake config
jeronimoalbi Oct 17, 2023
a641547
Merge branch 'main' into feat/plugin-system-improvements-versioning
Pantani Oct 18, 2023
857300d
Merge branch 'main' into feat/plugin-system-improvements-versioning
Pantani Oct 18, 2023
1ea85a3
Merge branch 'main' into feat/plugin-system-improvements-versioning
jeronimoalbi Oct 19, 2023
bf692a5
feat: add bidirectional communication to plugin system (#3544)
jeronimoalbi Oct 19, 2023
9aaf758
Merge branch 'main' into feat/plugin-system-improvements-versioning
jeronimoalbi Oct 19, 2023
e8cb887
Merge branch 'main' into feat/plugin-system-improvements-versioning
jeronimoalbi Oct 23, 2023
e55b4c9
Merge branch 'main' into feat/plugin-system-improvements-versioning
Pantani Oct 24, 2023
b4b128c
Merge branch 'main' into feat/plugin-system-improvements-versioning
jeronimoalbi Nov 6, 2023
e09eae6
Merge branch 'main' into feat/plugin-system-improvements-versioning
Pantani Nov 6, 2023
5357eb1
Merge branch 'main' into feat/plugin-system-improvements-versioning
Pantani Nov 6, 2023
1106fed
Merge branch 'main' into feat/plugin-system-improvements-versioning
Pantani Nov 6, 2023
0fbea44
Merge branch 'main' into feat/plugin-system-improvements-versioning
Pantani Nov 6, 2023
f810f29
Merge branch 'main' into feat/plugin-system-improvements-versioning
Pantani Nov 6, 2023
05a881e
Merge branch 'main' into feat/plugin-system-improvements-versioning
Pantani Nov 7, 2023
593de50
Merge branch 'main' into feat/plugin-system-improvements-versioning
Pantani Nov 9, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 31 additions & 0 deletions .github/workflows/proto-checker.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
name: Protobuf Files

on:
pull_request:
paths:
- "proto/**"

permissions:
contents: read

jobs:
lint:
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- uses: actions/checkout@v3
- uses: bufbuild/[email protected]
- uses: bufbuild/buf-lint-action@v1
with:
input: "proto"

# TODO: Uncomment after PR#3529 is merged
# break-check:
# runs-on: ubuntu-latest
# steps:
# - uses: actions/checkout@v3
# - uses: bufbuild/[email protected]
# - uses: bufbuild/buf-breaking-action@v1
# with:
# input: "proto"
# against: "https://github.com/${{ github.repository }}.git#branch=${{ github.event.pull_request.base.ref }},ref=HEAD~1,subdir=proto"
1 change: 0 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@ scripts/**/nodetime-*
dist/
node_modules
.DS_Store
apps/
.idea
.vscode
docs/.vuepress/dist
Expand Down
20 changes: 20 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,26 @@ lint:

.PHONY: govet format lint

## proto-all: Format, lint and generate code from proto files using buf.
proto-all: proto-format proto-lint proto-gen

## proto-gen: Run buf generate.
proto-gen:
@echo Generating code from proto...
@buf generate --template ./proto/buf.gen.yaml --output ./

## proto-format: Run buf format and update files with invalid proto format>
proto-format:
@echo Formatting proto files...
@buf format --write

## proto-lint: Run buf lint.
proto-lint:
@echo Linting proto files...
@buf lint

.PHONY: proto-all proto-gen proto-format proto-lint

## test-unit: Run the unit tests.
test-unit:
@echo Running unit tests...
Expand Down
3 changes: 3 additions & 0 deletions buf.work.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
version: v1
directories:
- proto
11 changes: 11 additions & 0 deletions changelog.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,17 @@

### Features

- [#3544](https://github.com/ignite/cli/pull/3544) Add bidirectional communication to plugin system
- [#3561](https://github.com/ignite/cli/pull/3561) Add GetChainInfo method to plugin system API

### Changes

- [#3529](https://github.com/ignite/cli/pull/3529) Refactor plugin system to use gRPC

## [`v0.27.1`](https://github.com/ignite/cli/releases/tag/v0.27.1)

### Features

- [#3476](https://github.com/ignite/cli/pull/3476) Use `buf.build` binary to code generate from proto files
- [#3614](https://github.com/ignite/cli/pull/3614) feat: use DefaultBaseappOptions for app.New method
- [#3536](https://github.com/ignite/cli/pull/3536) Change app.go to v2 and add AppWiring feature
Expand Down
108 changes: 59 additions & 49 deletions docs/docs/apps/02-developing-apps.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,31 +43,35 @@ All apps must implement a predefined interface:
```go title=ignite/services/plugin/interface.go
type Interface interface {
// Manifest declares app's Command(s) and Hook(s).
Manifest() (Manifest, error)
Manifest(context.Context) (*Manifest, error)

// Execute will be invoked by ignite when an app Command is executed.
// It is global for all commands declared in Manifest, if you have declared
// multiple commands, use cmd.Path to distinguish them.
Execute(cmd ExecutedCommand) error
// The ClientAPI argument can be used by plugins to get chain app analysis info.
Execute(context.Context, *ExecutedCommand, ClientAPI) error

// ExecuteHookPre is invoked by ignite when a command specified by the Hook
// path is invoked.
// It is global for all hooks declared in Manifest, if you have declared
// multiple hooks, use hook.Name to distinguish them.
ExecuteHookPre(hook ExecutedHook) error
// The ClientAPI argument can be used by plugins to get chain app analysis info.
ExecuteHookPre(context.Context, *ExecutedHook, ClientAPI) error

// ExecuteHookPost is invoked by ignite when a command specified by the hook
// path is invoked.
// It is global for all hooks declared in Manifest, if you have declared
// multiple hooks, use hook.Name to distinguish them.
ExecuteHookPost(hook ExecutedHook) error
// The ClientAPI argument can be used by plugins to get chain app analysis info.
ExecuteHookPost(context.Context, *ExecutedHook, ClientAPI) error

// ExecuteHookCleanUp is invoked by ignite when a command specified by the
// hook path is invoked. Unlike ExecuteHookPost, it is invoked regardless of
// execution status of the command and hooks.
// It is global for all hooks declared in Manifest, if you have declared
// multiple hooks, use hook.Name to distinguish them.
ExecuteHookCleanUp(hook ExecutedHook) error
// The ClientAPI argument can be used by plugins to get chain app analysis info.
ExecuteHookCleanUp(context.Context, *ExecutedHook, ClientAPI) error
}
```

Expand All @@ -76,35 +80,34 @@ the method's body.

## Defining app's manifest

Here is the `Manifest` struct:

```go title=ignite/services/plugin/interface.go
type Manifest struct {
Name string

// Commands contains the commands that will be added to the list of ignite
// commands. Each commands are independent, for nested commands use the
// inner Commands field.
Commands []Command

// Hooks contains the hooks that will be attached to the existing ignite
// commands.
Hooks []Hook

// SharedHost enables sharing a single app server across all running instances
// of an app. Useful if an app adds or extends long running commands
//
// Example: if an app defines a hook on `ignite chain serve`, a server is instanciated
// when the command is run. Now if you want to interact with that instance from commands
// defined in that app, you need to enable `SharedHost`, or else the commands will just
// instantiate separate app servers.
//
// When enabled, all apps of the same `Path` loaded from the same configuration will
// attach it's gRPC client to a an existing gRPC server.
//
// If an app instance has no other running app servers, it will create one and it
// will be the host.
SharedHost bool `yaml:"shared_host"`
Here is the `Manifest` proto message definition:

```protobuf title=proto/ignite/services/plugin/grpc/v1/types.proto
message Manifest {
// App name.
string name = 1;

// Commands contains the commands that will be added to the list of ignite commands.
// Each commands are independent, for nested commands use the inner Commands field.
bool shared_host = 2;

// Hooks contains the hooks that will be attached to the existing ignite commands.
repeated Command commands = 3;

// Enables sharing a single app server across all running instances of an Ignite App.
// Useful if an app adds or extends long running commands.
//
// Example: if an app defines a hook on `ignite chain serve`, a server is instanciated
// when the command is run. Now if you want to interact with that instance
// from commands defined in that app, you need to enable shared host, or else the
// commands will just instantiate separate app servers.
//
// When enabled, all apps of the same path loaded from the same configuration will
// attach it's RPC client to a an existing RPC server.
//
// If an app instance has no other running app servers, it will create one and it
// will be the host.
repeated Hook hooks = 4;
}
```

Expand Down Expand Up @@ -132,16 +135,16 @@ For instance, let's say your app adds a new `oracle` command to `ignite
scaffold`, then the `Manifest` method will look like :

```go
func (app) Manifest() (plugin.Manifest, error) {
return plugin.Manifest{
func (app) Manifest(context.Context) (*plugin.Manifest, error) {
return &plugin.Manifest{
Name: "oracle",
Commands: []plugin.Command{
Commands: []*plugin.Command{
{
Use: "oracle [name]",
Short: "Scaffold an oracle module",
Long: "Long description goes here...",
// Optionnal flags is required
Flags: []plugin.Flag{
Flags: []*plugin.Flag{
{Name: "source", Type: plugin.FlagTypeString, Usage: "the oracle source"},
},
// Attach the command to `scaffold`
Expand All @@ -156,14 +159,21 @@ To update the app execution, you have to change the `Execute` command. For
example:

```go
func (app) Execute(cmd plugin.ExecutedCommand) error {
func (app) Execute(_ context.Context, cmd *plugin.ExecutedCommand, _ plugin.ClientAPI) error {
if len(cmd.Args) == 0 {
return fmt.Errorf("oracle name missing")
}

flags, err := cmd.NewFlags()
if err != nil {
return err
}

var (
name = cmd.Args[0]
source, _ = cmd.Flags().GetString("source")
source, _ = flags.GetString("source")
)

// Read chain information
c, err := getChain(cmd)
if err != nil {
Expand Down Expand Up @@ -199,10 +209,10 @@ resulting in `post` and `clean up` not executing.
The following is an example of a `hook` definition.

```go
func (app) Manifest() (plugin.Manifest, error) {
return plugin.Manifest{
func (app) Manifest(context.Context) (*plugin.Manifest, error) {
return &plugin.Manifest{
Name: "oracle",
Hooks: []plugin.Hook{
Hooks: []*plugin.Hook{
{
Name: "my-hook",
PlaceHookOn: "ignite chain build",
Expand All @@ -211,8 +221,8 @@ func (app) Manifest() (plugin.Manifest, error) {
}, nil
}

func (app) ExecuteHookPre(hook plugin.ExecutedHook) error {
switch hook.Name {
func (app) ExecuteHookPre(_ context.Context, h *plugin.ExecutedHook, _ plugin.ClientAPI) error {
switch h.Hook.GetName() {
case "my-hook":
fmt.Println("I'm executed before ignite chain build")
default:
Expand All @@ -221,8 +231,8 @@ func (app) ExecuteHookPre(hook plugin.ExecutedHook) error {
return nil
}

func (app) ExecuteHookPost(hook plugin.ExecutedHook) error {
switch hook.Name {
func (app) ExecuteHookPost(_ context.Context, h *plugin.ExecutedHook, _ plugin.ClientAPI) error {
switch h.Hook.GetName() {
case "my-hook":
fmt.Println("I'm executed after ignite chain build (if no error)")
default:
Expand All @@ -231,8 +241,8 @@ func (app) ExecuteHookPost(hook plugin.ExecutedHook) error {
return nil
}

func (app) ExecuteHookCleanUp(hook plugin.ExecutedHook) error {
switch hook.Name {
func (app) ExecuteHookCleanUp(_ context.Context, h *plugin.ExecutedHook, _ plugin.ClientAPI) error {
switch h.Hook.GetName() {
case "my-hook":
fmt.Println("I'm executed after ignite chain build (regardless errors)")
default:
Expand Down
Loading
Loading