Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: tool for comparing different ignite versions #3718

Merged
merged 112 commits into from
Apr 19, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
112 commits
Select commit Hold shift + click to select a range
398b199
Add gen-migdoc script
Ehsan-saradar Oct 30, 2023
6a0d76e
Fix gen-migdoc script
Ehsan-saradar Oct 30, 2023
a588130
Merge branch 'main' into feat/auto-migdoc
Pantani Oct 31, 2023
0fd9a83
Merge branch 'main' into feat/auto-migdoc
Pantani Oct 31, 2023
dc39cf4
Merge branch 'main' into feat/auto-migdoc
Pantani Nov 7, 2023
50b63b0
Merge branch 'main' into feat/auto-migdoc
Pantani Nov 7, 2023
e5bd249
Add gen-mig-docs tool
Ehsan-saradar Nov 13, 2023
0dc54bd
Add exceptions and more test cases to migdoc
Ehsan-saradar Nov 18, 2023
2cad27b
Add diff subtraction and cleanup
Ehsan-saradar Nov 21, 2023
8917a47
Minor improvements
Ehsan-saradar Nov 27, 2023
0212ae7
Remove gen-migdoc bash script
Ehsan-saradar Nov 27, 2023
d05e49f
Merge branch 'main' into feat/auto-migdoc
Ehsan-saradar Jan 31, 2024
6406bab
Move exception files to different file
Ehsan-saradar Feb 1, 2024
c0151e8
Change the diff subtraction algorithm
Ehsan-saradar Feb 5, 2024
90b2d30
Clean up error handling
Ehsan-saradar Feb 6, 2024
eaeceee
Add diff package
Ehsan-saradar Feb 7, 2024
acabaaa
Add Subtract to diff package
Ehsan-saradar Feb 7, 2024
7ba7ece
Cleanup gen-mig-diffs
Ehsan-saradar Feb 8, 2024
23f7aad
Fix bugs
Ehsan-saradar Feb 8, 2024
49a8689
Fix bugs
Ehsan-saradar Feb 8, 2024
ad123b4
Add more test to subtract test cases
Ehsan-saradar Feb 9, 2024
6bdad4d
Update the subtract lines algorithm
Ehsan-saradar Feb 9, 2024
4b0ceff
Merge branch 'main' into feat/auto-migdoc
Ehsan-saradar Feb 9, 2024
8c3e661
go mod tidy
Ehsan-saradar Feb 9, 2024
075e46f
Add changelog
Ehsan-saradar Feb 9, 2024
8106e37
Fix lint errors
Ehsan-saradar Feb 9, 2024
f4368f4
Fix bugs
Ehsan-saradar Feb 9, 2024
7925875
Fix lint
Ehsan-saradar Feb 9, 2024
d4e7e0e
Merge branch 'main' into feat/auto-migdoc
Ehsan-saradar Feb 12, 2024
524b18c
Merge branch 'main' into feat/auto-migdoc
Ehsan-saradar Feb 14, 2024
e8e96c3
make format
Ehsan-saradar Feb 14, 2024
6b9cb95
Fix cleanup
Ehsan-saradar Feb 14, 2024
a25072f
Update ignite/internal/tools/gen-mig-diffs/migdiff/gen.go
Ehsan-saradar Feb 14, 2024
dda1437
Update ignite/internal/tools/gen-mig-diffs/migdiff/gen.go
Ehsan-saradar Feb 14, 2024
84c677c
Merge branch 'main' into feat/auto-migdoc
Ehsan-saradar Feb 14, 2024
1aac568
Fix lint
Ehsan-saradar Feb 14, 2024
08276af
Use cliui instead of log package
Ehsan-saradar Feb 15, 2024
e3acd22
Merge branch 'main' into feat/auto-migdoc
salmad3 Feb 16, 2024
124be65
Merge branch 'main' into feat/auto-migdoc
salmad3 Feb 22, 2024
8c8dc5b
Merge branch 'main' into feat/auto-migdoc
Pantani Feb 23, 2024
9636346
add gen options
Feb 23, 2024
f9df2d3
code clean
Feb 27, 2024
9c13c15
improve error log
Feb 27, 2024
9e7b2ae
reset and clean the repo before the checkout to avoid conflicts
Feb 27, 2024
8d1a7c4
use https for repo url
Feb 27, 2024
0d90d3d
draft refactor
Feb 27, 2024
ccd1cc7
improve the logs
Feb 27, 2024
94b8108
Merge remote-tracking branch 'origin/main' into feat/auto-migdoc
Feb 28, 2024
58d2d5d
remove useless code
Feb 28, 2024
33e64c2
add pkg options
Feb 29, 2024
cd9e917
pass the context to the commands
Feb 29, 2024
9b2f891
add cache file
Feb 29, 2024
381a16a
create Scaffold struct
Feb 29, 2024
0fd57fb
fix wrong output
Feb 29, 2024
ddedaf3
add the scaffold cache
Feb 29, 2024
60b96e9
fix folder search for diffs
Feb 29, 2024
4e6dd60
improve the logs and comments
Feb 29, 2024
302cc9e
remove unused temp folder creation
Feb 29, 2024
57ea7b3
fix binaries path
Mar 1, 2024
6f7c06e
generate migration docs inside a doc template
Mar 1, 2024
ba21ba9
Merge branch 'main' into feat/auto-migdoc
Pantani Mar 1, 2024
ee3ff49
format the markdown file
Mar 1, 2024
35279f4
unhandled error
Mar 1, 2024
4ce3fc6
fix changelog
Mar 1, 2024
82f9b7d
markdown breakline
Mar 1, 2024
e5d23fd
add CI to generate migration docs for each release
Mar 2, 2024
49d1ee2
skip diff if is empty
Mar 7, 2024
f16d265
Merge branch 'main' into feat/auto-migdoc
Pantani Mar 7, 2024
eeb33a6
Merge branch 'main' into feat/auto-migdoc
Pantani Mar 8, 2024
29d148d
use script to gen mig file
Mar 8, 2024
02b507f
Merge branch 'main' into feat/auto-migdoc
Pantani Mar 8, 2024
31971fd
Merge remote-tracking branch 'origin/main' into feat/auto-migdoc
Mar 8, 2024
9624d19
Merge branch 'main' into feat/auto-migdoc
Pantani Mar 11, 2024
72d5eef
Merge branch 'main' into feat/auto-migdoc
Pantani Mar 14, 2024
770d57b
Merge branch 'main' into feat/auto-migdoc
Pantani Mar 19, 2024
633abcb
Merge branch 'main' into feat/auto-migdoc
Pantani Mar 19, 2024
273a4cc
Merge branch 'main' into feat/auto-migdoc
Pantani Mar 19, 2024
68c9862
fix link for the genesis file
Mar 19, 2024
d780280
Merge branch 'fix/doc-genesis-link' into feat/auto-migdoc
Mar 19, 2024
c663fa9
Merge branch 'main' into feat/auto-migdoc
Pantani Mar 19, 2024
b44ebe5
Merge branch 'main' into feat/auto-migdoc
Pantani Mar 20, 2024
72c0c1f
Merge branch 'main' into feat/auto-migdoc
Pantani Mar 20, 2024
9a15942
use v29 import instead v28
Mar 20, 2024
2603157
Merge branch 'main' into feat/auto-migdoc
Pantani Mar 20, 2024
1235f88
Merge branch 'main' into feat/auto-migdoc
Pantani Mar 22, 2024
9fa39dd
Merge branch 'main' into feat/auto-migdoc
Pantani Mar 23, 2024
b528666
Merge branch 'main' into feat/auto-migdoc
Pantani Mar 28, 2024
f00c319
Merge branch 'main' into feat/auto-migdoc
Pantani Mar 28, 2024
94e348c
Merge branch 'main' into feat/auto-migdoc
Pantani Mar 28, 2024
935b55e
add new xgenny runner
Mar 29, 2024
503e69c
improve logs
Mar 29, 2024
9ff0b82
Merge branch 'main' into feat/auto-migdoc
Pantani Mar 29, 2024
e46e01d
Update ignite/internal/tools/gen-mig-diffs/cmd/root.go
Pantani Apr 1, 2024
f3e4e31
Merge branch 'main' into feat/auto-migdoc
Pantani Apr 2, 2024
74dc385
improve the logic if the repo-output already exist
Apr 3, 2024
da6dc8d
remove repo cleanup flag
Apr 3, 2024
3d05463
fix lint issue
Apr 3, 2024
858d156
fix diff output format
Apr 3, 2024
7b9f0ac
fix wrong log output
Apr 3, 2024
561a4d9
improve md format
Apr 3, 2024
68fd786
Merge branch 'main' into feat/auto-migdoc
Pantani Apr 6, 2024
b409a88
create a go.mod for the diff tool
Apr 6, 2024
d6cf7d7
change the go.mod namespace
Apr 7, 2024
e5a7841
Merge remote-tracking branch 'origin/main' into feat/auto-migdoc
Apr 8, 2024
7b94c77
Merge branch 'main' into feat/auto-migdoc
Pantani Apr 12, 2024
b73cd6d
Merge branch 'main' into feat/auto-migdoc
Pantani Apr 16, 2024
e33efdc
Merge branch 'main' into feat/auto-migdoc
Pantani Apr 17, 2024
c3b422a
rollback go.mod
Apr 17, 2024
2224a5f
Merge branch 'main' into feat/auto-migdoc
Pantani Apr 18, 2024
f74c50f
Merge branch 'main' into feat/auto-migdoc
Pantani Apr 18, 2024
550eb2c
break the compute function and add unit tests
Apr 18, 2024
aca6a09
Merge branch 'main' into feat/auto-migdoc
Pantani Apr 19, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 34 additions & 0 deletions .github/workflows/gen-docs-migration.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
name: Generate Migration Docs
on:
release:
types: [ published ]

jobs:
cli:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4

- uses: actions/setup-go@v5
with:
go-version: 'stable'

- name: Generate Scaffold Migration Docs
run: ./scripts/gen-mig-diffs

- name: Create Pull Request
id: cpr
uses: peter-evans/create-pull-request@v6
with:
title: "docs(migration): update generated docs"
commit-message: "docs(migration): update generated docs"
body: ""
branch: feat/gen-migration-docs
add-paths: |
docs/

- name: Check outputs
run: |
echo "Pull Request Number - ${{ steps.cpr.outputs.pull-request-number }}"
echo "Pull Request URL - ${{ steps.cpr.outputs.pull-request-url }}"

1 change: 1 addition & 0 deletions changelog.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@
- [#4001](https://github.com/ignite/cli/pull/4001) Improve `xgenny` dry run
- [#3967](https://github.com/ignite/cli/issues/3967) Add HD wallet parameters `address index` and `account number` to the chain account config
- [#4004](https://github.com/ignite/cli/pull/4004) Remove all import placeholders using the `xast` pkg
- [#3718](https://github.com/ignite/cli/pull/3718) Add `gen-mig-diffs` tool app to compare scaffold output of two versions of ignite
- [#4077](https://github.com/ignite/cli/pull/4077) Merge the swagger files manually instead use nodetime `swagger-combine`

### Changes
Expand Down
183 changes: 183 additions & 0 deletions ignite/internal/tools/gen-mig-diffs/cmd/root.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,183 @@
package cmd

import (
"fmt"
"path/filepath"

"github.com/Masterminds/semver/v3"
"github.com/spf13/cobra"

"github.com/ignite/cli/v29/ignite/pkg/cliui"
"github.com/ignite/cli/v29/ignite/pkg/errors"
"github.com/ignite/cli/v29/ignite/pkg/xgenny"

"github.com/ignite/cli/ignite/internal/tools/gen-mig-diffs/pkg/diff"
"github.com/ignite/cli/ignite/internal/tools/gen-mig-diffs/pkg/repo"
"github.com/ignite/cli/ignite/internal/tools/gen-mig-diffs/pkg/scaffold"
"github.com/ignite/cli/ignite/internal/tools/gen-mig-diffs/templates/doc"
)

const (
flagFrom = "from"
flagTo = "to"
flagOutput = "output"
flagSource = "repo-source"
flagRepoURL = "repo-url"
flagRepoOutput = "repo-output"
flagScaffoldOutput = "scaffold-output"
flagScaffoldCache = "scaffold-cache"

defaultDocPath = "docs/docs/06-migration"
)

// NewRootCmd creates a new root command.
func NewRootCmd() *cobra.Command {
cmd := &cobra.Command{
Use: "gen-mig-diffs",
Short: "GenerateBinaries migration diffs",
Long: "This tool is used to generate migration diff files for each of ignites scaffold commands",
RunE: func(cmd *cobra.Command, args []string) error {
session := cliui.New()
defer session.End()

var (
from, _ = cmd.Flags().GetString(flagFrom)
to, _ = cmd.Flags().GetString(flagTo)
repoSource, _ = cmd.Flags().GetString(flagSource)
output, _ = cmd.Flags().GetString(flagOutput)
repoURL, _ = cmd.Flags().GetString(flagRepoURL)
repoOutput, _ = cmd.Flags().GetString(flagRepoOutput)
scaffoldOutput, _ = cmd.Flags().GetString(flagScaffoldOutput)
scaffoldCache, _ = cmd.Flags().GetString(flagScaffoldCache)
)
fromVer, err := semver.NewVersion(from)
if err != nil && from != "" {
return errors.Wrapf(err, "failed to parse from version %s", from)
}
toVer, err := semver.NewVersion(to)
if err != nil && to != "" {
return errors.Wrapf(err, "failed to parse to version %s", to)
}

// Check or download the source and generate the binaries for each version.
repoOptions := make([]repo.Options, 0)
if repoSource != "" {
repoOptions = append(repoOptions, repo.WithSource(repoSource))
}
if repoURL != "" {
repoOptions = append(repoOptions, repo.WithRepoURL(repoURL))
}
if repoOutput != "" {
repoOptions = append(repoOptions, repo.WithRepoOutput(repoOutput))
}

igniteRepo, err := repo.New(fromVer, toVer, session, repoOptions...)
if err != nil {
return err
}
defer igniteRepo.Cleanup()

releaseDescription, err := igniteRepo.ReleaseDescription()
if err != nil {
return errors.Wrapf(err, "failed to fetch the release tag %s description", igniteRepo.To.Original())
}

fromBin, toBin, err := igniteRepo.GenerateBinaries(cmd.Context())
if err != nil {
return err
}

// Scaffold the default commands for each version.
scaffoldOptions := make([]scaffold.Options, 0)
if scaffoldOutput != "" {
scaffoldOptions = append(scaffoldOptions, scaffold.WithOutput(scaffoldOutput))
}
if scaffoldCache != "" {
scaffoldOptions = append(scaffoldOptions, scaffold.WithCachePath(scaffoldCache))
}

session.StartSpinner(fmt.Sprintf("Running scaffold commands for %s...", igniteRepo.From.Original()))
sFrom, err := scaffold.New(fromBin, igniteRepo.From, scaffoldOptions...)
if err != nil {
return err
}
defer sFrom.Cleanup()

if err := sFrom.Run(cmd.Context()); err != nil {
return err
}
session.StopSpinner()
session.EventBus().SendInfo(fmt.Sprintf("Scaffolded code for %s at %s", igniteRepo.From.Original(), sFrom.Output))

session.StartSpinner(fmt.Sprintf("Running scaffold commands for %s...", igniteRepo.To.Original()))
sTo, err := scaffold.New(toBin, igniteRepo.To, scaffoldOptions...)
if err != nil {
return err
}
defer sTo.Cleanup()

if err := sTo.Run(cmd.Context()); err != nil {
return err
}
session.StopSpinner()
session.EventBus().SendInfo(fmt.Sprintf("Scaffolded code for %s at %s", igniteRepo.To.Original(), sTo.Output))

// Calculate and save the diffs from the scaffolded code.
session.StartSpinner("Calculating diff...")
diffs, err := diff.CalculateDiffs(sFrom.Output, sTo.Output)
if err != nil {
return errors.Wrap(err, "failed to calculate diff")
}

formatedDiffs, err := diff.FormatDiffs(diffs)
if err != nil {
return errors.Wrap(err, "failed to save diff map")
}
session.StopSpinner()
session.EventBus().SendInfo("Diff calculated successfully")

output, err = filepath.Abs(output)
if err != nil {
return errors.Wrap(err, "failed to find the abs path")
}

// Generate the docs file.
g, err := doc.NewGenerator(doc.Options{
Path: output,
FromVersion: igniteRepo.From,
ToVersion: igniteRepo.To,
Diffs: string(formatedDiffs),
Description: releaseDescription,
})
if err != nil {
return errors.Wrap(err, "failed to create the doc generator object")
}

sm, err := xgenny.NewRunner(cmd.Context(), output).RunAndApply(g)
if err != nil {
return err
}

files := append(sm.CreatedFiles(), sm.ModifiedFiles()...)
if len(files) == 0 {
return errors.Errorf("migration doc not created at %s", output)
}
session.EventBus().SendInfo(
fmt.Sprintf("Migration doc generated successfully at %s", files[0]),
)

return nil
},
}

cmd.Flags().StringP(flagFrom, "f", "", "Version of Ignite or path to Ignite source code to generate the diff from")
cmd.Flags().StringP(flagTo, "t", "", "Version of Ignite or path to Ignite source code to generate the diff to")
cmd.Flags().StringP(flagOutput, "o", defaultDocPath, "Output directory to save the migration document")
cmd.Flags().StringP(flagSource, "s", "", "Path to Ignite source code repository. Set the source automatically set the cleanup to false")
cmd.Flags().String(flagRepoURL, repo.DefaultRepoURL, "Git URL for the Ignite repository")
cmd.Flags().String(flagRepoOutput, "", "Output path to clone the Ignite repository")
cmd.Flags().String(flagScaffoldOutput, "", "Output path to clone the Ignite repository")
cmd.Flags().String(flagScaffoldCache, "", "Path to cache directory")

return cmd
}
93 changes: 93 additions & 0 deletions ignite/internal/tools/gen-mig-diffs/go.mod
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
module github.com/ignite/cli/ignite/internal/tools/gen-mig-diffs

go 1.22.2

replace github.com/ignite/cli/v29 => ../../../../

require (
github.com/Masterminds/semver/v3 v3.2.1
github.com/go-git/go-git/v5 v5.12.0
github.com/gobuffalo/genny/v2 v2.1.0
github.com/gobuffalo/plush/v4 v4.1.19
github.com/gobwas/glob v0.2.3
github.com/hexops/gotextdiff v1.0.3
github.com/ignite/cli/v29 v29.0.0-00010101000000-000000000000
github.com/spf13/cobra v1.8.0
github.com/stretchr/testify v1.9.0
)

require (
dario.cat/mergo v1.0.0 // indirect
github.com/AlecAivazis/survey/v2 v2.3.7 // indirect
github.com/Microsoft/go-winio v0.6.1 // indirect
github.com/ProtonMail/go-crypto v1.0.0 // indirect
github.com/aymanbagabas/go-osc52/v2 v2.0.1 // indirect
github.com/aymerick/douceur v0.2.0 // indirect
github.com/briandowns/spinner v1.23.0 // indirect
github.com/charmbracelet/lipgloss v0.10.0 // indirect
github.com/chzyer/readline v1.5.1 // indirect
github.com/cloudflare/circl v1.3.7 // indirect
github.com/cockroachdb/errors v1.11.1 // indirect
github.com/cockroachdb/logtags v0.0.0-20230118201751-21c54148d20b // indirect
github.com/cockroachdb/redact v1.1.5 // indirect
github.com/cyphar/filepath-securejoin v0.2.4 // indirect
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect
github.com/emicklei/proto v1.13.2 // indirect
github.com/emicklei/proto-contrib v0.16.0 // indirect
github.com/emirpasic/gods v1.18.1 // indirect
github.com/fatih/color v1.16.0 // indirect
github.com/fatih/structs v1.1.0 // indirect
github.com/getsentry/sentry-go v0.27.0 // indirect
github.com/go-git/gcfg v1.5.1-0.20230307220236-3a3c6141e376 // indirect
github.com/go-git/go-billy/v5 v5.5.0 // indirect
github.com/gobuffalo/flect v1.0.2 // indirect
github.com/gobuffalo/github_flavored_markdown v1.1.4 // indirect
github.com/gobuffalo/helpers v0.6.7 // indirect
github.com/gobuffalo/logger v1.0.7 // indirect
github.com/gobuffalo/packd v1.0.2 // indirect
github.com/gobuffalo/tags/v3 v3.1.4 // indirect
github.com/gobuffalo/validate/v3 v3.3.3 // indirect
github.com/gofrs/uuid v4.4.0+incompatible // indirect
github.com/gogo/protobuf v1.3.2 // indirect
github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect
github.com/gorilla/css v1.0.1 // indirect
github.com/iancoleman/strcase v0.3.0 // indirect
github.com/inconshreveable/mousetrap v1.1.0 // indirect
github.com/jbenet/go-context v0.0.0-20150711004518-d14ea06fba99 // indirect
github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51 // indirect
github.com/kevinburke/ssh_config v1.2.0 // indirect
github.com/kr/pretty v0.3.1 // indirect
github.com/kr/text v0.2.0 // indirect
github.com/lucasb-eyer/go-colorful v1.2.0 // indirect
github.com/manifoldco/promptui v0.9.0 // indirect
github.com/mattn/go-colorable v0.1.13 // indirect
github.com/mattn/go-isatty v0.0.20 // indirect
github.com/mattn/go-runewidth v0.0.15 // indirect
github.com/mgutz/ansi v0.0.0-20200706080929-d51e80ef957d // indirect
github.com/microcosm-cc/bluemonday v1.0.26 // indirect
github.com/muesli/reflow v0.3.0 // indirect
github.com/muesli/termenv v0.15.2 // indirect
github.com/pjbgf/sha1cd v0.3.0 // indirect
github.com/pkg/errors v0.9.1 // indirect
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 // indirect
github.com/rivo/uniseg v0.4.7 // indirect
github.com/rogpeppe/go-internal v1.12.0 // indirect
github.com/sergi/go-diff v1.3.2-0.20230802210424-5b0b94c5c0d3 // indirect
github.com/sirupsen/logrus v1.9.3 // indirect
github.com/skeema/knownhosts v1.2.2 // indirect
github.com/sourcegraph/annotate v0.0.0-20160123013949-f4cad6c6324d // indirect
github.com/sourcegraph/syntaxhighlight v0.0.0-20170531221838-bd320f5d308e // indirect
github.com/spf13/pflag v1.0.5 // indirect
github.com/xanzy/ssh-agent v0.3.3 // indirect
golang.org/x/crypto v0.22.0 // indirect
golang.org/x/exp v0.0.0-20240404231335-c0f41cb1a7a0 // indirect
golang.org/x/mod v0.17.0 // indirect
golang.org/x/net v0.24.0 // indirect
golang.org/x/sync v0.7.0 // indirect
golang.org/x/sys v0.19.0 // indirect
golang.org/x/term v0.19.0 // indirect
golang.org/x/text v0.14.0 // indirect
golang.org/x/tools v0.20.0 // indirect
gopkg.in/warnings.v0 v0.1.2 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
)
Loading
Loading