Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactored requirement operation and added multi-target/multi-language compatibility #3487

Open
wants to merge 50 commits into
base: version/0-47-0-RC1
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 46 commits
Commits
Show all changes
50 commits
Select commit Hold shift + click to select a range
d859109
Implemented package install runner that is self contained (not using …
Naatan Sep 5, 2024
e5da35e
Implemented new requirement opts for `state platforms add`
Naatan Sep 6, 2024
2dacc37
Use new runner for `bundles install`
Naatan Sep 9, 2024
60c7ee1
Fix suggestions
Naatan Sep 9, 2024
ac10a21
languages install now uses new runner
Naatan Sep 9, 2024
589c236
Implemented uninstall runner
Naatan Sep 9, 2024
db3c6c6
Added `--expand` flag to `state manifest`
Naatan Sep 10, 2024
f72b5b1
Tests and error conditions for `state uninstall`
Naatan Sep 10, 2024
dabd14a
Added convenience method to run termtest command in debugger
Naatan Sep 10, 2024
a8e99ce
Enforce namespace type
Naatan Sep 10, 2024
c2aea6b
Bundle uninstall uses new runner
Naatan Sep 10, 2024
e8e9355
Implemented new runner for `state platforms remove`
Naatan Sep 10, 2024
7dc7511
Fixed debug output not supporting multiple invocations of same comman…
Naatan Sep 10, 2024
3f582c4
Remove debug code
Naatan Sep 10, 2024
448d343
Drop requirement operation and clean up
Naatan Sep 10, 2024
6b825d7
Add test for ParseNamespace
Naatan Sep 10, 2024
05927a6
Fix bundle uninstall not working
Naatan Sep 10, 2024
c8ef37f
Centralize commit error handling
Naatan Sep 10, 2024
38b3f86
Added JSON output for `state install`
Naatan Sep 11, 2024
48418db
Add json output for uninstall
Naatan Sep 11, 2024
6fef453
Added structured output for platforms add/remove
Naatan Sep 11, 2024
d3339da
Added standalone install test
Naatan Sep 11, 2024
1277e45
Drop redundant test code
Naatan Sep 11, 2024
c3efc42
Fix assertions
Naatan Sep 11, 2024
63914e7
Ensure our default platforms are prioritized
Naatan Sep 11, 2024
20e326d
Drop tests that are no longer relevant or redundant
Naatan Sep 11, 2024
98aca2e
Fix assertion
Naatan Sep 11, 2024
24cc190
Fix assertions
Naatan Sep 12, 2024
8f7920a
Remove unused locale
Naatan Sep 12, 2024
d35e45b
Make all our scripts standalone
Naatan Sep 12, 2024
0021f03
Add back auto wildcarding
Naatan Sep 12, 2024
e12c615
Reduce API requests
Naatan Sep 12, 2024
74e7e6a
Remove unused var
Naatan Sep 12, 2024
6d344d9
Fix assertion
Naatan Sep 13, 2024
2520098
Mark input errors
Naatan Sep 13, 2024
02ba527
Increase search timeout
Naatan Sep 13, 2024
d43b31a
Add comment explaining async config condition
Naatan Sep 13, 2024
0fa03bf
Drop badly worded but also useless comments
Naatan Sep 13, 2024
9d7a94a
Add comment for `SpawnDebuggerWithOpts`
Naatan Sep 13, 2024
b4ada37
Remove unused function
Naatan Sep 13, 2024
70a6f30
Rename method to be consistent
Naatan Sep 13, 2024
60bc059
Expectation can use full word
Naatan Sep 13, 2024
1ee70c9
Differentiate between solving and sourcing runtime timeouts
Naatan Sep 13, 2024
2f767aa
Retaining dirs is by definition something we ought to only do for deb…
Naatan Sep 13, 2024
6dc93a5
Merge remote-tracking branch 'origin/version/0-47-0-RC1' into DX-1897
Naatan Sep 13, 2024
05e069c
Add increased timeout
Naatan Sep 13, 2024
7a0c29d
Fix test; not guaranteed to return django
Naatan Sep 19, 2024
ed949e5
Satisfy newline checker which is apparently bugged cause I didn't tou…
Naatan Sep 19, 2024
4b0cca8
Fix unit test using removed function
Naatan Sep 19, 2024
2ae56e0
Give more time for double solve
Naatan Sep 19, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 37 additions & 4 deletions activestate.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,7 @@ constants:
scripts:
- name: install-deps-dev
language: bash
standalone: true
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm curious why this was added, and if this is needed for other activestate.yaml files with custom scripts.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because I find often while testing that it re-sources my runtime because I invoked it with another state tool version (ie. the one I'm working on vs my installed state tool). And end of the day none of our scripts actually require the runtime, they were always standalone, just never marked as such.

if: ne .Shell "cmd"
value: |
if ! type "go" &> /dev/null; then
Expand All @@ -73,11 +74,13 @@ scripts:
fi
- name: install-deps-os
language: bash
standalone: true
if: ne .OS.Name "Linux"
description: Install OS specific deps
value: ""
- name: install-deps-ci
language: bash
standalone: true
if: ne .Shell "cmd"
value: |
if { [[ "$GOOS" == "windows" ]] || [[ "$OS" == "Windows_NT" ]]; } && ! type "goversioninfo" &> /dev/null; then
Expand All @@ -86,6 +89,7 @@ scripts:
fi
- name: preprocess
language: bash
standalone: true
description: Generates assets required by the project that aren't just specific to the build
value: |
set -e
Expand All @@ -97,6 +101,7 @@ scripts:
fi
- name: build
language: bash
standalone: true
description: Builds the project with the host OS as the target OS.
value: |
set -e
Expand All @@ -111,6 +116,7 @@ scripts:
go build -tags "$GO_BUILD_TAGS" -o $BUILD_TARGET_DIR/$constants.BUILD_TARGET $constants.CLI_BUILDFLAGS $constants.CLI_PKGS
- name: build-for
language: bash
standalone: true
description: Builds the project with the specified OS as the target OS. (valid values darwin, linux, windows)
value: |
set -e
Expand All @@ -121,6 +127,7 @@ scripts:
go build -tags "internal $GO_BUILD_TAGS" -o ${2} $constants.CLI_BUILDFLAGS $constants.CLI_PKGS
- name: build-svc
language: bash
standalone: true
description: Builds the state-svc daemon
value: |
set -e
Expand All @@ -136,15 +143,16 @@ scripts:
- name: build-exec
description: Builds the State Executor application
language: bash
standalone: true
value: |
set -e
$constants.SET_ENV

go build -tags "$GO_BUILD_TAGS" -o $BUILD_TARGET_DIR/$constants.BUILD_EXEC_TARGET $constants.CLI_BUILDFLAGS $constants.EXECUTOR_PKGS
- name: build-all
description: Builds all our tools
standalone: true
language: bash
standalone: true
value: |
set -e
echo "Building State Tool"
Expand All @@ -159,6 +167,7 @@ scripts:
$scripts.build-exec.path()
- name: build-installer
language: bash
standalone: true
description: Builds the state-installer
value: |
set -e
Expand All @@ -167,6 +176,7 @@ scripts:
go build -tags "$GO_BUILD_TAGS" -o $BUILD_TARGET_DIR/$constants.BUILD_INSTALLER_TARGET $constants.INSTALLER_PKGS
- name: build-remote-installer
language: bash
standalone: true
description: Builds the state-remote-installer
value: |
set -e
Expand All @@ -181,12 +191,14 @@ scripts:
go build -tags "$GO_BUILD_TAGS" -o ../../$BUILD_TARGET_DIR/$TARGET .
- name: install
language: bash
standalone: true
description: Installs the current HEAD version into GOBIN
value: |
$constants.SET_ENV
go install $constants.CLI_BUILDFLAGS $CLI_PKGS
- name: deploy-updates
language: bash
standalone: true
description: Deploys update files to S3. This steps is automated by CI and should never be ran manually unless you KNOW WHAT YOU'RE DOING.
value: |
set -e
Expand Down Expand Up @@ -232,17 +244,20 @@ scripts:
cp installers/stop${constants.SCRIPT_EXT} $INSTALLERS_DIR/stop${constants.SCRIPT_EXT}
- name: deploy-installers
language: bash
standalone: true
description: Deploys update files to S3. This steps is automated by CI and should never be ran manually unless you KNOW WHAT YOU'RE DOING.
value: |
go run scripts/ci/s3-deployer/main.go build/installers us-east-1 state-tool update/state
- name: deploy-remote-installer
language: bash
standalone: true
value: |
set -e
$constants.SET_ENV
go run scripts/ci/s3-deployer/main.go $BUILD_TARGET_DIR/remote-installer us-east-1 state-tool remote-installer
- name: build-workflow-assets
language: bash
standalone: true
description: Generates our github workflows
value: |
$scripts.build-for.path() "windows" ./.github/deps/Windows/bin/state.exe
Expand All @@ -254,6 +269,7 @@ scripts:
GOOS=darwin go build -o .github/deps/macOS/bin/parallelize github.com/ActiveState/cli/scripts/ci/parallelize/
- name: update-workflow-assets
language: bash
standalone: true
description: Generates our github workflows
value: |
[ -z "${2}" ] && >&2 echo "Usage: update-workflow-assets [branch] [version]" && exit 1
Expand Down Expand Up @@ -286,12 +302,13 @@ scripts:
rm -Rf $tmpDir
- name: test
language: bash
standalone: true
description: Runs unit tests (not integration tests)
value: |
go test -v `go list ./... | grep -v integration | grep -v automation | grep -v expect | grep -v state-svc | grep -v state-offline` $@
standalone: true
- name: integration-tests
language: bash
standalone: true
description: Runs integration tests.
value: |
unset ACTIVESTATE_ACTIVATED
Expand All @@ -300,35 +317,39 @@ scripts:
go test `go list ./... | grep "${INTEGRATION_TEST_REGEX}"` -v "${@:1}" -timeout 20m
- name: integration-tests-build-check
language: bash
standalone: true
description: Builds integration tests and removes the executable artifact(s).
value: |
out="x.test"
go list ./... | grep "${INTEGRATION_TEST_REGEX}" | xargs -L 1 go test -o $out -c
[ -f $out ] && rm $out
- name: clean
language: bash
standalone: true
description: Cleans out the build dir.
value: |
go clean
rm -Rf build
- name: run
language: bash
standalone: true
description: Builds the State Tool and runs it with `--help`
value: |
$scripts.build.path()
build/state --help
- name: debug
language: bash
description: "Runs a remote debugger that can be hooked into from your IDE. Example usage: `state run debug activate` (will debug `state activate`)"
standalone: true
description: "Runs a remote debugger that can be hooked into from your IDE. Example usage: `state run debug activate` (will debug `state activate`)"
value: dlv debug --headless --listen=:2346 --api-version=2 github.com/ActiveState/cli/cmd/state -- $@
- name: scripted
language: bash
description: "Runs a command via 'go run'"
standalone: true
description: "Runs a command via 'go run'"
value: go run github.com/ActiveState/cli/cmd/state $@
- name: story-cleanup
language: bash
standalone: true
description: "Runs Python script to move old stories from primary project to storage project"
value: |
export PT_API_TOKEN=$secrets.project.PT_API_TOKEN
Expand All @@ -338,12 +359,14 @@ scripts:
python3 ./scripts/story-cleanup/story-cleanup.py
- name: lint
language: bash
standalone: true
description: "Runs linting for untracked and unstaged changes (if any), or staged changes"
value: |
golangci-lint run --new
actionlint
- name: lint-staged
language: bash
standalone: true
description: "Runs linting for staged changes (skipping untracked and unstaged-only files)"
value: |
golangci-lint run \
Expand All @@ -356,12 +379,14 @@ scripts:
actionlint $actionfiles
- name: lint-all
language: bash
standalone: true
description: "Runs linting for all files"
value: |
golangci-lint run --no-config
actionlint
- name: check-format
language: bash
standalone: true
description: "Checks if the code is formatted correctly"
value: |
set -e
Expand Down Expand Up @@ -392,48 +417,55 @@ scripts:
fi
- name: grab-mergecommits
language: bash
standalone: true
value: |
export JIRA_USERNAME=${secrets.user.JIRA_USERNAME}
export JIRA_TOKEN=${secrets.user.JIRA_TOKEN}
export GITHUB_TOKEN=${secrets.user.GITHUB_TOKEN}
go run $project.path()/scripts/grab-mergecommits/main.go $1
- name: target-version-pr
language: bash
standalone: true
value: |
export JIRA_USERNAME=${secrets.user.JIRA_USERNAME}
export JIRA_TOKEN=${secrets.user.JIRA_TOKEN}
export GITHUB_TOKEN=${secrets.user.GITHUB_TOKEN}
go run $project.path()/scripts/ci/target-version-pr/main.go $1
- name: create-version-pr
language: bash
standalone: true
value: |
export JIRA_USERNAME=${secrets.user.JIRA_USERNAME}
export JIRA_TOKEN=${secrets.user.JIRA_TOKEN}
export GITHUB_TOKEN=${secrets.user.GITHUB_TOKEN}
go run $project.path()/scripts/create-version-pr/main.go $1
- name: propagate-pr
language: bash
standalone: true
value: |
export JIRA_USERNAME=${secrets.user.JIRA_USERNAME}
export JIRA_TOKEN=${secrets.user.JIRA_TOKEN}
export GITHUB_TOKEN=${secrets.user.GITHUB_TOKEN}
go run $project.path()/scripts/ci/propagate-pr/main.go $1
- name: verify-pr
language: bash
standalone: true
value: |
export JIRA_USERNAME=${secrets.user.JIRA_USERNAME}
export JIRA_TOKEN=${secrets.user.JIRA_TOKEN}
export GITHUB_TOKEN=${secrets.user.GITHUB_TOKEN}
go run $project.path()/scripts/ci/verify-pr/main.go $1
- name: start-story
language: bash
standalone: true
value: |
export JIRA_USERNAME=${secrets.user.JIRA_USERNAME}
export JIRA_TOKEN=${secrets.user.JIRA_TOKEN}
export GITHUB_TOKEN=${secrets.user.GITHUB_TOKEN}
go run $project.path()/scripts/start-story/main.go "$@"
- name: ghapi
language: bash
standalone: true
value: |
curl \
-H "Accept: application/vnd.github+json" \
Expand All @@ -442,6 +474,7 @@ scripts:
- name: benchmark-exec
if: eq .OS.Name "Linux"
language: bash
standalone: true
description: "Benchmarks executable leveraging highly sensitive/accurate tooling"
value: |
# example usage:
Expand Down
30 changes: 21 additions & 9 deletions cmd/state/internal/cmdtree/bundles.go
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,9 @@ import (
"github.com/ActiveState/cli/internal/captain"
"github.com/ActiveState/cli/internal/locale"
"github.com/ActiveState/cli/internal/primer"
"github.com/ActiveState/cli/internal/runners/install"
"github.com/ActiveState/cli/internal/runners/packages"
"github.com/ActiveState/cli/internal/runners/uninstall"
"github.com/ActiveState/cli/pkg/platform/model"
)

Expand Down Expand Up @@ -43,9 +45,8 @@ func newBundlesCommand(prime *primer.Values) *captain.Command {
}

func newBundleInstallCommand(prime *primer.Values) *captain.Command {
runner := packages.NewInstall(prime)

params := packages.InstallRunParams{}
runner := install.New(prime, model.NamespaceBundle)
params := install.Params{}

return captain.NewCommand(
"install",
Expand All @@ -61,16 +62,22 @@ func newBundleInstallCommand(prime *primer.Values) *captain.Command {
Required: true,
},
},
func(_ *captain.Command, _ []string) error {
return runner.Run(params, model.NamespaceBundle)
func(_ *captain.Command, args []string) error {
for _, p := range args {
_, err := params.Packages.Add(p)
if err != nil {
return locale.WrapInputError(err, "err_install_packages_args", "Invalid install arguments")
}
}
return runner.Run(params)
},
).SetSupportsStructuredOutput()
}

func newBundleUninstallCommand(prime *primer.Values) *captain.Command {
runner := packages.NewUninstall(prime)
runner := uninstall.New(prime, model.NamespaceBundle)

params := packages.UninstallRunParams{}
params := uninstall.Params{}

return captain.NewCommand(
"uninstall",
Expand All @@ -86,8 +93,13 @@ func newBundleUninstallCommand(prime *primer.Values) *captain.Command {
Required: true,
},
},
func(_ *captain.Command, _ []string) error {
return runner.Run(params, model.NamespaceBundle)
func(_ *captain.Command, args []string) error {
for _, p := range args {
if _, err := params.Packages.Add(p); err != nil {
return locale.WrapInputError(err, "err_uninstall_packages_args", "Invalid package uninstall arguments")
}
}
return runner.Run(params)
},
).SetSupportsStructuredOutput()
}
Expand Down
20 changes: 14 additions & 6 deletions cmd/state/internal/cmdtree/languages.go
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,9 @@ import (
"github.com/ActiveState/cli/internal/captain"
"github.com/ActiveState/cli/internal/locale"
"github.com/ActiveState/cli/internal/primer"
"github.com/ActiveState/cli/internal/runners/install"
"github.com/ActiveState/cli/internal/runners/languages"
"github.com/ActiveState/cli/pkg/platform/model"
)

func newLanguagesCommand(prime *primer.Values) *captain.Command {
Expand All @@ -24,9 +26,8 @@ func newLanguagesCommand(prime *primer.Values) *captain.Command {
}

func newLanguageInstallCommand(prime *primer.Values) *captain.Command {
runner := languages.NewUpdate(prime)

params := languages.UpdateParams{}
runner := install.New(prime, model.NamespaceLanguage)
params := install.Params{}

return captain.NewCommand(
"install",
Expand All @@ -39,11 +40,18 @@ func newLanguageInstallCommand(prime *primer.Values) *captain.Command {
Name: "language",
Description: locale.T("arg_languages_install_description"),
Required: true,
Value: &params.Language,
Value: &params.Packages,
},
},
func(ccmd *captain.Command, _ []string) error {
return runner.Run(&params)
func(ccmd *captain.Command, args []string) error {
for _, p := range args {
pkg, err := params.Packages.Add(p)
if err != nil {
return locale.WrapInputError(err, "err_install_packages_args", "Invalid install arguments")
}
pkg.Namespace = model.NamespaceLanguage.String()
}
return runner.Run(params)
},
).SetSupportsStructuredOutput()
}
Expand Down
Loading
Loading