Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Tighter Acceptance Criteria in TestCustomResources #6006

Closed
coryrc opened this issue Nov 12, 2019 · 13 comments
Closed

Tighter Acceptance Criteria in TestCustomResources #6006

coryrc opened this issue Nov 12, 2019 · 13 comments
Labels
area/test-and-release It flags unit/e2e/conformance/perf test issues for product features good first issue Denotes an issue ready for a new contributor, according to the "help wanted" guidelines. kind/bug Categorizes issue or PR as related to a bug. lifecycle/frozen Indicates that an issue or PR should not be auto-closed due to staleness. triage/accepted Issues which should be fixed (post-triage)

Comments

@coryrc
Copy link
Contributor

coryrc commented Nov 12, 2019

/area test-and-release

serving/test/conformance/api/v1/resources_test.go

Line 114 in aba4def

if err := pokeCowForMB(500); err == nil {

this accepts any indication other than StatusOK, like BadRequest, rather than just not getting a response/closed socket. Should be more precise in acceptance.

/assign
@coryrc coryrc added the kind/bug Categorizes issue or PR as related to a bug. label Nov 12, 2019
@knative-prow-robot knative-prow-robot added the area/test-and-release It flags unit/e2e/conformance/perf test issues for product features label Nov 12, 2019
@coryrc
Copy link
Contributor Author

coryrc commented Nov 12, 2019

/assign @coryrc

@coryrc
Copy link
Contributor Author

coryrc commented Nov 14, 2019

Also move to e2e to address this comment: #6014 (comment)

This was referenced Nov 14, 2019
Closed
Closed
coryrc added a commit to coryrc/serving that referenced this issue Jan 14, 2020
@coryrc
Fix TestCustomResourceLimits to be a conformance test
cc512cf 
Include changes from knative#6006 work
coryrc added a commit to coryrc/serving that referenced this issue Jan 15, 2020
@coryrc
Fix TestCustomResourceLimits to be a conformance test
6d43264 
Include changes from knative#6006 work
coryrc added a commit to coryrc/serving that referenced this issue Jan 15, 2020
@coryrc
Fix TestCustomResourceLimits to be a conformance test
0410be2 
Include changes from knative#6006 work
coryrc added a commit to coryrc/serving that referenced this issue Jan 15, 2020
@coryrc
Fix TestCustomResourceLimits to be a conformance test
dd5e3ed 
Include changes from knative#6006 work
coryrc added a commit to coryrc/serving that referenced this issue Jan 15, 2020
@coryrc
Fix TestCustomResourceLimits to be a conformance test
077aa1e 
Include changes from knative#6006 work
coryrc added a commit to coryrc/serving that referenced this issue Jan 18, 2020
@coryrc
Fix TestCustomResourceLimits to be a conformance test
77a781d 
Include changes from knative#6006 work
coryrc added a commit to coryrc/serving that referenced this issue Jan 22, 2020
@coryrc
Fix TestCustomResourceLimits to be a conformance test
bbb8079 
Include changes from knative#6006 work
coryrc added a commit to coryrc/serving that referenced this issue Jan 22, 2020
@coryrc
Fix TestCustomResourceLimits to be a conformance test
6e0f951 
Include changes from knative#6006 work
coryrc added a commit to coryrc/serving that referenced this issue Jan 25, 2020
@coryrc
Fix TestCustomResourceLimits to be a conformance test
e1abb8b 
Include changes from knative#6006 work
coryrc added a commit to coryrc/serving that referenced this issue Jan 25, 2020
@coryrc
Fix TestCustomResourceLimits to be a conformance test
a121303 
Include changes from knative#6006 work
coryrc added a commit to coryrc/serving that referenced this issue Jan 25, 2020
@coryrc
Fix TestCustomResourceLimits to be a conformance test
9ed11e0 
Include changes from knative#6006 work
coryrc added a commit to coryrc/serving that referenced this issue Jan 25, 2020
@coryrc
Fix TestCustomResourceLimits to be a conformance test
319f87c 
Include changes from knative#6006 work
@knative-housekeeping-robot
Copy link

Issues go stale after 90 days of inactivity.
Mark the issue as fresh by adding the comment /remove-lifecycle stale.
Stale issues rot after an additional 30 days of inactivity and eventually close.
If this issue is safe to close now please do so by adding the comment /close.

Send feedback to Knative Productivity Slack channel or file an issue in knative/test-infra.

/lifecycle stale

@knative-prow-robot knative-prow-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Feb 13, 2020
coryrc added a commit to coryrc/serving that referenced this issue Feb 23, 2020
@coryrc
Restructure Conformance API tests to match Spec
0c07394 
Fixes knative#6006 by making TestCustomResourceLimits to be a conformance
test and moving e2e parts out.

For knative#6226 fixes it for TestContainerErrorMsg and
TestContainerExitingMsg, though could be other tests checking errors
incorrectly.

Addresses knative#6264 though continuing work is needed.

Don't mix E2E and conformance:
Strip E2E portions of conformance out into tests in the test/e2e
directory. Commonality goes into test/scenarios or test/v1 depending
on specificity of the common functionality.

Rename to more align with runtime test naming (Must/Should).
@coryrc coryrc mentioned this issue Feb 23, 2020
Closed
coryrc added a commit to coryrc/serving that referenced this issue Feb 28, 2020
@coryrc
Restructure Conformance API tests to match Spec
84acbac 
Fixes knative#6006 by making TestCustomResourceLimits to be a conformance
test and moving e2e parts out.

For knative#6226 fixes it for TestContainerErrorMsg and
TestContainerExitingMsg, though could be other tests checking errors
incorrectly.

Addresses knative#6264 though continuing work is needed.

Don't mix E2E and conformance:
Strip E2E portions of conformance out into tests in the test/e2e
directory. Commonality goes into test/scenarios or test/v1 depending
on specificity of the common functionality.

Rename to more align with runtime test naming (Must/Should).
@knative-housekeeping-robot
Copy link

Stale issues rot after 30 days of inactivity.
Mark the issue as fresh by adding the comment /remove-lifecycle rotten.
Rotten issues close after an additional 30 days of inactivity.
If this issue is safe to close now please do so by adding the comment /close.

Send feedback to Knative Productivity Slack channel or file an issue in knative/test-infra.

/lifecycle rotten

@knative-prow-robot knative-prow-robot added lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. and removed lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. labels Mar 14, 2020
@knative-housekeeping-robot
Copy link

Rotten issues close after 30 days of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh by adding the comment /remove-lifecycle rotten.

Send feedback to Knative Productivity Slack channel or file an issue in knative/test-infra.

/close

@knative-prow-robot
Copy link
Contributor

@knative-housekeeping-robot: Closing this issue.

In response to this:

Rotten issues close after 30 days of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh by adding the comment /remove-lifecycle rotten.

Send feedback to Knative Productivity Slack channel or file an issue in knative/test-infra.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@dprotaso
Copy link
Member

/reopen
/remove-lifecycle rotten

@knative-prow-robot
Copy link
Contributor

@dprotaso: Reopened this issue.

In response to this:

/reopen
/remove-lifecycle rotten

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@knative-prow-robot knative-prow-robot removed the lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. label May 28, 2020
@coryrc
Copy link
Contributor Author

coryrc commented May 28, 2020

There's a fix in #6978 for anyone who wants to break it out.
/unassign
/kind good-first-issue

@nthnca
Copy link
Contributor

nthnca commented Jun 24, 2020

/assign

@nthnca nthnca removed their assignment Aug 25, 2020
@github-actions
Copy link

This issue is stale because it has been open for 90 days with no
activity. It will automatically close after 30 more days of
inactivity. Reopen the issue with /reopen. Mark the issue as
fresh by adding the comment /remove-lifecycle stale.

@github-actions github-actions bot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Nov 24, 2020
@dprotaso
Copy link
Member

/lifecycle frozen
/remove-lifecycle stale

@knative-prow-robot knative-prow-robot added lifecycle/frozen Indicates that an issue or PR should not be auto-closed due to staleness. and removed lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. labels Nov 24, 2020
@dprotaso dprotaso added good first issue Denotes an issue ready for a new contributor, according to the "help wanted" guidelines. and removed kind/good-first-issue labels Jan 19, 2021
@evankanderson
Copy link
Member

/triage accepted

Should this move to https://github.com/knative/specs ?

@knative-prow-robot knative-prow-robot added the triage/accepted Issues which should be fixed (post-triage) label Mar 22, 2021
@coryrc coryrc closed this as completed Mar 15, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
area/test-and-release It flags unit/e2e/conformance/perf test issues for product features good first issue Denotes an issue ready for a new contributor, according to the "help wanted" guidelines. kind/bug Categorizes issue or PR as related to a bug. lifecycle/frozen Indicates that an issue or PR should not be auto-closed due to staleness. triage/accepted Issues which should be fixed (post-triage)
Projects
None yet
Milestone
No milestone
6 participants
@dprotaso @coryrc @evankanderson @nthnca @knative-prow-robot @knative-housekeeping-robot