diff --git a/README.md b/README.md index 9af5badb92..8369bc964a 100644 --- a/README.md +++ b/README.md @@ -17,26 +17,47 @@ See the License for the specific language governing permissions and For nightly and release builds, there are test jobs running as part of the AdoptOpenJDK continuous delivery pipelines. There is a [blog post and brief presentation](https://blog.adoptopenjdk.net/2017/12/testing-java-help-count-ways) that explains what testing we run and how they fit into the overall delivery pipeline. As the world of testing at AdoptOpenJDK is evolving and improving quickly, some documentation may fall behind the march of progress. Please let us know and help us keep it up-to-date, and ask questions at the [AdoptOpenJDK testing Slack channel](https://adoptopenjdk.slack.com/messages/C5219G28G)! -![CI pipeline view](doc/ciPipeline.jpg) +![CI pipeline view](doc/diagrams/ciPipeline.jpg) -See the different test jobs here: +#### Test 'Inventory' -- [openjdk](https://ci.adoptopenjdk.net/view/Test_openjdk/) - OpenJDK regression tests (from OpenJDK) +The directory structure in this openjdk-tests repository is meant to reflect the different types of test we run (and pull from lots of other locations). The diagrams below show the test make target for each of the types, along with in-plan, upcoming additions (denoted by dotted line grey boxes). The provided links jump to test jobs in Jenkins (ci.adoptopenjdk.net). -- [system](https://ci.adoptopenjdk.net/view/Test_system/) - System and load tests (from the AdoptOpenJDK/openjdk-systemtest repo) +![overview of tests](doc/diagrams/overviewOfAdoptTests.svg) -- [external](https://ci.adoptopenjdk.net/view/Test_external/) - 3rd party application tests run in Docker containers (from different application projects such as scala, tomcat, etc) +--- -- [perf](https://ci.adoptopenjdk.net/view/Test_perf/) - Open-source benchmark suites (from different open-source projects such as Acme-Air) +##### [openjdk](https://ci.adoptopenjdk.net/view/Test_openjdk/) tests - OpenJDK regression tests +Tests from OpenJDK -- [jck](https://ci.adoptopenjdk.net/view/Test_jck/) - TCK tests (under the OpenJDK Community TCK License Agreement), in compliance with the license agreement, are run on a private Jenkins server (jobs linked here are merely remote triggers to the private server) +--- +##### [system](https://ci.adoptopenjdk.net/view/Test_system/) tests - System and load tests +Tests from the AdoptOpenJDK/openjdk-systemtest repo -#### Test 'Inventory' +--- + +##### [external](https://ci.adoptopenjdk.net/view/Test_external/) tests - 3rd party application tests +Test suites from a variety of applications, along with microprofile TCKs, run in Docker containers +![external tests](doc/diagrams/externaltests.svg) + +--- + +##### [perf](https://ci.adoptopenjdk.net/view/Test_perf/) tests - Performance benchmark suites +Performance benchmark tests (both full suites and microbenches) from different open-source projects such as Acme-Air and AdoptOpenJDK/bumblebench +![perf tests](doc/diagrams/perftests.svg) + +--- + +##### [functional](https://ci.adoptopenjdk.net/view/Test_functional/) tests - Unit and functional tests +Functional tests not originating from the openjdk regression suite, that include locale/language tests and a subset of implementation agnostic tests from the openj9 project. + +--- -The directory structure in this openjdk-tests repository is meant to reflect the different types of test we run (and pull from lots of other locations). The diagram below shows the test make target for each of the types, along with in-plan, upcoming additions (denoted by dotted line grey boxes). +##### jck tests - Compliance tests +TCK tests (under the OpenJDK Community TCK License Agreement), in compliance with the license agreement, are run on a private Jenkins server -![Overview of Tests](doc/overviewOfAdoptTests.svg) +--- #### Guide to Running the Tests Yourself For more details on how to run the same tests that we run at AdoptOpenJDK on your laptop or in your build farm, please consult our [User Guide](doc/userGuide.md) (work in progress). diff --git a/buildenv/jenkins/openjdk_aarch32_linux b/buildenv/jenkins/openjdk_aarch32_linux new file mode 100644 index 0000000000..35ed7a26a1 --- /dev/null +++ b/buildenv/jenkins/openjdk_aarch32_linux @@ -0,0 +1,20 @@ +#!groovy +LABEL='sw.os.linux&&hw.arch.aarch32' + +node ("master") { + checkout scm + def jenkinsfile = load "${WORKSPACE}/openjdk-tests/buildenv/jenkins/JenkinsfileBase" + jenkinsfile.setLabelParam() + cleanWs() +} + +stage('Queue') { + node("$LABEL") { + PLATFORM = 'aarch32_linux' + SDK_RESOURCE = 'upstream' + SPEC='linux_arm' + checkout scm + def jenkinsfile = load "${WORKSPACE}/openjdk-tests/buildenv/jenkins/JenkinsfileBase" + jenkinsfile.testBuild() + } +} diff --git a/doc/appTestTriageScenarios.png b/doc/diagrams/appTestTriageScenarios.png similarity index 100% rename from doc/appTestTriageScenarios.png rename to doc/diagrams/appTestTriageScenarios.png diff --git a/doc/ciPipeline.jpg b/doc/diagrams/ciPipeline.jpg similarity index 100% rename from doc/ciPipeline.jpg rename to doc/diagrams/ciPipeline.jpg diff --git a/doc/diagrams/externaltests.svg b/doc/diagrams/externaltests.svg new file mode 100644 index 0000000000..7a1eae815b --- /dev/null +++ b/doc/diagrams/externaltests.svg @@ -0,0 +1,350 @@ +
_sanity.external
_extended.external
make _external
thirdparty_containers
derby
elasticsearch
example
jenkins
kafka
lucene-solr
scala
tomcat
wildfly
openliberty
geode
hbase
akka
logstash
openliberty-mp-tck
payara-mp-tck
thorntail-mp-tck
openjdk-tests
\ No newline at end of file diff --git a/doc/jenkinsParameters.jpg b/doc/diagrams/jenkinsParameters.jpg similarity index 100% rename from doc/jenkinsParameters.jpg rename to doc/diagrams/jenkinsParameters.jpg diff --git a/doc/diagrams/overviewOfAdoptTests.svg b/doc/diagrams/overviewOfAdoptTests.svg new file mode 100644 index 0000000000..ba632d714a --- /dev/null +++ b/doc/diagrams/overviewOfAdoptTests.svg @@ -0,0 +1,350 @@ +
_sanity.perf
_extended.perf
_sanity.external
_extended.external
external
perf
openjdk
system
functional
jck
make _openjdk
make _system
make _perf
make _functional
make _jck
make _external
.....
..
...
....
thirdparty_containers
performance
openjdk_regression
systemtest
functional
jck
openjdk-tests
\ No newline at end of file diff --git a/doc/diagrams/perftests.svg b/doc/diagrams/perftests.svg new file mode 100644 index 0000000000..977975459a --- /dev/null +++ b/doc/diagrams/perftests.svg @@ -0,0 +1,350 @@ +
_sanity.perf
_extended.perf
make _perf
performance
bbench
idle_micro
odm
liberty_dt
acme_air
openjdk-tests
\ No newline at end of file diff --git a/doc/pipelineFromSCM.jpg b/doc/diagrams/pipelineFromSCM.jpg similarity index 100% rename from doc/pipelineFromSCM.jpg rename to doc/diagrams/pipelineFromSCM.jpg diff --git a/doc/overviewDiagram.md b/doc/overviewDiagram.md deleted file mode 100644 index ced1c2c05d..0000000000 --- a/doc/overviewDiagram.md +++ /dev/null @@ -1,31 +0,0 @@ -graph LR; - -style elasticsearch fill:#eee,stroke:#cca,stroke-width:2px,stroke-dasharray: 5, 5 -style openliberty fill:#eee,stroke:#cca,stroke-width:2px,stroke-dasharray: 5, 5 -style derby fill:#eee,stroke:#cca,stroke-width:2px,stroke-dasharray: 5, 5 -style solr/lucene fill:#eee,stroke:#cca,stroke-width:2px,stroke-dasharray: 5, 5 -style acmeAir fill:#eee,stroke:#cca,stroke-width:2px,stroke-dasharray: 5, 5 -style libertyDayTrader fill:#eee,stroke:#cca,stroke-width:2px,stroke-dasharray: 5, 5 - -openjdk-tests--make _openjdk-->openjdk_regression; - -openjdk-tests--make _system-->systemtest; - -openjdk-tests--make _perf-->performance; -openjdk-tests--make _functional-->functional; -openjdk-tests--make _jck-->jck; -openjdk-tests--make _external-->thirdparty_containers; - -performance-->idleMicroBenchmark; -performance-->libertyDayTrader; -performance-->acmeAir; - -thirdparty_containers-->scala; -thirdparty_containers-->tomcat; -thirdparty_containers-->jenkins; -thirdparty_containers-->example; -thirdparty_containers-->openliberty; -thirdparty_containers-->elasticsearch; -thirdparty_containers-->derby; -thirdparty_containers-->solr/lucene; - diff --git a/doc/overviewOfAdoptTests.svg b/doc/overviewOfAdoptTests.svg deleted file mode 100644 index fee3d6b088..0000000000 --- a/doc/overviewOfAdoptTests.svg +++ /dev/null @@ -1,350 +0,0 @@ -
make _openjdk
make _system
make _perf
make _functional
make _jck
make _external
elasticsearch
openliberty
derby
solr/lucene
acmeAir
libertyDayTrader
openjdk-tests
openjdk_regression
systemtest
performance
functional
jck
thirdparty_containers
idleMicroBenchmark
scala
tomcat
jenkins
example
\ No newline at end of file diff --git a/doc/userGuide.md b/doc/userGuide.md index 3d77812a78..61018ff9e7 100644 --- a/doc/userGuide.md +++ b/doc/userGuide.md @@ -23,7 +23,7 @@ You can set up your own Jenkins-based test builds using the AdoptOpenJDK openjdk - Repository url - :https://github.com/AdoptOpenJDK/openjdk-tests.git - Branches to build - */master - Script path - buildenv/jenkins/fileToMatchVersionAndPlatformToTest, example openjdk8_x86-64_linux -![pipeline from SCM](/doc/pipelineFromSCM.jpg) +![pipeline from SCM](/doc/diagrams/pipelineFromSCM.jpg) 2. Create necessary parameters @@ -31,7 +31,7 @@ You can set up your own Jenkins-based test builds using the AdoptOpenJDK openjdk * JVM_VERSION - depending on what SDK you are testing against (some possible values are: openjdk8, openjdk8-openj9, openjdk9, openjdk9-openj9, openjdk10, openjdk10-openj9, openjdk10-sap) * CUSTOMIZED_SDK_URL - the URL for where to pick up the SDK to test (if you are picking up builds from AdoptOpenJDK, please refer to the [openjdk-api README](https://github.com/AdoptOpenJDK/openjdk-api/blob/master/README.md) for more details) -![jenkins parameters](/doc/jenkinsParameters.jpg) +![jenkins parameters](/doc/diagrams/jenkinsParameters.jpg) ## Local testing via make targets on the commandline diff --git a/thirdparty_containers/README.md b/thirdparty_containers/README.md index 999b9dfecd..8bf097f582 100644 --- a/thirdparty_containers/README.md +++ b/thirdparty_containers/README.md @@ -23,7 +23,7 @@ Our next steps to improve and expand this set of external tests is divided into ### Triage Rules There are 4 common triage scenarios, with associated appropriate actions to take: -![3rd Party App Test Triage Scenarios](../doc/appTestTriageScenarios.png) +![3rd Party App Test Triage Scenarios](../doc/diagrams/appTestTriageScenarios.png) ### How to Add New Tests - Learn how to run the application tests that you intend to automate in the build manually first, and find out any special dependencies the application testing may have.