Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

V2 Unit Testing - Implementation and Client Testing #246

Open
wants to merge 82 commits into
base: master
Choose a base branch
from
Open

V2 Unit Testing - Implementation and Client Testing #246

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
82 commits
Select commit Hold shift + click to select a range
cc89064
feat: Add V2 clients
Mar 23, 2023
0f5989d
fix: clean up + switching to webclient exception handling
Mar 23, 2023
efe65c6
fix: better exception handling
Mar 24, 2023
66d2ca4
feat: [SRE-26521] upload/download for scripts/datafiles
Mar 20, 2023
2fe9b7e
Revert "feat: [SRE-26521] upload/download for scripts/datafiles"
Mar 27, 2023
630a5cb
Merge branch 'master' into zkofiro/v2-clients
Mar 27, 2023
96c58f8
feat: Updated clients, new routes, status response from agent/job ops…
Mar 28, 2023
c550a9e
fix: listing only user-facing endpoints
Mar 28, 2023
ae9923d
fix: small updates to deps + docs for client changes
Mar 28, 2023
2e70ebb
fix: temp fix for tools while testing agent
Mar 28, 2023
5c96250
fix: removing unneeded files
Mar 28, 2023
567757d
Merge branch 'master' into zkofiro/v2-clients
Apr 7, 2023
bb9e9eb
fix: temp fix for client tools
Apr 7, 2023
24c8ba5
fix: missing file
Apr 7, 2023
b84720f
moved last backend V1 deps over with models + tested locally
Apr 13, 2023
b2a0e7a
updating file documentation
Apr 13, 2023
5167739
reverting debug flag
Apr 13, 2023
d98d611
moving other misc backend changes to this branch
Apr 24, 2023
6259919
moving over one more backend change
Apr 24, 2023
7ac6a61
updated in UI ref branch
May 3, 2023
a096470
Merge branch 'zkofiro/move-remaining-V1' into zkofiro/SRE-26707
May 3, 2023
42d5651
Merge branch 'master' into zkofiro/v2-clients
Jun 5, 2023
99c5754
Merge branch 'master' into zkofiro/SRE-26707
Jun 5, 2023
e29c166
Merge branch 'zkofiro/SRE-26707' into zkofiro/v2-clients
Jun 6, 2023
997ebde
cleaning up, removing old files
Jun 6, 2023
cf8a544
testing base agent client
Jun 6, 2023
875e8f6
small clean up
Jun 6, 2023
9287885
dep injection fix
Jun 7, 2023
59ba285
need to shorten V2 url due to dep inj fix
Jun 7, 2023
f587553
missing dependency
Jun 8, 2023
b3e268d
unneeded files
Jun 8, 2023
e072422
reverting cloud models until move away from Java EE CDI
Jun 8, 2023
6cdbd45
updating vmtracker as well
Jun 8, 2023
c3d5b4d
TankAPI fix
Jun 8, 2023
c0531a0
moving vmmanger + models outside of rest api
Jun 9, 2023
101e509
update pom for logging
Jun 9, 2023
abb65a0
vmTracker models moved to api
Jun 9, 2023
b44a5c5
updating import ref to cloud
Jun 9, 2023
c1247bc
reverting defaultcontroller
Jun 9, 2023
816d1c5
revert ui for now
Jun 9, 2023
3a7b524
reverting module deletion, saving for V1 removal
Jun 9, 2023
834f218
reverting local files
Jun 9, 2023
d089dce
updating package names
Jun 9, 2023
ea8d78a
update UI refs to /v2 path
Jun 9, 2023
ee46bcc
banner fixed to stay closed each session
Jun 11, 2023
10bc49e
updated tools for changes
Jun 12, 2023
10ff8ab
banner logic for SSOlogin
Jun 12, 2023
ec0bae4
exclude to reenable agent logging
Jun 12, 2023
66b281b
commenting back httpclient tests
Jun 12, 2023
69b9c5b
Merge branch 'master' into zkofiro/v2-clients
Jun 12, 2023
5985ffa
should fix agent logging
Jun 12, 2023
af5f468
reverting files
Jun 12, 2023
5173403
adding to child pom
Jun 12, 2023
a12061e
logback vs log4j2 mismatch - probable cause for log drop between modules
Jun 13, 2023
5932a36
attempt to force spring to use log4j
Jun 13, 2023
f63c882
correct file path for log4j2
Jun 13, 2023
7f031e2
revert after testing logging fix
Jun 13, 2023
d2ae166
legacy spring startup, removing from web/WEB-INF
Jun 13, 2023
a57438c
re-adding bridge deps
Jun 13, 2023
37ff905
fixed versions
Jun 13, 2023
3778440
logging config issue fix: including maven-shade-plugin fixes issue lo…
Jun 13, 2023
094ef10
updating gitignore
Jun 13, 2023
30d448b
logs showing locally, just needed to exclude extra servlet fragment
Jun 14, 2023
95f4295
switching scope out for absolute-ordering, logging to file locally
Jun 14, 2023
99c4239
moving from tank-api to agent module
Jun 14, 2023
e358551
exclude Log4j2plugins.dat w/ maven shaded plugin instead, avoiding de…
Jun 14, 2023
72bb839
updating fix for other modules, tested locally
Jun 14, 2023
5b9ec82
uncommenting tests now that builds passing
Jun 14, 2023
932b170
updated not needed, pending deletion
Jun 15, 2023
6db2e74
filter endpoint fix - moving script models out to api module
Jun 28, 2023
f627fec
Revert "filter endpoint fix - moving script models out to api module"
Jun 28, 2023
0a2cc49
new filter fix: reverting all script models to v1 until external scri…
Jun 28, 2023
5779312
reduced war file size by 80M, still needs more dep exclusions from sp…
Jun 29, 2023
4a2ca07
updated unit testing
Jun 29, 2023
e8eb83e
reverting
Jun 29, 2023
092df98
Merge branch 'master' into zkofiro/update-v2
Jun 29, 2023
eedcbe0
apply filters fix
Jun 30, 2023
0769640
increase file upload size in properties file
Jun 30, 2023
f0f2114
reverting since validated in QA
Jul 6, 2023
182f72e
Merge branch 'master' into zkofiro/update-v2
Dec 1, 2023
9414a43
100% client code cov, can improve impl tests
Dec 5, 2023
5c46723
no longer needed
Dec 6, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 24 additions & 0 deletions pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -801,6 +801,30 @@
<version>${version.wiremock}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.squareup.okhttp3</groupId>
<artifactId>mockwebserver</artifactId>
<version>4.10.0</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>io.projectreactor</groupId>
<artifactId>reactor-test</artifactId>
<version>3.6.0</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-test</artifactId>
<version>5.3.29</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<version>${version.spring-webflux}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-annotations</artifactId>
Expand Down
25 changes: 25 additions & 0 deletions rest-mvc/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,31 @@
<artifactId>tank-script-processor</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>com.squareup.okhttp3</groupId>
<artifactId>mockwebserver</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.wiremock</groupId>
<artifactId>wiremock</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>io.projectreactor</groupId>
<artifactId>reactor-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
</dependencies>


Expand Down
28 changes: 14 additions & 14 deletions rest-mvc/src/main/java/com/intuit/tank/rest/mvc/rest/clients/AgentClient.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,6 @@
import org.springframework.http.MediaType;
import reactor.core.publisher.Mono;

import java.util.Map;

public class AgentClient extends BaseClient{

private static final String SERVICE_BASE_URL = "/v2/agent";
Expand All @@ -44,7 +42,7 @@ public String getSettings() {
.uri(urlBuilder.buildUrl("/settings"))
.accept(MediaType.APPLICATION_XML)
.retrieve()
.onStatus(status -> status.isError(),
.onStatus(HttpStatus::isError,
response -> response.bodyToMono(String.class)
.flatMap(body -> Mono.error(new ClientException(body,
response.statusCode().value()))))
Expand All @@ -57,7 +55,7 @@ public Mono<DataBuffer> getSupportFiles() {
.uri(urlBuilder.buildUrl("/support-files"))
.accept(MediaType.APPLICATION_OCTET_STREAM)
.retrieve()
.onStatus(status -> status.isError(),
.onStatus(HttpStatus::isError,
response -> response.bodyToMono(String.class)
.flatMap(body -> Mono.error(new ClientException(body,
response.statusCode().value()))))
Expand All @@ -71,7 +69,7 @@ public AgentTestStartData agentReady(AgentData agentData) {
.accept(MediaType.APPLICATION_JSON)
.body(Mono.just(agentData), AgentData.class)
.retrieve()
.onStatus(status -> status.isError(),
.onStatus(HttpStatus::isError,
response -> response.bodyToMono(String.class)
.flatMap(body -> Mono.error(new ClientException(body,
response.statusCode().value()))))
Expand All @@ -84,7 +82,7 @@ public Headers getHeaders() {
.uri(urlBuilder.buildUrl("/headers"))
.accept(MediaType.APPLICATION_XML)
.retrieve()
.onStatus(status -> status.isError(),
.onStatus(HttpStatus::isError,
response -> response.bodyToMono(String.class)
.flatMap(body -> Mono.error(new ClientException(body,
response.statusCode().value()))))
Expand All @@ -97,7 +95,7 @@ public TankHttpClientDefinitionContainer getClients() {
.uri(urlBuilder.buildUrl("/clients"))
.accept(MediaType.APPLICATION_XML)
.retrieve()
.onStatus(status -> status.isError(),
.onStatus(HttpStatus::isError,
response -> response.bodyToMono(String.class)
.flatMap(body -> Mono.error(new ClientException(body,
response.statusCode().value()))))
Expand All @@ -111,7 +109,7 @@ public Void setStandaloneAgentAvailability(AgentAvailability availability) {
.contentType(MediaType.APPLICATION_JSON)
.body(Mono.just(availability), AgentAvailability.class)
.retrieve()
.onStatus(status -> status.isError(),
.onStatus(HttpStatus::isError,
response -> response.bodyToMono(String.class)
.flatMap(body -> Mono.error(new ClientException(body,
response.statusCode().value()))))
Expand All @@ -124,7 +122,7 @@ public CloudVmStatus getInstanceStatus(String instanceId) {
.uri(urlBuilder.buildUrl("/instance/status/", instanceId))
.accept(MediaType.APPLICATION_JSON)
.retrieve()
.onStatus(status -> status.isError(),
.onStatus(HttpStatus::isError,
response -> response.bodyToMono(String.class)
.flatMap(body -> Mono.error(new ClientException(body,
response.statusCode().value()))))
Expand All @@ -138,20 +136,22 @@ public Void setInstanceStatus(String instanceId, CloudVmStatus VmStatus) {
.contentType(MediaType.APPLICATION_JSON)
.body(Mono.just(VmStatus), CloudVmStatus.class)
.retrieve()
.onStatus(status -> status.isError(),
.onStatus(HttpStatus::isError,
response -> response.bodyToMono(String.class)
.flatMap(body -> Mono.error(new ClientException(body,
response.statusCode().value()))))
.bodyToMono(Void.class)
.block();
}

// Instance Actions

public String stopInstance(String instanceId) {
return client.get()
.uri(urlBuilder.buildUrl("/instance/stop/", instanceId))
.accept(MediaType.TEXT_PLAIN)
.retrieve()
.onStatus(status -> status.isError(),
.onStatus(HttpStatus::isError,
response -> response.bodyToMono(String.class)
.flatMap(body -> Mono.error(new ClientException(body,
response.statusCode().value()))))
Expand All @@ -164,7 +164,7 @@ public String pauseInstance(String instanceId) {
.uri(urlBuilder.buildUrl("/instance/pause/", instanceId))
.accept(MediaType.TEXT_PLAIN)
.retrieve()
.onStatus(status -> status.isError(),
.onStatus(HttpStatus::isError,
response -> response.bodyToMono(String.class)
.flatMap(body -> Mono.error(new ClientException(body,
response.statusCode().value()))))
Expand All @@ -177,7 +177,7 @@ public String resumeInstance(String instanceId) {
.uri(urlBuilder.buildUrl("/instance/resume/", instanceId))
.accept(MediaType.TEXT_PLAIN)
.retrieve()
.onStatus(status -> status.isError(),
.onStatus(HttpStatus::isError,
response -> response.bodyToMono(String.class)
.flatMap(body -> Mono.error(new ClientException(body,
response.statusCode().value()))))
Expand All @@ -190,7 +190,7 @@ public String killInstance(String instanceId) {
.uri(urlBuilder.buildUrl("/instance/kill/", instanceId))
.accept(MediaType.TEXT_PLAIN)
.retrieve()
.onStatus(status -> status.isError(),
.onStatus(HttpStatus::isError,
response -> response.bodyToMono(String.class)
.flatMap(body -> Mono.error(new ClientException(body,
response.statusCode().value()))))
Expand Down
2 changes: 1 addition & 1 deletion rest-mvc/src/main/java/com/intuit/tank/rest/mvc/rest/clients/BaseClient.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,7 @@ public String ping() {
.uri(urlBuilder.buildUrl("/ping"))
.accept(MediaType.TEXT_PLAIN)
.retrieve()
.onStatus(status -> status.isError(),
.onStatus(HttpStatus::isError,
response -> response.bodyToMono(String.class)
.flatMap(body -> Mono.error(new ClientException(body,
response.statusCode().value()))))
Expand Down
35 changes: 21 additions & 14 deletions rest-mvc/src/main/java/com/intuit/tank/rest/mvc/rest/clients/DataFileClient.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,12 @@
import org.springframework.web.reactive.function.client.WebClient;
import org.springframework.web.multipart.MultipartFile;
import org.springframework.core.io.buffer.DataBuffer;
import org.springframework.web.reactive.function.BodyInserters;
import org.springframework.http.ResponseEntity;
import org.springframework.core.ParameterizedTypeReference;
import org.springframework.http.HttpStatus;
import org.springframework.util.MultiValueMap;
import org.springframework.util.LinkedMultiValueMap;
import org.springframework.http.MediaType;
import reactor.core.publisher.Flux;
import reactor.core.publisher.Mono;
Expand Down Expand Up @@ -43,7 +49,7 @@ public DataFileDescriptorContainer getDatafiles() {
.uri(baseUrl)
.accept(MediaType.APPLICATION_JSON)
.retrieve()
.onStatus(status -> status.isError(),
.onStatus(HttpStatus::isError,
response -> response.bodyToMono(String.class)
.flatMap(body -> Mono.error(new ClientException(body,
response.statusCode().value()))))
Expand All @@ -56,7 +62,7 @@ public DataFileDescriptor getDatafile(Integer datafileId) {
.uri(urlBuilder.buildUrl("", datafileId))
.accept(MediaType.APPLICATION_JSON)
.retrieve()
.onStatus(status -> status.isError(),
.onStatus(HttpStatus::isError,
response -> response.bodyToMono(String.class)
.flatMap(body -> Mono.error(new ClientException(body,
response.statusCode().value()))))
Expand All @@ -71,7 +77,7 @@ public String getDatafileContent(Integer datafileId) {
.queryParam("id", datafileId.toString())
.build())
.retrieve()
.onStatus(status -> status.isError(),
.onStatus(HttpStatus::isError,
response -> response.bodyToMono(String.class)
.flatMap(body -> Mono.error(new ClientException(body,
response.statusCode().value()))))
Expand All @@ -90,37 +96,38 @@ public Mono<DataBuffer> downloadDatafile(Integer datafileId) {
.uri(urlBuilder.buildUrl("/download", datafileId))
.accept(MediaType.APPLICATION_OCTET_STREAM)
.retrieve()
.onStatus(status -> status.isError(),
.onStatus(HttpStatus::isError,
response -> response.bodyToMono(String.class)
.flatMap(body -> Mono.error(new ClientException(body,
response.statusCode().value()))))
.bodyToMono(DataBuffer.class);
}

public Map<String, String> uploadDatafile(Integer id, MultipartFile file) {
return client.post()
.uri(uriBuilder -> uriBuilder
.path(urlBuilder.buildUrl("/upload"))
public Mono<ResponseEntity<Map<String, String>>> uploadDatafile(Integer id, MultipartFile file) {
MultiValueMap<String, Object> formData = new LinkedMultiValueMap<>();
formData.add("file", file.getResource());

return WebClient.create(urlBuilder.buildUrl(""))
.post()
.uri(uriBuilder -> uriBuilder.path("/upload")
.queryParam("id", id)
.build())
.contentType(MediaType.MULTIPART_FORM_DATA)
.accept(MediaType.APPLICATION_JSON)
.body(Mono.just(file), MultipartFile.class)
.body(BodyInserters.fromMultipartData(formData))
.retrieve()
.onStatus(status -> status.isError(),
.onStatus(HttpStatus::isError,
response -> response.bodyToMono(String.class)
.flatMap(body -> Mono.error(new ClientException(body,
response.statusCode().value()))))
.bodyToMono(Map.class)
.block();
.toEntity(new ParameterizedTypeReference<>() {});
}

public String deleteDatafile(Integer datafileID) {
return client.delete()
.uri(urlBuilder.buildUrl("", datafileID))
.accept(MediaType.TEXT_PLAIN)
.retrieve()
.onStatus(status -> status.isError(),
.onStatus(HttpStatus::isError,
response -> response.bodyToMono(String.class)
.flatMap(body -> Mono.error(new ClientException(body,
response.statusCode().value()))))
Expand Down
15 changes: 8 additions & 7 deletions rest-mvc/src/main/java/com/intuit/tank/rest/mvc/rest/clients/FilterClient.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@

import com.intuit.tank.rest.mvc.rest.clients.util.ClientException;
import com.intuit.tank.rest.mvc.rest.models.filters.*;
import org.springframework.http.HttpStatus;
import org.springframework.http.MediaType;
import reactor.core.publisher.Mono;

Expand Down Expand Up @@ -36,7 +37,7 @@ public FilterContainer getFilters() {
.uri(baseUrl)
.accept(MediaType.APPLICATION_JSON)
.retrieve()
.onStatus(status -> status.isError(),
.onStatus(HttpStatus::isError,
response -> response.bodyToMono(String.class)
.flatMap(body -> Mono.error(new ClientException(body,
response.statusCode().value()))))
Expand All @@ -49,7 +50,7 @@ public FilterGroupContainer getFilterGroups() {
.uri(urlBuilder.buildUrl("/groups"))
.accept(MediaType.APPLICATION_JSON)
.retrieve()
.onStatus(status -> status.isError(),
.onStatus(HttpStatus::isError,
response -> response.bodyToMono(String.class)
.flatMap(body -> Mono.error(new ClientException(body,
response.statusCode().value()))))
Expand All @@ -62,7 +63,7 @@ public FilterTO getFilter(Integer filterId) {
.uri(urlBuilder.buildUrl("", filterId))
.accept(MediaType.APPLICATION_JSON)
.retrieve()
.onStatus(status -> status.isError(),
.onStatus(HttpStatus::isError,
response -> response.bodyToMono(String.class)
.flatMap(body -> Mono.error(new ClientException(body,
response.statusCode().value()))))
Expand All @@ -75,7 +76,7 @@ public FilterGroupTO getFilterGroup(Integer filterGroupId) {
.uri(urlBuilder.buildUrl("", filterGroupId))
.accept(MediaType.APPLICATION_JSON)
.retrieve()
.onStatus(status -> status.isError(),
.onStatus(HttpStatus::isError,
response -> response.bodyToMono(String.class)
.flatMap(body -> Mono.error(new ClientException(body,
response.statusCode().value()))))
Expand All @@ -90,7 +91,7 @@ public String applyFilters(Integer scriptId, ApplyFiltersRequest request) {
.accept(MediaType.TEXT_PLAIN)
.body(Mono.just(request), ApplyFiltersRequest.class)
.retrieve()
.onStatus(status -> status.isError(),
.onStatus(HttpStatus::isError,
response -> response.bodyToMono(String.class)
.flatMap(body -> Mono.error(new ClientException(body,
response.statusCode().value()))))
Expand All @@ -103,7 +104,7 @@ public String deleteFilter(Integer filterId) {
.uri(urlBuilder.buildUrl("", filterId))
.accept(MediaType.TEXT_PLAIN)
.retrieve()
.onStatus(status -> status.isError(),
.onStatus(HttpStatus::isError,
response -> response.bodyToMono(String.class)
.flatMap(body -> Mono.error(new ClientException(body,
response.statusCode().value()))))
Expand All @@ -116,7 +117,7 @@ public String deleteFilterGroup(Integer filterGroupId) {
.uri(urlBuilder.buildUrl("", filterGroupId))
.accept(MediaType.TEXT_PLAIN)
.retrieve()
.onStatus(status -> status.isError(),
.onStatus(HttpStatus::isError,
response -> response.bodyToMono(String.class)
.flatMap(body -> Mono.error(new ClientException(body,
response.statusCode().value()))))
Expand Down
Loading