Skip to content

Connect to dynamic simulation API with full parameters #37

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
57 commits
Select commit Hold shift + click to select a range
5c34298
Writing test with full parameters for service
thangqp Nov 29, 2022
6c867ae
Config dynawaltz in test environment
thangqp Nov 30, 2022
35b9c7a
Correct style
thangqp Nov 30, 2022
bde07d2
writing test and evolve worker service
thangqp Dec 1, 2022
fcb3e52
load dynamically *.par files into config directory
thangqp Dec 2, 2022
8544af0
more dynamically *.par files in filelist
thangqp Dec 5, 2022
4f77201
Provide TimeSeriesService client
thangqp Dec 5, 2022
9a60a2b
- send time series and time line to time-series-server
thangqp Dec 6, 2022
84bee70
Remove unused imports
thangqp Dec 6, 2022
56ac185
Test CI non-blocking thread
thangqp Dec 6, 2022
58d35d2
Provide DynamicMappingService client, ParametersService client and No…
thangqp Dec 7, 2022
1c607ae
Correct tests
thangqp Dec 8, 2022
7ecc52b
Correct TimeSeriesService
thangqp Dec 8, 2022
0bb81e5
ParameterService now take the current default config file system to p…
thangqp Dec 8, 2022
ea64d49
Correct checkstyle
thangqp Dec 8, 2022
c462ada
code clean on test
thangqp Dec 9, 2022
1ca560d
Extract into interfaces
thangqp Dec 9, 2022
33d6035
Restructure packages and add licences
thangqp Dec 9, 2022
7f769f8
Merge branch 'main' into branching_to_dynamic_simulation_API_with_ful…
thangqp Dec 9, 2022
36567fe
Try a single unit test for detect block() problem
thangqp Dec 9, 2022
29d805b
Try to subscribe() instead of block()
thangqp Dec 9, 2022
fc6b904
Check whether dynamo shell exists at the beginning of test
thangqp Dec 12, 2022
a1cd783
Check whether dynamo shell exists at the end of setup
thangqp Dec 12, 2022
59cdac8
Check whether dynamo directory and shell exists at the beginning of test
thangqp Dec 12, 2022
6acb303
Unzip dynawo zip file into the parent of current directory
thangqp Dec 12, 2022
6bfccce
Enhance code coverage test
thangqp Dec 12, 2022
2adf501
Enhance code coverage
thangqp Dec 12, 2022
e2b29c0
Check with/clean : false when checkout source
thangqp Dec 12, 2022
59f2e73
Indicate working directory as dynawo
thangqp Dec 12, 2022
1d9c7ec
Test checkoutv2 with clean = false
thangqp Dec 12, 2022
c7b425f
Restore code with unzip dynawo into parent WORKSPACE dir
thangqp Dec 12, 2022
6461108
remove ls
thangqp Dec 12, 2022
59d3f33
increase timeout
thangqp Dec 12, 2022
1897a59
Clean, reorganize packages, decouple tests into Service vs Rest layer…
thangqp Dec 13, 2022
426f8ed
Merge branch 'main' into branching_to_dynamic_simulation_API_with_ful…
thangqp Dec 13, 2022
285f29c
Merge PR https://github.com/gridsuite/dynamic-simulation-server/pull/38
thangqp Dec 13, 2022
4b7cc11
Sonar : A "NullPointerException" could be thrown; "scriptObj" is null…
thangqp Dec 13, 2022
81e53f3
Provide a merge util to merge dynamic parameter file with event param…
thangqp Dec 14, 2022
d2d91b1
Some updates in order to align to timeseries-server REST ENDPOINT
thangqp Dec 15, 2022
0b52864
Update port timesseries-server
thangqp Dec 15, 2022
81f23a6
Remove body on endpoint POST ""/networks/{networkUuid}/run"
thangqp Dec 16, 2022
13a9940
1/ Using toFuture().get() instead of subcribe() for consumer in order…
thangqp Dec 19, 2022
15ab5cc
Merge remote-tracking branch 'origin/main' into branching_to_dynamic_…
thangqp Jan 9, 2023
20dc599
update path baseUri after merging PR : Homogenize keys backing servic…
thangqp Jan 9, 2023
5b4ab3b
renaming client services
thangqp Jan 11, 2023
c06e198
Revisite code following the comment by reviewer
thangqp Jan 13, 2023
89b02dc
Add more comments on using toFuture().get() instead of block()
thangqp Jan 13, 2023
99d33f0
1 - Change type for result column in the second changeset
thangqp Jan 16, 2023
b06b615
Correct checkstyles
thangqp Jan 16, 2023
d889815
Notification for a receiver
thangqp Jan 16, 2023
7bc3083
Clean DynamicSimulationResultSerializer.java
thangqp Jan 17, 2023
e4092da
Add TimeSeriesGroupInfos following to timeseries-server DTO
thangqp Jan 17, 2023
92caf98
Remove unnecessary parameters in config.yml and parameters.json
thangqp Jan 23, 2023
f9ab2c0
1 - Switch using toFuture().get() to block() in consumRun()
thangqp Jan 24, 2023
886ca6f
Correct following to comments of reviewers
thangqp Jan 26, 2023
2de1581
Change return type of endpoint getStatus() to DynamicSimulationStatus
thangqp Jan 29, 2023
3143c41
Close opened output stream
thangqp Jan 30, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 15 additions & 4 deletions pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,8 @@

<dependencyManagement>
<dependencies>
<!-- overrides of imports -->

<!-- imports -->
<dependency>
<groupId>org.gridsuite</groupId>
Expand All @@ -98,13 +100,16 @@
<type>pom</type>
<scope>import</scope>
</dependency>

<!-- project specific dependencies -->
</dependencies>
</dependencyManagement>

<dependencies>
<!-- Compilation dependencies -->
<dependency>
<groupId>com.powsybl</groupId>
<artifactId>powsybl-network-store-iidm-impl</artifactId>
<artifactId>powsybl-iidm-api</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
Expand Down Expand Up @@ -132,15 +137,15 @@
</dependency>
<dependency>
<groupId>com.powsybl</groupId>
<artifactId>powsybl-dynawaltz</artifactId>
<artifactId>powsybl-dynamic-simulation-dsl</artifactId>
</dependency>
<dependency>
<groupId>com.powsybl</groupId>
<artifactId>powsybl-dynawaltz-dsl</artifactId>
<artifactId>powsybl-dynawaltz</artifactId>
</dependency>
<dependency>
<groupId>com.powsybl</groupId>
<artifactId>powsybl-dynamic-simulation-dsl</artifactId>
<artifactId>powsybl-dynawaltz-dsl</artifactId>
</dependency>
<dependency>
<groupId>com.powsybl</groupId>
Expand Down Expand Up @@ -189,6 +194,12 @@
<version>${h2database.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.powsybl</groupId>
<artifactId>powsybl-commons</artifactId>
<type>test-jar</type>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.powsybl</groupId>
<artifactId>powsybl-config-test</artifactId>
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
/**
* Copyright (c) 2020, RTE (http://www.rte-france.com)
* This Source Code Form is subject to the terms of the Mozilla Public
* License, v. 2.0. If a copy of the MPL was not distributed with this
* file, You can obtain one at http://mozilla.org/MPL/2.0/.
*/

package com.powsybl.dynamicsimulation.groovy;

import com.powsybl.dsl.ExpressionDslLoader;
import com.powsybl.dsl.GroovyScripts;
import com.powsybl.dynamicsimulation.Curve;
import com.powsybl.dynamicsimulation.CurvesSupplier;
import com.powsybl.iidm.network.Network;
import groovy.lang.Binding;
import groovy.lang.GroovyCodeSource;
import groovy.lang.GroovyShell;
import org.codehaus.groovy.control.CompilerConfiguration;

import java.io.InputStream;
import java.util.ArrayList;
import java.util.List;
import java.util.Objects;

/**
* TODO: remove and use core one when switching to 5.1.0
* @author Mathieu Bague <[email protected]>
*/
public class GroovyCurvesSupplier implements CurvesSupplier {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To remove and use core one when switching to 5.1.0

Copy link
Contributor Author

@thangqp thangqp Jan 26, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Noted by add comment into code


private final GroovyCodeSource codeSource;

private final List<CurveGroovyExtension> extensions;

public GroovyCurvesSupplier(InputStream is, List<CurveGroovyExtension> extensions) {
this.codeSource = GroovyScripts.load(is);
this.extensions = Objects.requireNonNull(extensions);
}

@Override
public String getName() {
return null;
}

@Override
public List<Curve> get(Network network) {
List<Curve> curves = new ArrayList<>();

Binding binding = new Binding();
binding.setVariable("network", network);

ExpressionDslLoader.prepareClosures(binding);
extensions.forEach(e -> e.load(binding, curves::add));

GroovyShell shell = new GroovyShell(binding, new CompilerConfiguration());
shell.evaluate(codeSource);

return curves;
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
/**
* Copyright (c) 2020, RTE (http://www.rte-france.com)
* This Source Code Form is subject to the terms of the Mozilla Public
* License, v. 2.0. If a copy of the MPL was not distributed with this
* file, You can obtain one at http://mozilla.org/MPL/2.0/.
*/

package com.powsybl.dynamicsimulation.groovy;

import com.powsybl.dsl.ExpressionDslLoader;
import com.powsybl.dsl.GroovyScripts;
import com.powsybl.dynamicsimulation.EventModel;
import com.powsybl.dynamicsimulation.EventModelsSupplier;
import com.powsybl.iidm.network.Network;
import groovy.lang.Binding;
import groovy.lang.GroovyCodeSource;
import groovy.lang.GroovyShell;
import org.codehaus.groovy.control.CompilerConfiguration;

import java.io.InputStream;
import java.util.ArrayList;
import java.util.List;
import java.util.Objects;

/**
* TODO: remove and use core one when switching to 5.1.0
* @author Marcos de Miguel <demiguelm at aia.es>
*/
public class GroovyEventModelsSupplier implements EventModelsSupplier {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To remove and use core one when switching to 5.1.0

Copy link
Contributor Author

@thangqp thangqp Jan 26, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Noted by add comment into code


private final GroovyCodeSource codeSource;

private final List<EventModelGroovyExtension> extensions;

public GroovyEventModelsSupplier(InputStream is, List<EventModelGroovyExtension> extensions) {
this.codeSource = GroovyScripts.load(is);
this.extensions = Objects.requireNonNull(extensions);
}

@Override
public String getName() {
return null;
}

@Override
public List<EventModel> get(Network network) {
List<EventModel> eventModels = new ArrayList<>();

Binding binding = new Binding();
binding.setVariable("network", network);

ExpressionDslLoader.prepareClosures(binding);
extensions.forEach(e -> e.load(binding, eventModels::add));

GroovyShell shell = new GroovyShell(binding, new CompilerConfiguration());
shell.evaluate(codeSource);

return eventModels;
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,10 @@
* @author Abdelsalem Hedhili <abdelsalem.hedhili at rte-france.com>
*/

final class DynamicSimulationApi {
public final class DynamicSimulationApi {

private DynamicSimulationApi() {
}

static final String API_VERSION = "v1";
public static final String API_VERSION = "v1";
}
Original file line number Diff line number Diff line change
@@ -1,21 +1,22 @@
/**
* Copyright (c) 2021, RTE (http://www.rte-france.com)
/*
* Copyright (c) 2021-2022, RTE (http://www.rte-france.com)
* This Source Code Form is subject to the terms of the Mozilla Public
* License, v. 2.0. If a copy of the MPL was not distributed with this
* file, You can obtain one at http://mozilla.org/MPL/2.0/.
*/
package org.gridsuite.ds.server;
package org.gridsuite.ds.server.config;

import io.swagger.v3.oas.models.OpenAPI;
import io.swagger.v3.oas.models.info.Info;
import org.gridsuite.ds.server.DynamicSimulationApi;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

/**
* @author Abdelsalem Hedhili <abdelsalem.hedhili at rte-france.com>
*/
@Configuration
public class DynamicSimulationSwaggerConfig {
public class SwaggerConfig {

@Bean
public OpenAPI createOpenApi() {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,30 +4,31 @@
* License, v. 2.0. If a copy of the MPL was not distributed with this
* file, You can obtain one at http://mozilla.org/MPL/2.0/.
*/
package org.gridsuite.ds.server;
package org.gridsuite.ds.server.controller;

import io.swagger.v3.oas.annotations.Operation;
import io.swagger.v3.oas.annotations.Parameter;
import io.swagger.v3.oas.annotations.responses.ApiResponse;
import io.swagger.v3.oas.annotations.responses.ApiResponses;
import io.swagger.v3.oas.annotations.tags.Tag;
import org.gridsuite.ds.server.dto.DynamicSimulationStatus;
import org.gridsuite.ds.server.service.DynamicSimulationService;
import org.springframework.boot.context.properties.bind.DefaultValue;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.http.codec.multipart.FilePart;
import org.springframework.web.bind.annotation.*;
import reactor.core.publisher.Mono;

import java.util.UUID;

import static org.springframework.http.MediaType.*;
import static org.gridsuite.ds.server.DynamicSimulationApi.API_VERSION;
import static org.springframework.http.MediaType.APPLICATION_JSON_VALUE;

/**
* @author Abdelsalem Hedhili <abdelsalem.hedhili at rte-france.com>
*/
@RestController
@RequestMapping(value = "/" + DynamicSimulationApi.API_VERSION)
@RequestMapping(value = "/" + API_VERSION)
@Tag(name = "Dynamic simulation server")
public class DynamicSimulationController {

Expand All @@ -44,17 +45,28 @@ public ResponseEntity<Mono<UUID>> run(@PathVariable("networkUuid") UUID networkU
@RequestParam(name = "variantId", required = false) String variantId,
@DefaultValue("0") @RequestParam("startTime") int startTime,
@RequestParam("stopTime") int stopTime,
@RequestPart("dynamicModel") FilePart dynamicModel) {
Mono<UUID> resultUuid = dynamicSimulationService.runAndSaveResult(networkUuid, variantId, startTime, stopTime, dynamicModel);
@RequestParam("mappingName") String mappingName,
@RequestParam(name = "receiver", required = false) String receiver) {
Mono<UUID> resultUuid = dynamicSimulationService.runAndSaveResult(receiver, networkUuid, variantId, startTime, stopTime, mappingName);
return ResponseEntity.ok().contentType(MediaType.APPLICATION_JSON).body(resultUuid);
}

@GetMapping(value = "/results/{resultUuid}", produces = "application/json")
@GetMapping(value = "/results/{resultUuid}/timeseries", produces = "application/json")
@Operation(summary = "Get a dynamic simulation result from the database")
@ApiResponses(value = {@ApiResponse(responseCode = "200", description = "The dynamic simulation result"),
@ApiResponse(responseCode = "404", description = "Dynamic simulation result has not been found")})
public Mono<ResponseEntity<Boolean>> getResult(@Parameter(description = "Result UUID") @PathVariable("resultUuid") UUID resultUuid) {
Mono<Boolean> result = dynamicSimulationService.getResult(resultUuid);
public Mono<ResponseEntity<UUID>> getTimeSeriesResult(@Parameter(description = "Result UUID") @PathVariable("resultUuid") UUID resultUuid) {
Mono<UUID> result = dynamicSimulationService.getTimeSeriesId(resultUuid);
return result.map(r -> ResponseEntity.ok().contentType(MediaType.APPLICATION_JSON).body(r))
.defaultIfEmpty(ResponseEntity.notFound().build());
}

@GetMapping(value = "/results/{resultUuid}/timeline", produces = "application/json")
@Operation(summary = "Get a dynamic simulation result from the database")
@ApiResponses(value = {@ApiResponse(responseCode = "200", description = "The dynamic simulation result"),
@ApiResponse(responseCode = "404", description = "Dynamic simulation result has not been found")})
public Mono<ResponseEntity<UUID>> getTimeLineResult(@Parameter(description = "Result UUID") @PathVariable("resultUuid") UUID resultUuid) {
Mono<UUID> result = dynamicSimulationService.getTimeLineId(resultUuid);
return result.map(r -> ResponseEntity.ok().contentType(MediaType.APPLICATION_JSON).body(r))
.defaultIfEmpty(ResponseEntity.notFound().build());
}
Expand All @@ -63,8 +75,8 @@ public Mono<ResponseEntity<Boolean>> getResult(@Parameter(description = "Result
@Operation(summary = "Get the dynamic simulation status from the database")
@ApiResponses(value = {@ApiResponse(responseCode = "200", description = "The dynamic simulation status"),
@ApiResponse(responseCode = "404", description = "Dynamic simulation status has not been found")})
public Mono<ResponseEntity<String>> getStatus(@Parameter(description = "Result UUID") @PathVariable("resultUuid") UUID resultUuid) {
Mono<String> result = dynamicSimulationService.getStatus(resultUuid);
public Mono<ResponseEntity<DynamicSimulationStatus>> getStatus(@Parameter(description = "Result UUID") @PathVariable("resultUuid") UUID resultUuid) {
Mono<DynamicSimulationStatus> result = dynamicSimulationService.getStatus(resultUuid);
return result.map(r -> ResponseEntity.ok().contentType(MediaType.APPLICATION_JSON).body(r))
.defaultIfEmpty(ResponseEntity.notFound().build());
}
Expand All @@ -85,4 +97,12 @@ public ResponseEntity<Mono<Void>> deleteResults() {
return ResponseEntity.ok().body(result);
}

@PutMapping(value = "/results/{resultUuid}/stop", produces = APPLICATION_JSON_VALUE)
@Operation(summary = "Stop a dynamic simulation computation")
@ApiResponses(value = {@ApiResponse(responseCode = "200", description = "The dynamic simulation has been stopped")})
public ResponseEntity<Mono<Void>> stop(@Parameter(description = "Result UUID") @PathVariable("resultUuid") UUID resultUuid,
@Parameter(description = "Result receiver") @RequestParam(name = "receiver", required = false) String receiver) {
Mono<Void> result = dynamicSimulationService.stop(receiver, resultUuid);
return ResponseEntity.ok().body(result);
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,8 @@
* @author Abdelsalem Hedhili <abdelsalem.hedhili at rte-france.com>
*/
public enum DynamicSimulationStatus {
NOT_DONE,
RUNNING,
COMPLETED
CONVERGED,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To check but I think SUCCEED and FAILED is more accurate than CONVERGED and DIVERGED for a dynamic simulation.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But we can see that later

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In my opinion, CONVERGED and DIVERGED are two sub-states of SUCCEED. In contrast, FAILED is another state which relates to a technical issue.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok to discuss later

DIVERGED
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
/**
* Copyright (c) 2022, RTE (http://www.rte-france.com)
* This Source Code Form is subject to the terms of the Mozilla Public
* License, v. 2.0. If a copy of the MPL was not distributed with this
* file, You can obtain one at http://mozilla.org/MPL/2.0/.
*/
package org.gridsuite.ds.server.dto.dynamicmapping;

import lombok.AllArgsConstructor;
import lombok.Getter;
import lombok.NoArgsConstructor;
import lombok.Setter;

import java.util.Date;

/**
* @author Thang PHAM <quyet-thang.pham at rte-france.com>
*/
@NoArgsConstructor
@AllArgsConstructor
@Getter
@Setter
public class Script {

private String name;

// name of the original mapping
private String parentName;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What are the purposes of parentName and current?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This Dto is from the one in dynamic-mapping-server, so may be some fields are not really useful in dynamic-simulation-server.
'parentName' is the name of mapping
'name' is the name of script given by dynamic-mapping-server during the generation
In fact, these two names are not used for the computation but may be useful for the log, so I will keep these fields.

The field 'current', as I see in the implementation ScriptServiceImpl#isScriptCurrent(Script) in dynamic-mapping-server, is used to say that the script generated at some moment is out of date or not (in comparing to the modified date of the mapping). This field is not used in dynamic-simulation-server since the script is always generated on-demand.

I will remove this field in this Dto in dynamic-simulation-server


private String script;

private Date createdDate;

private String parametersFile;

}
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
/**
* Copyright (c) 2023, RTE (http://www.rte-france.com)
* This Source Code Form is subject to the terms of the Mozilla Public
* License, v. 2.0. If a copy of the MPL was not distributed with this
* file, You can obtain one at http://mozilla.org/MPL/2.0/.
*/
package org.gridsuite.ds.server.dto.timeseries;

import lombok.AllArgsConstructor;
import lombok.Getter;
import lombok.NoArgsConstructor;
import lombok.Setter;

import java.util.UUID;

/**
* @author Thang PHAM <quyet-thang.pham at rte-france.com>
*/
@NoArgsConstructor
@AllArgsConstructor
@Getter
@Setter
public class TimeSeriesGroupInfos {

private UUID id;
}
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
License, v. 2.0. If a copy of the MPL was not distributed with this
file, You can obtain one at http://mozilla.org/MPL/2.0/.
*/
package org.gridsuite.ds.server.repository;
package org.gridsuite.ds.server.model;

import org.springframework.data.domain.Persistable;

Expand Down
Loading