Transformer angle shift is not considered in DC OPF

[lthurner]

When testing the PowerModels DC OPF through pandapower, I encountered an unexpected behaviour in the DC OPF. While transformer shift is considered in AC OPF, it seems to be ignored in the DC OPF. This can be reproduced with:

using PowerModels
using Ipopt

run = run_dc_opf
network_data = PowerModels.parse_file("case5.m")
network_data["branch"]["5"]["shift"] = 0.17 #setting the shift of the transformer
result = run(network_data, IpoptSolver())
va_before = [value["va"] for value in values(result["solution"]["bus"])]
network_data["branch"]["5"]["shift"] = 0.0
result = run(network_data, IpoptSolver()) #setting the shift of the transformer
va_after = [value["va"] for value in values(result["solution"]["bus"])]
println(va_before)
println(va_after)

Which gives the exact same voltage angles before and after changing the transformer shift. If run_dc_opf is replaced by run_ac_opf in the first line, it can be seen that in the AC OPF case it does lead to a change in voltage angles. Is this intended behaviour, or a bug? I ran the same test with PYPOWER, and there the transformer shift is considered in the DC OPF - so I assume it is probably a bug rather than intended?

[ccoffrin]

When first implementing the AC-OPD and DC-OPF models back in 2016, my primary goal was to match Matpower, more or less exactly. The omission of phase shifters from DC-OPF probably came from there.

If you would like PowerModels to move toward a more realistic DC-OPF model, I think that is a good idea at this point in time. Such a chance would need to wait for the next breaking release of PowerModels, v0.9.

[lthurner]

When first implementing the AC-OPD and DC-OPF models back in 2016, my primary goal was to match Matpower, more or less exactly. The omission of phase shifters from DC-OPF probably came from there.

PYPOWER does consider angle shift in DC OPF though, so I am assuming MATPOWER does too?

If you would like PowerModels to move toward a more realistic DC-OPF model, I think that is a good idea at this point in time. Such a chance would need to wait for the next breaking release of PowerModels, v0.9.

I understand, its also not a very pressing issue for me right now, just something I came across when testing out the interface and comparing results in pandapower when running the OPF through PYPOWER vs. running it through PowerModels. But if you think this is something I could also look into, I would willing to have a look at it if you point me into the right direction?

[ccoffrin]

Before we get too deep into improving the DC Power Flow formulation, it would be helpful for me to have a better understanding of what types of problems PowerModels users want to solve with the DC Power Flow and what are the reasonable assumptions of problem domains. For example, is it primarily production cost models, market clearing, operations?

Remarks from @jd-lara, @claytonpbarrows, @lsindoni, @frederikgeth, @hakanergun, @kersulis, @lroald, on this topic would be much appreciated.

[jd-lara]

@ccoffrin The short answer is that they are useful when AC models are too slow or the system is too weak to find a solution in AC. Although, with new convex relaxations this is just partially true because if the system is infeasible in AC the convex relaxation can help find out why. But this is not common practice yet.

DC models are useful for operational problems with binary variables in multiperiod problems, not that there aren't powerful MINLP solvers, but the question of scalability can hinder the use of MINLP formulations for this.

Honestly, the only necessary case I can't think of is solving the UC problem for day-ahead market clearing modeling.

[frederikgeth]

I think it is still common practice to use linearized DC OPF in SCOPF (due to sheer size + sometimes binaries), economic dispatch, unit commitment (including SCUC) and generation capacity expansion planning. Having a high-quality DC OPF implementation is useful for benchmarking (and convincing people to move away from DC). Another use case is simply lack of data on reactive power sources (e.g. here) to build an AC-feasible case.

Not considering a phase shift in linearized DC OPF is a bug to me, and if matpower's behavior is confirmed, an issue should be opened.

[lroald]

To add to this:
DC OPF is very valuable in many settings where you develop new models and formulations.
Take the example of doing optimization under uncertainty: Modelling propagation of uncertainty through a non-linear system is a nightmare, and hence there are good reasons to first tackle the problem in the DC setting, gain some insights and intuition, before attempting to handle the full AC.
I believe there are many such problems where it is a reasonable first step to do a DC implementation (and I consider the implementation in PowerModels to be a very useful starting point for this kind of endavours, so a good DC model is definitely a plus).

Frederik is mentioning cases where you lack information about reactive power - there are also cases where the OPF is just a minor part of your overall problem and you don't want to run into issues of handling either non-convexity or dealing with potentially badly performing relaxations (which are not always great if you push them in the wrong direction, such as maximizing losses). For example when you co-optimize gas and electric systems - as you are starting out, it may be nice to focus on the ugliness of the dynamic gas and rely on a DC model for the electricity side.

Beyond that, DC OPF is simply one of the most prevalent and important models used in power systems modelling, operations and planning.

Some things that have been of value to me in DC OPF in the past and the present:

• Phase shifters for power flow control :)
• Linear sensitivity factors (the so-called PTDFs and LODFs etc)

[ccoffrin]

Thanks all for your input! My follow up question; there are many possible variants of the DC Power Flow, which may be more appropriate in different contexts / types of analysis.

Which ones would be the most valuable in PowerModels? What would be the best "default" version?

[jd-lara]

I would say a Lossy DC Power Flow model based on injections is the most useful and flexible for extension. Also is a more natural follow up from the general AC formulation used in PowerModels.

This is a similar conversation to the one at NREL. The most efficient model depends on the ratio of "dispatchable devices" and "branches.

My guess is that for a regular user of DC Power Flow representing devices like Demand Response, EV's etc is more important. So I would pick the formulation with injection terms even if it adds some extra constraints.

[claytonpbarrows]

From my perspective, there are two reasons to use linearized DC OPF instead of ACOPF

1. Computational tractability...
2. Economic/market analysis... all electricity markets clear using some form of DC OPF and most investment decisions seem to be made from DC OPF results (LMP, congestion prices, etc.)

It seems perfectly reasonable to make the B-theta the default formulation. To me, that makes the most sense as it nicely connects with other PowerModels specifications, such as TNEP and OTS.

[lthurner]

I think all of the important points have been mentioned, I just wanted to add that I believe you can differentiate broadly between two use cases for DC OPF:

1. Use cases where an AC OPF would be preferred, but isn't feasible due to convergence issues or computational limitations.
2. Use cases where a linearized OPF is completely satisfactory, for example whe when the power grid is not the main focus of investigation. As an example, PyPSA is a very popular tool for energy modelers that only implements a linearized OPF. Joint gas and electricity optimization has also been mentioned as an example of this above.

Some of the users in 1. might be persuaded to use AC models with relaxations that allow better convergence. But there will be many who will still rely on linearization (SCOPF has been mentioned as one example), and many who don't really see an advantage in upgrading to a full AC model within their use case. So the DC OPF will certainly continued to be used a lot in the future, even with improving AC methods. So having a well maintained DC OPF in addition to the AC OPF would definetly a huge plus for PowerModels.

Not considering a phase shift in linearized DC OPF is a bug to me, and if matpower's behavior is confirmed, an issue should be opened.

MATPOWER seems to behave correctly though, the bug is in PowerModels. Of course if the DC OPF is up for a major overhaul anyways, it might not be worth it to fix it in the current implementation.

[lsindoni]

As it has already been remarked by others, while AC is the real deal in principle, in practice my (and mine colleagues') goal is to either match market practices (basically DC version of the various optimisations) or to quickly get to something that generates a realistic power flow as input of other problems (especially for large or very large grids). In general, while there are for sure more qualified opinions than mine on the technical side, my main concern would be to have something that allows me to quickly run very large systems, even at the risk of not having a perfectly converged solution.

[ccoffrin]

Thanks everyone for your input on this. I have started to explore how we might be able to improve the DC-OPF model in the dc-opf branch.

Currently I am exploring two points,

1. Incorporating transformer parameters into the DC-OPF model.
2. Checking if some variants of the formulation improve performance.

Concerning transformer parameters. The crux of the challenge is that the transformer parameter should be in the polar form, to include it in the DC-OPF model; but until now PowerModels has focused on representing the transformer in rectangular form (this is preferable for relaxations).

Concerning problem variants. Currently PowerModels adds explicit line flow variables (i.e. p_{i,j}) in the DC-OPF model. This could impact performance in LP solvers, so I double checked if these are a source of performance degradation.

This document provides some detailed results on the PGLib cases with more than 1000 buses.

Based on this experiment conclusion is as follows,

• The default AC-OPF form in PowerModels can have transformer parameter in polar form; then the default DC-OPF formulation will naturally incorporate the angle shift parameter.
• At last of the OPF problems I am testing on, there does not seems to be a significant performance degradation when adding the extra flow variables (probably the LP pre-solver is being clever); so leaving those in seems acceptable for the moment.

An important remark. If this change is made, intermediate PowerModels users will need to have some understanding of the different ways of stating the transformer parameter; and its implications on the quality and performance of different methods (e.g. approximations vs relaxations).

Note, my usual DC-OPF tests are quite fast, usually <1 second. Are there any other prototypical problems that would be recommended for testing a DC power flow formulation's performance?

[ccoffrin]

@lsindoni, if you run into any issues with the performance of an AC model or the solution quality of a DC model, let me know. There are a variety of options inbetween that might be of interest.

[ccoffrin]

Following up on this discussion. This week @kaarthiksundar has been exploring the possibility of supporting PTDF style models in PowerModels. After some discussions my conclusion is that these models may be too specialized to readily support in PowerModels' core (although they are a good candidate for PowerModelsAnnex). My primary concern is that even if we constrain our selves to the active-power-only case, the existence any dispatchable edge in the network precludes a PTDF formulation (or have I missed something?). Notable breaking examples would be HVDC lines (already in PowerModels standard OPF model) and FACTS devices.

Are there any objections and/or counter arguments to this claim?

This said, if folks really want PTDF formulations to be supported in PowerModels core, I will support them. However, various data filters would be added and they would only work on very specific subsets of the PowerModels data model (e.g. without any HVDC lines).

[ccoffrin]

Closing as I belive the bulk of these features are now supported. If something was missed please open a new issue.