Feature/optimization discrete variables (#670)

* glop solver activated intVariable array added to store information on discrete variables * Adding discrete value array in problem creation Renaming intVar into VariablesEntieres Allocation + filling values during optimization variables creation * Giving discrete variables info to solvers * Making use of MakeVar(min, max, isInteger, name) * filling VariablesEntieres complete * Fix xpress identifier, use MIP solver if the problem is MIP * Deactivating heuristic and second optimization (by commenting) Only bringing back reduced costs and dual values when using MIP model * Remove commented second optimization ERRATUM for preceding commit : reduced costs and dual values when using *LP* model * Reactivated heuristics / second optimization Adding an epsilon to int vars to measure stability through heuristics * Use of round function for int vars out of XPRESS * Test : remove epsilon to output of XPRESS Check influence on heuristic * Going back without 2nd optimization Enabling solver output log * remove commented code * Renaming ucHeuristic and ucMILP (resp. to ucFast and ucAccurate) * also renaming selection methods * Adding "milp" parameter as option of optimisation conditionning "VariablesEntieres" to the use of milp parameter * activating 2nd optimization (if not milp) * No NODU smoothing needed for MILP resolution * changed memory required for input with MILP * (first draft) check of ortools if milp model * error messages changes * refactoring yearEndBuildFromThermalClusterIndex separating smoothing and costs computing not activating smoothing when MILP resolution * reactivating NODU smoothing for MILP solve * fix case * Cleaning, renaming after comments Moving isMIP to named_problem * adding methods to compute ON_Min and ON_Max * remove unused variable * Apply clang-format * remaking isMIP into a method * removing const typing * [TEMP] Test log xpress + 1 thread * remove xpress parameters * adding parameters --ortools-parameters="" for specific solver parameters string --ortools-verbose to activate verbosity * [TEMP] removing ortools presolve * Revert "[TEMP] removing ortools presolve" This reverts commit 9500cec. * changing parameters setting for ortools solvers * 2 solvers kept in memory for optim 1/2 for milp, 2nd optim with min/max fixed for M * [TEMP] individual year objectives * Revert "[TEMP] individual year objectives" This reverts commit 7d6164c. * Apply clang-format * fix : fixing NODU min and max for ALL timesteps in MILP mode * Fix build, formatting * Remove ortoolsVerbose & ortoolsParamsString * Simplify condition a bit * Fix variable type in MPS writer * [skip ci] Add TODO * Add forgotten data member * Revert state.* * One more fix * Fix build for GUI * Factorize UI code a bit * Move `checkOrtoolsUsage` * Add comment * Restore existing behavior for unitCommitmentMode = ucHeuristicAccurate * Small clean-up * Restore dual values & reduced costs * Renaming * Setup relaxation * Revert "Setup relaxation" This reverts commit 1c6411b. * Remove relaxation, use 0 for reduced cost & marginal values * Format * Partial revert * Disable variables requiring dual values / marg costs when ucMode = milp * Add comments * Add comments in `enum UnitCommitmentMode` * Remove obsolete TODO note * Add OutputRetriever::thermalNbUnitsON * Add 2 unit tests with COIN * Remove obsolete TODO note * Fix formatting * Fix formatting * Skip 2nd optimization when solving with integer variables --------- Co-authored-by: Florian OMNES <[email protected]> Co-authored-by: Florian Omnes <[email protected]>
AntaresSimulatorTeam · Oct 13, 2023 · 8dbba7e · 8dbba7e
1 parent 27de580
commit 8dbba7e
Show file tree

Hide file tree

Showing 29 changed files with 351 additions and 110 deletions.
diff --git a/docs/reference-guide/13-file-format.md b/docs/reference-guide/13-file-format.md
@@ -1,10 +1,19 @@
 # Study format changes
 This is a list of all recent changes that came with new Antares Simulator features. The main goal of this document is to lower the costs of changing existing interfaces, both GUI and scripts.
-## v8.7.1
+## v8.8.0
 ### Input
 ### Short-term storage
 If no value is specified for `initiallevel`, then a default value of 50% is used. Note that this value is used only if `initialleveloptim=false`, and that `false` is the default value for `initialleveloptim`.
 
+### Experimental "MILP" mode
+New value `milp` for existing property `other preferences/unit-commitment-mode` in file **settings/generaldata.ini**.
+
+Using this property requires OR-Tools and a MILP solver (XPRESS, COIN)
+
+```
+antares-8.8-solver --use-ortools --ortools-solver coin|xpress ...
+```
+
 ## v8.7.0
 ### Input
 #### Scenarized RHS for binding constraints

diff --git a/src/libs/antares/checks/antares/checks/checkLoadedInputData.h b/src/libs/antares/checks/antares/checks/checkLoadedInputData.h
@@ -26,6 +26,10 @@
 */
 namespace Antares::Check
 {
+void checkOrtoolsUsage(Antares::Data::UnitCommitmentMode ucMode,
+                       bool ortoolsUsed,
+                       const std::string& solverName);
+
 void checkStudyVersion(const AnyString& optStudyFolder);
 
 void checkSimplexRangeHydroPricing(Antares::Data::SimplexOptimization optRange,

diff --git a/src/libs/antares/checks/checkLoadedInputData.cpp b/src/libs/antares/checks/checkLoadedInputData.cpp
@@ -34,6 +34,25 @@
 
 namespace Antares::Check
 {
+void checkOrtoolsUsage(Antares::Data::UnitCommitmentMode ucMode,
+                       bool ortoolsUsed,
+                       const std::string& solverName)
+{
+    using namespace Antares::Data;
+    if (ucMode == UnitCommitmentMode::ucMILP)
+    {
+        if (!ortoolsUsed)
+        {
+            throw Error::IncompatibleMILPWithoutOrtools();
+        }
+
+        if (solverName == "sirius")
+        {
+            throw Error::IncompatibleMILPOrtoolsSolver();
+        }
+    }
+}
+
 void checkStudyVersion(const AnyString& optStudyFolder)
 {
     using namespace Antares::Data;

diff --git a/src/libs/antares/exception/LoadingError.cpp b/src/libs/antares/exception/LoadingError.cpp
@@ -72,6 +72,16 @@ IncompatibleParallelOptions::IncompatibleParallelOptions() :
 {
 }
 
+IncompatibleMILPWithoutOrtools::IncompatibleMILPWithoutOrtools() :
+ LoadingError("Unit Commitment mode 'milp' must be used with an OR-Tools solver ")
+{
+}
+
+IncompatibleMILPOrtoolsSolver::IncompatibleMILPOrtoolsSolver() :
+ LoadingError("'milp' mode does not work with OR-Tools using Sirius solver")
+{
+}
+
 IncompatibleOptRangeHydroPricing::IncompatibleOptRangeHydroPricing() :
  LoadingError("Simplex optimization range and hydro pricing mode : values are not compatible ")
 {

diff --git a/src/libs/antares/exception/antares/exception/LoadingError.hpp b/src/libs/antares/exception/antares/exception/LoadingError.hpp
@@ -92,6 +92,18 @@ class IncompatibleParallelOptions : public LoadingError
     IncompatibleParallelOptions();
 };
 
+class IncompatibleMILPWithoutOrtools : public LoadingError
+{
+public:
+    IncompatibleMILPWithoutOrtools();
+};
+
+class IncompatibleMILPOrtoolsSolver : public LoadingError
+{
+public:
+    IncompatibleMILPOrtoolsSolver();
+};
+
 class IncompatibleOptRangeHydroPricing : public LoadingError
 {
 public:

diff --git a/src/libs/antares/study/fwd.cpp b/src/libs/antares/study/fwd.cpp
@@ -264,8 +264,10 @@ UnitCommitmentMode StringToUnitCommitmentMode(const AnyString& text)
     s.trim();
     s.toLower();
     if (s == "fast")
-        return ucHeuristic;
-    if (s == "accurate") // mixed integer linear problem
+        return ucHeuristicFast;
+    if (s == "accurate")
+        return ucHeuristicAccurate;
+    if (s == "milp") // mixed integer linear problem
         return ucMILP;
 
     return ucUnknown;
@@ -275,10 +277,12 @@ const char* UnitCommitmentModeToCString(UnitCommitmentMode ucommitment)
 {
     switch (ucommitment)
     {
-    case ucHeuristic:
+    case ucHeuristicFast:
         return "fast";
-    case ucMILP:
+    case ucHeuristicAccurate:
         return "accurate"; // (slow)
+    case ucMILP:
+        return "milp"; // (possibly very slow)
     case ucUnknown:
         return "";
     }

diff --git a/src/libs/antares/study/fwd.h b/src/libs/antares/study/fwd.h
@@ -464,8 +464,16 @@ SheddingPolicy StringToSheddingPolicy(const AnyString& text);
 
 enum UnitCommitmentMode
 {
-    ucHeuristic = 0,
-    ucMILP, // mixed integer linear problem
+    //! Heuristic in which 2 LP problems are solved
+    //! No explicit modelling for the number of ON/OFF units
+    ucHeuristicFast = 0,
+    //! Heuristic in which 2 LP problems are solved
+    //! Explicit modelling for the number of ON/OFF units
+    ucHeuristicAccurate,
+    //! A single MILP problem is solved, with explicit modelling
+    //! for the number of ON/OFF units
+    ucMILP,
+    //! Unknown mode, mainly used for error handling
     ucUnknown,
 };
 

diff --git a/src/libs/antares/study/parameters.cpp b/src/libs/antares/study/parameters.cpp
@@ -302,7 +302,7 @@ void Parameters::reset()
     power.fluctuations = lssFreeModulations;
     shedding.policy = shpShavePeaks;
 
-    unitCommitment.ucMode = ucHeuristic;
+    unitCommitment.ucMode = ucHeuristicFast;
     nbCores.ncMode = ncAvg;
     renewableGeneration.rgModelling = rgAggregated;
 
@@ -730,7 +730,7 @@ static bool SGDIntLoadFamily_OtherPreferences(Parameters& d,
         }
         logs.warning() << "parameters: invalid unit commitment mode. Got '" << value
                        << "'. reset to fast mode";
-        d.unitCommitment.ucMode = ucHeuristic;
+        d.unitCommitment.ucMode = ucHeuristicFast;
         return false;
     }
     // Renewable generation modelling
@@ -1344,6 +1344,7 @@ void Parameters::prepareForSimulation(const StudyLoadOptions& options)
     std::vector<std::string> excluded_vars;
     renewableGeneration.addExcludedVariables(excluded_vars);
     adqPatchParams.addExcludedVariables(excluded_vars);
+    unitCommitment.addExcludedVariables(excluded_vars);
 
     variablesPrintInfo.prepareForSimulation(thematicTrimming, excluded_vars);
 
@@ -1792,4 +1793,25 @@ bool Parameters::RenewableGeneration::isClusters() const
 {
     return rgModelling == Antares::Data::rgClusters;
 }
+
+// Some variables rely on dual values & marginal costs
+void Parameters::UCMode::addExcludedVariables(std::vector<std::string>& out) const
+{
+    // These variables rely on dual values & marginal costs
+    // these don't really make sense for MILP problems
+    // TODO : solve a LP problem with fixed values for integer variables
+    //        extract values for dual variables & marginal costs from LP problem
+    const static std::vector<std::string> milpExclude = {{"MARG. COST"},
+                                                         {"BC. MARG. COST"},
+                                                         {"CONG. FEE (ALG.)"},
+                                                         {"CONG. FEE (ABS.)"},
+                                                         {"MRG. PRICE"},
+                                                         {"STS Cashflow By Cluster"},
+                                                         {"Profit by plant"}};
+
+    if (ucMode == ucMILP)
+    {
+        out.insert(out.end(), milpExclude.begin(), milpExclude.end());
+    }
+}
 } // namespace Antares::Data
diff --git a/src/libs/antares/study/parameters.h b/src/libs/antares/study/parameters.h
@@ -43,7 +43,6 @@
 
 #include <antares/study/UnfeasibleProblemBehavior.hpp>
 
-
 namespace Antares::Data
 {
 /*!
@@ -433,11 +432,15 @@ class Parameters final
         PowerFluctuations fluctuations;
     } power;
 
-    struct
+    struct UCMode
     {
         //! Unit Commitment Mode
         UnitCommitmentMode ucMode;
-    } unitCommitment;
+
+        //! Some variables rely on dual values & marginal costs
+        void addExcludedVariables(std::vector<std::string>&) const;
+    };
+    UCMode unitCommitment;
 
     struct
     {
@@ -488,7 +491,6 @@ class Parameters final
     SimplexOptimization simplexOptimizationRange;
     //@}
 
-
     AdequacyPatch::AdqPatchParams adqPatchParams;
 
     //! \name Scenariio Builder - Rules

diff --git a/src/solver/application.cpp b/src/solver/application.cpp
@@ -140,6 +140,9 @@ void Application::prepare(int argc, char* argv[])
 
     // Some more checks require the existence of pParameters, hence of a study.
     // Their execution is delayed up to this point.
+    checkOrtoolsUsage(
+      pParameters->unitCommitment.ucMode, pParameters->ortoolsUsed, pParameters->ortoolsSolver);
+
     checkSimplexRangeHydroPricing(pParameters->simplexOptimizationRange,
                                   pParameters->hydroPricing.hpMode);
 

diff --git a/src/solver/optimisation/opt_alloc_probleme_a_optimiser.cpp b/src/solver/optimisation/opt_alloc_probleme_a_optimiser.cpp
@@ -82,8 +82,11 @@ void OPT_AllocateFromNumberOfVariableConstraints(PROBLEME_ANTARES_A_RESOUDRE* Pr
     ProblemeAResoudre->Pi.assign(nbVariables, 0.);
     ProblemeAResoudre->Colonne.assign(nbVariables, 0);
 
+    // Names
     ProblemeAResoudre->NomDesVariables.resize(nbVariables);
     ProblemeAResoudre->NomDesContraintes.resize(nbConstraints);
+    // Integer variables ? (MILP)
+    ProblemeAResoudre->VariablesEntieres.resize(nbVariables);
 }
 
 static void optimisationAllocateProblem(PROBLEME_HEBDO* problemeHebdo, const int mxPaliers)

diff --git a/src/solver/optimisation/opt_appel_solveur_lineaire.cpp b/src/solver/optimisation/opt_appel_solveur_lineaire.cpp
@@ -104,8 +104,7 @@ static SimplexResult OPT_TryToCallSimplex(
         const int optimizationNumber,
         const OptPeriodStringGenerator& optPeriodStringGenerator,
         bool PremierPassage,
-        IResultWriter& writer
-        )
+        IResultWriter& writer)
 {
     const auto& ProblemeAResoudre = problemeHebdo->ProblemeAResoudre;
     auto ProbSpx
@@ -307,6 +306,7 @@ bool OPT_AppelDuSimplexe(const OptimizationOptions& options,
     const auto& ProblemeAResoudre = problemeHebdo->ProblemeAResoudre;
     Optimization::PROBLEME_SIMPLEXE_NOMME Probleme(ProblemeAResoudre->NomDesVariables,
                                                    ProblemeAResoudre->NomDesContraintes,
+                                                   ProblemeAResoudre->VariablesEntieres,
                                                    ProblemeAResoudre->StatutDesVariables,
                                                    ProblemeAResoudre->StatutDesContraintes,
                                                    problemeHebdo->NamedProblems);

diff --git a/src/solver/optimisation/opt_construction_variables_couts_demarrages.cpp b/src/solver/optimisation/opt_construction_variables_couts_demarrages.cpp
@@ -42,8 +42,9 @@ void OPT_ConstruireLaListeDesVariablesOptimiseesDuProblemeLineaireCoutsDeDemarra
 
     int nombreDePasDeTempsPourUneOptimisation
       = problemeHebdo->NombreDePasDeTempsPourUneOptimisation;
-    int nombreDeVariables = ProblemeAResoudre->NombreDeVariables;
+    int& nombreDeVariables = ProblemeAResoudre->NombreDeVariables;
     VariableNamer variableNamer(ProblemeAResoudre->NomDesVariables);
+    const bool intVariables = problemeHebdo->OptimisationAvecVariablesEntieres;
     for (uint32_t pays = 0; pays < problemeHebdo->NombreDePays; pays++)
     {
         variableNamer.UpdateArea(problemeHebdo->NomsDesPays[pays]);
@@ -71,6 +72,8 @@ void OPT_ConstruireLaListeDesVariablesOptimiseesDuProblemeLineaireCoutsDeDemarra
                   = nombreDeVariables;
                 ProblemeAResoudre->TypeDeVariable[nombreDeVariables]
                   = VARIABLE_BORNEE_DES_DEUX_COTES;
+
+                ProblemeAResoudre->VariablesEntieres[nombreDeVariables] = intVariables;
                 variableNamer.NODU(nombreDeVariables, clusterName);
                 nombreDeVariables++;
 
@@ -80,6 +83,7 @@ void OPT_ConstruireLaListeDesVariablesOptimiseesDuProblemeLineaireCoutsDeDemarra
 
                 ProblemeAResoudre->TypeDeVariable[nombreDeVariables]
                   = VARIABLE_BORNEE_INFERIEUREMENT;
+                ProblemeAResoudre->VariablesEntieres[nombreDeVariables] = intVariables;
                 variableNamer.NumberStartingDispatchableUnits(nombreDeVariables, clusterName);
                 nombreDeVariables++;
 
@@ -88,6 +92,7 @@ void OPT_ConstruireLaListeDesVariablesOptimiseesDuProblemeLineaireCoutsDeDemarra
                   = nombreDeVariables;
                 ProblemeAResoudre->TypeDeVariable[nombreDeVariables]
                   = VARIABLE_BORNEE_INFERIEUREMENT;
+                ProblemeAResoudre->VariablesEntieres[nombreDeVariables] = intVariables;
                 variableNamer.NumberStoppingDispatchableUnits(nombreDeVariables, clusterName);
                 nombreDeVariables++;
 
@@ -96,13 +101,11 @@ void OPT_ConstruireLaListeDesVariablesOptimiseesDuProblemeLineaireCoutsDeDemarra
                   = nombreDeVariables;
                 ProblemeAResoudre->TypeDeVariable[nombreDeVariables]
                   = VARIABLE_BORNEE_DES_DEUX_COTES;
+
+                ProblemeAResoudre->VariablesEntieres[nombreDeVariables] = intVariables;
                 variableNamer.NumberBreakingDownDispatchableUnits(nombreDeVariables, clusterName);
                 nombreDeVariables++;
             }
         }
     }
-
-    ProblemeAResoudre->NombreDeVariables = nombreDeVariables;
-
-    return;
 }