How to run portfolio optimization backtest with model forecasts?

[mtomic-quant]

I have daily prices of an asset going back 10 years, and I have a model which every day predicts the prices for the next 30 days. This means that the predictions for the same future days can change day-to-day as the model will get newer information.

I want to run Multi-Period optimization with horizon = 30, however the optimizer should take into account new forecasts daily and adjust positions accordingly. This is the code I started with where my_forecasted_returns are historical returns of the Asset forecasted by my model, and historical_returns are the actual returns of the Asset.

objective = cvx.ReturnsForecast(my_forecasted_returns) - cvx.FullCovariance()
constraints = [cvx.LeverageLimit(1)]

policy = cvx.MultiPeriodOptimization(objective, constraints, planning_horizon=30)

simulator = cvx.MarketSimulator(returns=historical_returns)

results = simulator.backtest(policy, initial_value=1000000)

This sketch does not take into account that I have multiple sets of returns (each day for 30 days ahead) rather than a single set.

[enzbus]

Discussion in #117 is relevant, let me know if that answers (my first post there). You need to specify the MPO policy with a list of 30 objective terms and list of 30 lists of constraints...

[mtomic-quant]

I ask as I keep getting the following error:

MissingTimesError: ('%s.values_in_time_recursive could not find data for time %s. This could be due to wrong timezone setting: in general Cvxportfolio objects are timezone aware, the data you pass should be as well.', pd.DataFrame(hash_values=23447de5f8, hash_index=1b1902af9a, hash_columns=38ffdc57cb), Timestamp('2023-01-04 00:00:00+0000', tz='UTC'))

with the code:

objectives = []
# these are all forecasts dataframe where each dataframe has 30 rows for 30 rows ahead forecasts
for df in dataframes:
    objectives.append(cvx.ReturnsForecast(df) - cvx.FullCovariance())

constraints = [cvx.LeverageLimit(1)]
constraints = [constraints] * len(dataframes)

policy = cvx.MultiPeriodOptimization(objective=objectives, constraints=constraints)

simulator = cvx.MarketSimulator(returns=historical_returns)

results = simulator.backtest(policy, initial_value=1000000, start_time='2023-01-04')

I used df = df.tz_localize('UTC') for all DataFrames to make them timezone-aware.

[enzbus]

Ok, that code snippet seems right, a few things:

• you should pass forecasts of returns, not prices; the return at time t is the ratio between the price at time t+1 and time t
• the timestamps of execution are implied by the returns inside the market data server used by the market simulator, you get them (in your code) by simulator.market_data.trading_calendar(); the index of the dataframes of the forecasts you pass should match it (all this is explained in the passing data page

[enzbus]

GH discussions are not supported, if you have an issue (bug, ...) please file an issue.

[mtomic-quant]

I meant returns not prices, sorry.

I checked the dates in simulator.market_data.trading_calendar() which in my case are in the range from 2021-08-13 until 2024-01-19. The list of objectives I pass include forecasted returns dataframes each with length 30 (30 days ahead forecast predictions). This means I have first objective with df starting from 2021-08-13 (same start date as simulator.market_data.trading_calendar()) and with 30 rows (30 days ahead), then the second objective has df starting from 2021-08-14..., third df starting from 2021-08-15 etc.

With this I keep getting

KeyError: Timestamp('2021-08-13 00:00:00+0000', tz='UTC')
MissingTimesError: ('%s.values_in_time_recursive could not find data for time %s. This could be due to wrong timezone setting: in general Cvxportfolio objects are timezone aware, the data you pass should be as well.', pd.DataFrame(hash_values=b95d35168d, hash_index=4e59de9756, hash_columns=38ffdc57cb), Timestamp('2021-08-13 00:00:00+0000', tz='UTC'))

[enzbus]

Seems like you haven't read the discussion in issue #117 😀

Your dataframes need to be indexed by the time of execution, not the future times (which are not known, in general).

[enzbus]

An improved explanation of the multi-period optimization model was just merged in master, can be seen on the development version of the docs https://www.cvxportfolio.com/en/master/manual.html#multi-period-optimization

[mtomic-quant]

Thanks, that was very useful.

I got it to work by having one single-row dataframe for each forecast. However my case is slightly different to the issue #117 as I have multiple forecasts for each timestamp as each day my model outputs 'fresher' predictions, from day ahead to 30 days ahead. Perhaps this is due to my lack of knowledge in portfolio optimization, what I assumed that we could do was feed 30 day ahead forecasts each day to the portfolio optimizer and have it run optimization with forecast_horizon = 30, then the next day feed it fresh predictions for the next 30 days and the optimizer will readjust.

[enzbus]

It doesn't sound like you're using it correctly; each dataframe is indexed by the time of execution, the first dataframe contains (each row) the forecast of the return for the same period, the second dataframe (each row) contains the forecast of the return for the period 1 step ahead, and so on. Why aren't you able to assemble your forecasts in dataframes?

[mtomic-quant]

I still don't understand the required structure of these DataFrames. Lets consider my starting point which is a collection of DataFrames where columns are asset names, indices are datetime, and values are forecasted returns for each of those dates. Since my model predicts 30 days ahead, this means my model generates each day a new DataFrame with forecasted returns of length 30.

For example, on 01-01-23 I generate predictions for next 30 days and get the following DataFrame:

Forecast for date	Asset_1	Asset_2
02-01-23	0.0025	0.003
03-01-23	0.001	0.002
04-01-23	0.0017	0.0019
…	…	…

Then next day on 02-01-23 I do this again and get the following DataFrame:

Forecast for date	Asset_1	Asset_2
03-01-23	0.0025	0.003
04-01-23	0.001	0.002
05-01-23	0.0017	0.0019
…	…	…

etc.

From what I understood I now need to create a collection of new DataFrames where each DataFrame should be indexed by time of execution (this would mean the index is the date the predictions were made rather than the date for which we are predicting). I assume the columns remain the same which are the names of the assets.

For the first DataFrame, I would then have:

Date of Forecast	Asset_1	Asset_2
02-01-23	0.0025	0.003

This only represents the day-ahead forecast, there are 29 remaining forecasts for other 29 days. Where do these other forecasts go? Are they added as new rows underneath in the following manner:

Forecast for date	Asset_1	Asset_2
02-01-23	0.0025	0.003
03-01-23	0.001	0.002
04-01-23	0.0017	0.0019
…	…	…

or does the index stay constant to represent date of execution (i.e. date when predictions were made):

Forecast for date	Asset_1	Asset_2
02-01-23	0.0025	0.003
02-01-23	0.001	0.002
02-01-23	0.0017	0.0019
…	…	…

Thank you for your time, it is much appreciated.

[enzbus]

I think if you were starting from SinglePeriodOptimization (which you should anyways) it would be clearer to you how it's done.

The datetime indexes are used to provide data for many times of execution, so you can back-test the policy. If you simply want one execution (e.g., for using Cvxportfolio online) you may as well pass a bunch of Series indexed by the assets names. This is all explained in the passing data manual section.

For multi-period optimization policies, if you want to pass data, the datetimes for which the predictions are made are not used, only the concept of "how many steps ahead" is used. So you have dataframes that are all indexed by the time of execution, and each prediction is made only with data available up to that time (that is enforced by Cvxportfolio internal forecasting logic, and you have to make sure you do the same otherwise you'd have look ahead biases).

In your case, the dataframe passed to the ReturnsForecast in the second step of the objective function contains the prediction for the market returns from timestamp 2023-04-24 to timestamp 2023-04-25 and is indexed by timestamp 2023-04-23. By the way, you should use full timestamps (with timezone and everything), that helps you thinking more clearly about what is being done.

The values_in_time method of Cvxportfolio estimators, which is the main method called on them, takes a mpo_step argument (0 for same period, 1 for next period, ...). Estimators are also aware of the trading calendar ahead of them in back-test, so in principle you could unwind those dataframes as you assembled them inside a Cvxportfolio estimator.

That is way more advanced, however, and I would suggest you start from a simple usecase (SPO), adding further step aheads slowly and observing the back-tested performance. Your workflow as you describe it sounds confusing, apologies if I'm mistaken, but it sounds hard for you to keep track of correctly consuming data (without look ahead biases) at the moment.

There's also another simpler way to have MPO aware return forecasts, and that's using the decay argument to ReturnsForecast. That internally also uses mpo_step.

Let me know how this continues.

[mtomic-quant]

Thank you very much for that info.

Quick last question: is there a way to run only the backtest given we already have the weights? (basically skip the portfolio optimisation part and just input the weights into the simulator)

[enzbus]

cvx.FixedWeights

But, please, let me know if you get your MPO workflow going. The initial development versions (0.0.*) of Cvxportfolio used something like you described for model forecasts of MPO assets returns, but it was confusing and depended on various deprecated things (like Pandas panels). We decided to simplify it. MPO is logically an extension of SPO, you should start from SPO and move to MPO once you get the first working; the interface should reflect that.

[mtomic-quant]

Can't we just:

• run MPO daily with horizon=30 and with fresh forecasts as a loop for the whole chosen backtest period,
• collect all the weights dataframes,
• select only the day ahead weight from each weights dataframe to build a new weights dataframe
• pass the new weights dataframe as an input to cvx.FixedWeights which we then pass as a policy into the simulator backtest?

[enzbus]

That sounds complicated 😃, why can't you just back-test the MPO policy using correct indexing of the model forecast dataframes? If you want to inspect the weights afterwards you get them with the w attribute of the BacktestResult object. Why don't you familiarize yourself with the library by looking at the examples, or, the docs?