diff --git a/README.md b/README.md index 7b90736..f042f1b 100644 --- a/README.md +++ b/README.md @@ -2,7 +2,7 @@ --------------- -This repository contains code and resources for automated algorithmic trading implementations. All content provided in this repository is for educational and informational purposes only. It is **NOT** intended to be a financial product or investment advice. By viewing, citing, using, or contributing to the contents and resources provided in this repository, you automatically acknowledge and agree to all terms and conditions found in [`DISCLAIMERS`](DISCLAIMER.MD), both present and future, unless explicitly stated otherwise. +This repository contains code and resources for automated algorithmic trading implementations. All content provided in this repository is for educational and informational purposes only. It is **NOT** intended to be a financial product or investment advice. By viewing, citing, using, or contributing to the contents and resources provided in this repository, you automatically acknowledge and agree to all terms and conditions found in [`DISCLAIMERS`](docs/DISCLAIMER.md), both present and future, unless explicitly stated otherwise. # Table of Contents @@ -92,7 +92,7 @@ The current version of this project stands on the shoulders of... its past versi # Contribution -Like software and finance? Have ideas on how to improve this project? Consider contributing! Refer to [`CONTRIBUTING.MD`](.github/CONTRIBUTING.MD) for more information. +Like software and finance? Have ideas on how to improve this project? Consider contributing! Refer to [`CONTRIBUTING`](docs/CONTRIBUTING.md) for more information. # Citation diff --git a/automoonbot/README.MD b/automoonbot/README.MD index eb01c3b..e262947 100644 --- a/automoonbot/README.MD +++ b/automoonbot/README.MD @@ -1,5 +1,15 @@ -# Planned File Structure +# Modules +## Portfolio Modeling + +Please refer to [here](session/README.md) for details. + +## Data + +... + + +## Planned File Structure ```plaintext backend/ ├── data/ diff --git a/automoonbot/session/README.MD b/automoonbot/session/README.MD index 0d38856..d80b64b 100644 --- a/automoonbot/session/README.MD +++ b/automoonbot/session/README.MD @@ -15,50 +15,246 @@ class Portfolio: name: { "size": size, "entry": price, + "price": price, + "exit": None, } }) + def update_position(self, name, price): + positions[name].update({ + "price": price, + }) + def close_position(self, name, price): position = positions.pop(name) position["exit"] = price - def position_value(self, name, price): + def position_value(self, name): position = positions.get(name) - return position["size"] * (price / position["entry"]) + return position["size"] * ( + position["price"] / position["entry"] + ) ``` -This implementation would work fine for most cases. It creates an easy way to open, close positions, and to compute the current value of any open positios. However, if a large number of operations need to be performed simultaneously, the execution time grows linearly. Of course, modules suck as `multiprocessing` can be used to parallelize the operations, but it also adds a lot of complexity to the design and possibility for errors. +This implementation would work fine for most cases. It creates an easy and intuitive way to open, close, and update positions. However, if a large number of operations need to be performed in quick succession or simultaneously (*which is the intention*), the performance can quickly become untenable. Of course, modules such as `multiprocessing` can be used to parallelize the operations, but it also adds a lot of complexity to the design and possibility for errors. This approach also comes with the added downside of needing to track and update a number of attributes for each position, which can get extremely tedious. + +## Matrix Representation -One great property for the positions to have is additivity, such that opening a new position is simply represented as `portfolio += position`, and closing a position `portfolio -= position`. To achieve that, we can represent positions as tensors. +Like many other things in life, a portfolio can be represented by matrices and would allow us to use matrix operations to execute transactions, which could bring significant benefits to the performance, especially when it comes to a large number of frequent transactions. -## Tensor Representation +For this purpose, we represent a portfolio as matrix $\mathbf{P} \in \mathbb{R}^{n \times m}$, where -Let's represent a single position as a matrix, where the row indicates the asset type (*e.g. equity, currency ...*), and columns representing the specific properties associated with the position (*e.g. size, value ...*). Then the entire portfolio can be expressed as an array of positions, or a 3D tensor with axes `[num_positions, position_types, position_attrs]`. +- $n$ is the number of rows, each row represents an unique tradable asset, which also includes the fiat currency -The cash balance of the portfolio can be represented as a `currency` position, specifically in `USD` for most viewers. To represent a trade, for instance 'buying $100 worth of TSLA stock', we first decompose it into two operations: +- $m$ is the number of columns, each column represents an attribute for the asset -1. Take $100 off cash balance: +As an example, say we have assets `{USD, BTC, TSLA, SPY}`, where `USD` is the fiat currency. The row index for each asset follows the same order - ``` - t1 = empty_portfolio_tensor - t1[balance_row, USD, [size]] = [-100] - ``` +```json +{USD: 0, BTC: 1, TSLA: 2, SPY: 3} +``` -2. Convert the $100 to the equivalent amount of TSLA stock +We then define the following attributes for each asset - ``` - t2 = empty_portfolio_tensor - t2[some_row, TSLA, [size, value]] = [TSLA_price / 100, TSLA_price] - ``` +- `value`: current value of the asset within the portfolio, in unit of fiat currency -Then, the entire trade can simply be represented as +- `log_quote`: the most recent market price quote for the asset, in $\log$ scale +- `lag_quote`: the previous market price quote for the asset, also in $\log$ scale + +The column attribute index also follows the same order + +```json +{value: 0, log_quote: 1, lag_quote: 2} ``` -t = t1 + t2 -``` -Finally, to execute the trade +If we start with $\$1000$ and hold no other assets, then we can initialize the matrix $\mathbf{P}$ as follows + +$$ +\mathbf{P} = +\begin{bmatrix} +1000 & 0 & 0 \\ +0 & 0 & 0 \\ +0 & 0 & 0 \\ +0 & 0 & 0 \\ +\end{bmatrix} +$$ + +The first thing we need to do is update the prices with some data, note since `USD` is in unit of the fiat currency itself, its price value will always be $1$, and since $\log(1)=0$, the `log_quote` and `lag_quote` attributes for `USD` will stay constant at $0$ + +Say `BTC` last traded at $\$50,000$, `TSLA` at $\$200$, and `SPY` at $\$500$, then + +$$ +\mathbf{P} = +\begin{bmatrix} +1000 & 0 & 0 \\ +0 & \log(50000) & 0 \\ +0 & \log(200) & 0 \\ +0 & \log(500) & 0 \\ +\end{bmatrix} +$$ +If we assume that we can fetch new data at each time step $t$, then we can write the data update loop as + +$\textbf{for} \, t=0 \, ... \, T \, \textbf{do}$ \ +$\quad \textbf{p} \leftarrow \textit{data}(t)$ \ +$\quad \textbf{for} \, i=1 \, ... \, 3 \, \textbf{do}$ \ +$\qquad \mathbf{P_{i, 2}} \leftarrow \mathbf{P_{i, 1}}$ \ +$\qquad \mathbf{P_{i, 1}} \leftarrow \log(\textbf{p}_i)$ \ +$\quad \textbf{end for}$ \ +$\textbf{end for}$ + +Or in other words, at each time step $t$, we get the prices for each asset, and we first set the `lag_quote` to be the previous `log_quote`, then we set `log_quote` to the current $\log$ price for each asset. Say after one time step, `BTC` trades at $\$45,000$, `TSLA` at $\$205$, and `SPY` at $\$510$, then + +$$ +\mathbf{P} = +\begin{bmatrix} +1000 & 0 & 0 \\ +0 & \log(45000) & \log(50000) \\ +0 & \log(205) & \log(200) \\ +0 & \log(510) & \log(500) \\ +\end{bmatrix} +$$ + +After at least 1 time step from initialization, we can compute the update matrix as + +$$ +\mathbf{U_{i}} = \mathbf{P_{i, 1}} - \mathbf{P_{i, 2}} +$$ + +but for purposes we'll soon get into, as a diagnoal matrix, where each diagnoal value represents the asset log-return for the time step + +$$ +\mathbf{U} = +\begin{bmatrix} +1 & 0 & 0 & 0 \\ +0 & \log(45000) - \log(50000) & 0 & 0 \\ +0 & 0 & \log(205) - \log(200) & 0 \\ +0 & 0 & 0 &\log(510) - \log(500) \\ +\end{bmatrix} +$$ + +At first glance, all of this may seem confusing and redundant, but in practice, since these are matrix operations, they can all be performed in a single step for all assets + +```python +import numpy as np + +quotes = get_market_prices(...) + +P[:, lag_quote_index] = P[:, log_quote_index] +P[:, log_quote_index] = np.log(quotes) +U = np.diag(np.diff(P[:, [lag_quote_index, log_quote_index]])) ``` -portfolio += t -``` \ No newline at end of file + +Now comes the best part. Transactions such as buying or selling assets can also be expressed as a matrix $\mathbf{T} \in \mathbb{R}^{n \times n}$, where each row represents the source asset, and each column represents the target asset. Say we want to buy $\$500$ ($50\%$ *of the cash balance*) worth of `TSLA`, then the transaction matrix can be written as + +$$ +\mathbf{T} = +\begin{bmatrix} +1-500/1000 & 0 & 500/1000 & 0 \\ +0 & 1 & 0 & 0 \\ +0 & 0 & 1 & 0 \\ +0 & 0 & 0 & 1 \\ +\end{bmatrix} +$$ + +Once the transaction matrix is constructed, it can be executed by multiplying it to the value column of the portfolio matrix + +$$ +\mathbf{P_v} = +\begin{bmatrix} +1000 \\ +0 \\ +0 \\ +0 \\ +\end{bmatrix} +\begin{bmatrix} +-0.5 & 0 & 0.5 & 0 \\ +0 & 1 & 0 & 0 \\ +0 & 0 & 1 & 0 \\ +0 & 0 & 0 & 1 \\ +\end{bmatrix} += +\begin{bmatrix} +500 \\ +0 \\ +500 \\ +0 \\ +\end{bmatrix} +$$ + +Simplying by multiplying the portfolio by the transaction matrix, the portfolio updated the correct values for both `USD` and `TSLA`. Even better than that, we can hold multiple transactions in a single matrix for it to be executed at once. For example, if we then want to buy $\$100$ worth of `BTC` and sell $\$200$ worth of `TSLA` + +$$ +\mathbf{T} = +\begin{bmatrix} +1 - 100/500 & 100/500 & 0 & 0 \\ +0 & 1 & 0 & 0 \\ +200/500 & 0 & 1 - 200/500 & 0 \\ +0 & 0 & 0 & 1 \\ +\end{bmatrix} +$$ + +Then after executating the transaction matrix + +$$ +\mathbf{P_v} = +\begin{bmatrix} +500 \\ +0 \\ +500 \\ +0 \\ +\end{bmatrix} + +\begin{bmatrix} +1 - 100/500 & 100/500 & 0 & 0 \\ +0 & 1 & 0 & 0 \\ +200/500 & 0 & 1 - 200/500 & 0 \\ +0 & 0 & 0 & 1 \\ +\end{bmatrix} += +\begin{bmatrix} +600 \\ +100 \\ +300 \\ +0 \\ +\end{bmatrix} +$$ + +As a added bonus, instead of manually computing the position values using initial entry price and current price, we can use the update matrix $\mathbf{U}$ (*after taking the exponent to convert it from log scale to standard scale*) we computed earlier directly, by performing element-wise-multiplication to the transaction matrix first + +$$ +\mathbf{T_U} = \mathbf{T} \circ \exp(\mathbf{U}) +$$ + +Now if $\mathbf{T_U}$ is executed, the transactions will be performed, and the values for all assets will automatically be updated as well. At any given time, if we want to obtain the total value of our portfolio, we simply need to sum the value column from $\mathbf{P}$. + +$$ +\mathbf{V} = \sum_{i=0}^{n} \mathbf{P_{i, 0}} +$$ + +When working with the transaction matrix, it's better to think of it as a '*transfer of value*' rather than a financial transaction. Since each row represents the transaction source, and each column represents the target, it can be interpreted as $X\%$ of the cash value was taken away from `source` and added to `target`. To simplify the unit conversions, it's ideal to work with a dimensionless value instead of dollars, or + +$$ +\mathbf{P} = +\begin{bmatrix} +1 & 0 & 0 \\ +0 & 0 & 0 \\ +0 & 0 & 0 \\ +0 & 0 & 0 \\ +\end{bmatrix} +$$ + +$$ +\mathbf{T} = +\begin{bmatrix} +1-0.5 & 0 & 0.5 & 0 \\ +0 & 1 & 0 & 0 \\ +0 & 0 & 1 & 0 \\ +0 & 0 & 0 & 1 \\ +\end{bmatrix} +$$ + +Now it can be thought of as *0.5 units of value was transferred from USD to TSLA*. To convert it back to the fiat unit, we simply multiply the values by the initial cash balance. + +This set of tools provide very powerful abstractions to portfolio modeling, allowing very efficient executions of transactions, price and value updates to be done. \ No newline at end of file diff --git a/automoonbot/session/portfolio.py b/automoonbot/session/portfolio.py index e20a12b..25def18 100644 --- a/automoonbot/session/portfolio.py +++ b/automoonbot/session/portfolio.py @@ -1,6 +1,8 @@ import numpy as np +from enum import Enum +from typing import List, Dict + -# TODO I need to spend some time thinking about how to do this... class Portfolio: """ An object representing the portfolio of one session. @@ -8,127 +10,125 @@ class Portfolio: High precision float ops are required. All floats use `np.float64` for numeric stability. All float ops use `numpy` for the same reason. - May move to Fortran to utilize higher precision floats than `np.float64`. - - Positions (trades) represented as numpy array. - Rows are used to index a specific position. - Columns are used to index specific fields. """ - # Row Indices - _balance_index: int = 0 - - # Col Indices - _cols: int = 8 # Total number of columns - _time_index: int = 0 # Float timestamp for creation time - _type_index: int = 1 # Type of position, e.g. currency - _subtype_index: int = 2 # Subtype of position, e.g. USD - _size_index: int = 3 # Position size, in ratio of portfolio - _entry_index: int = 4 # Asset price at entry - _margin_index: int = 5 # The amount of margin used, default 1 - _exchange_index: int = 6 # Whether or not it can be traded currently - _expire_index: int = 7 # When the position expires, 0 for None - - # Position Types - _currency: np.float64 = 0 - _equity: np.float64 = 1 - _crypto: np.float64 = 2 - _options: np.float64 = 3 - _commodities: np.float64 = 4 + class ColAttr(Enum): + Value = 0 + LogQuote = 1 + LagQuote = 2 def __init__( self, - timestamp: np.float64, - init_balance: np.float64 = 1.0, - max_positions: int = 100, + fiat: str, + tradables: List[str], ) -> None: - self._initialize_positions(timestamp, init_balance, max_positions) + self.index_map = {t: i for t, i in enumerate(tradables)} + self.fiat = self.index_map[fiat] + self._portfolio = self._reset_portfolio(self.fiat, len(tradables)) - def reset( + def _reset_portfolio( self, - timestamp: np.float64, - init_balance: np.float64 = 1.0, - max_positions: int = 100, - ) -> None: - self._initialize_positions(timestamp, init_balance, max_positions) - - def _initialize_positions( - self, timestamp: np.float64, init_balance: np.float64, max_positions: int - ) -> None: - self._positions = np.zeros( - (max_positions, self.__class__._cols), + fiat: int, + rows: int, + ) -> np.ndarray: + portfolio = np.zeros( + ( + rows, + len( + self.__class__.ColAttr, + ), + ), dtype=np.float64, ) - self._open_slots = set(range(max_positions)) - self._open_slots.pop(self.__class__._balance_index) - self._positions[ - self.__class__._balance_index, + portfolio[ + :, [ - self.__class__._time_index, - self.__class__._type_index, - self.__class__._subtype_index, - self.__class__._size_index, - self.__class__._entry_index, - self.__class__._margin_index, - self.__class__._exchange_index, - self.__class__._expire_index, + self.__class__.ColAttr.LogQuote, + self.__class__.ColAttr.LagQuote, ], - ] = np.array( - [ - timestamp, - self.__class__._currency, - np.float64(hash("usd")), - init_balance, - 1.0, - 1.0, - 1.0, - 0.0, - ], - dtype=np.float64, - ) + ] = [1.0, 1.0] + portfolio[fiat, self.__class__.ColAttr.Value] = 1.0 + return portfolio - def open_position( + def _reset_lag(self) -> None: + self._portfolio[ + :, + self.__class__.ColAttr.LagQuote, + ] = self._portfolio[:, self.__class__.ColAttr.LogQuote] + + @property + def U( self, - timestamp: np.float64, - ptype: np.float64, - subtype: np.float64, - size: np.float64, - entry: np.float64, - margin: np.float64 = 1.0, - exchange: np.float64 = 1.0, - expire: np.float64 = 0.0, - ) -> int: - if not self._open_slots: - return -1 - slot = self._open_slots.pop() - self._positions[ - slot, - [ - self.__class__._time_index, - self.__class__._type_index, - self.__class__._subtype_index, - self.__class__._size_index, - self.__class__._entry_index, - self.__class__._exchange_index, - self.__class__._margin_index, - self.__class__._expire_index, - ], - ] = np.array( - [ - timestamp, - ptype, - subtype, - size, - entry, - margin, - exchange, - expire, - ], - dtype=np.float64, - ) - if ...: # TODO Checks if we have enough money first - pass - return slot + diag: bool = False, + ) -> np.ndarray: + u = np.exp( + np.diff( + self._portfolio[ + :, + [ + self.__class__.ColAttr.LagQuote, + self.__class__.ColAttr.LogQuote, + ], + ] + ) + )[:, 0] + self._reset_lag() + if diag: + return np.diag(u) + return u + + def update_quotes( + self, + quotes: Dict[str, float], + ) -> None: + keys = list(quotes.keys()) + index = [self.index_map[key] for key in keys] + value = [np.log(quotes[key]) for key in keys] + self._reset_lag() + self._portfolio[ + index, + self.__class__.ColAttr.LogQuote, + ] = value - def close_position(self): - pass + def _value_transfer( + self, + src: List[int] | int, + tgt: List[int] | int, + size: np.ndarray | np.float64, + ) -> np.ndarray: + t = self.U(diag=True) + t[src, src] -= size + t[src, tgt] += size + return t + + def _build_transaction( + self, + transactions: Dict[str, float], + ) -> np.ndarray: + src, tgt, size = [], [], [] + for transaction in transactions: + if transaction["type"] == "buy": + src.append(self.fiat) + tgt.append(self.index_map[transaction["asset"]]) + size.append(transaction["size"]) + elif transaction["type"] == "sell": + src.append(self.index_map[transaction["asset"]]) + tgt.append(self.fiat) + size.append(transaction["size"]) + size = np.array(size, dtype=np.float64) + return self._value_transfer(src, tgt, size) + + def apply_transaction( + self, + transaction: np.ndarray, + ) -> None: + self._portfolio[ + :, + self.__class__.ColAttr.Value, + ] = ( + self._portfolio[ + :, + self.__class__.ColAttr.Value, + ] + @ transaction + ) diff --git a/CONTRIBUTING.md b/docs/CONTRIBUTING.md similarity index 92% rename from CONTRIBUTING.md rename to docs/CONTRIBUTING.md index 55e0b5f..cf098f3 100644 --- a/CONTRIBUTING.md +++ b/docs/CONTRIBUTING.md @@ -16,7 +16,7 @@ As mentioned, AutoMoonBot is still super alpha and likely full of bugs. If you w ## Help Implement Incomplete Features -At this stage, many features are either incomplete or sub-optimal. +At this stage, many features are either incomplete or sub-optimal. Contribution in this category is highly appreciated. Check out [`FEATURES`](docs/FEATURES.md) for more information. ## Brand New Features diff --git a/DISCLAIMER.MD b/docs/DISCLAIMER.md similarity index 100% rename from DISCLAIMER.MD rename to docs/DISCLAIMER.md diff --git a/docs/FEATURES.MD b/docs/FEATURES.MD index 4e768b5..5e16e57 100644 --- a/docs/FEATURES.MD +++ b/docs/FEATURES.MD @@ -1 +1,7 @@ -# \ No newline at end of file +# Features + +## Priority + +- [`Tensor representation of portfolio`]() + +- ... \ No newline at end of file diff --git a/legacy/classical_ensemble/README.md b/legacy/classical_ensemble/README.md index edf95eb..cc8f17b 100644 --- a/legacy/classical_ensemble/README.md +++ b/legacy/classical_ensemble/README.md @@ -1,6 +1,6 @@ # An Ensemble of Models Trading Together -*Please review the disclaimers [`here`](../../DISCLAIMER.MD) before proceeding. By continue visiting the rest of the content, you agree to all the terms and conditions of the disclaimers.* +*Please review the disclaimers [`here`](../../docs/DISCLAIMER.md) before proceeding. By continue visiting the rest of the content, you agree to all the terms and conditions of the disclaimers.* ## Introduction diff --git a/legacy/policy_gradient/README.md b/legacy/policy_gradient/README.md index b4866d4..8cde99b 100644 --- a/legacy/policy_gradient/README.md +++ b/legacy/policy_gradient/README.md @@ -1,6 +1,6 @@ # Automated Trading with Policy Gradient -*Please review the disclaimers [`here`](../../DISCLAIMER.MD) before proceeding. By continue visiting the rest of the content, you agree to all the terms and conditions of the disclaimers.* +Please review the disclaimers [`here`](../../docs/DISCLAIMER.md) before proceeding. By continue visiting the rest of the content, you agree to all the terms and conditions of the disclaimers. ## Introduction @@ -10,7 +10,7 @@ ![](../../media/demo.gif) -- *Note that the .gif might take some time to load.* +- The gif might take some time to load. - Demonstration of system back-testing. Data used for this test was *SPDR S&P 500 ETF Trust (SPY)* with *1 hour* interval, with a start date in *June 2022.*