Stock Market Analysis.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 151,
   "id": "f5c50e8c-1d91-4159-bf11-c82a8b399a2e",
   "metadata": {},
   "outputs": [],
   "source": [
    "import yfinance as yf\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "from sklearn.preprocessing import MinMaxScaler\n",
    "from sklearn.ensemble import RandomForestRegressor\n",
    "from sklearn.metrics import make_scorer, mean_squared_error, mean_absolute_error, r2_score\n",
    "from sklearn.model_selection import cross_val_score, KFold\n",
    "from sklearn.neural_network import MLPRegressor\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "from sklearn.model_selection import GridSearchCV\n",
    "from sklearn.linear_model import LinearRegression"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c6fe85b6-3238-4b6d-96c4-33972d7f8f43",
   "metadata": {},
   "source": [
    "# Data Collection and Preprocessing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 154,
   "id": "18597fbf-5a1f-413e-953d-ad9abbfee699",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[*********************100%***********************]  3 of 3 completed\n"
     ]
    }
   ],
   "source": [
    "# Define tickers and date range\n",
    "tickers = ['^GSPC', '^IXIC', '^N225']\n",
    "start_date = '2011-01-01'\n",
    "end_date = '2022-01-01'\n",
    "\n",
    "# Fetch data\n",
    "data = yf.download(tickers, start=start_date, end=end_date, group_by='ticker')\n",
    "\n",
    "# Drop 'Close' from each ticker in the DataFrame\n",
    "for ticker in tickers:\n",
    "    if (ticker, 'Close') in data.columns:  # Check if the 'Close' column exists\n",
    "        data.drop(columns=[(ticker, 'Close')], inplace=True)  # Drop the 'Close' column\n",
    "\n",
    "# Apply LOCF for missing values across the entire dataset\n",
    "data = data.ffill()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0c3e03bb-2576-48f9-8192-dc30f7e3b7d4",
   "metadata": {},
   "source": [
    "# Feature Engineering\n",
    "\n",
    "The following code calculates several technical indicators for stock data, including Simple Moving Average (SMA), Relative Strength Index (RSI), Moving Average Convergence Divergence (MACD), and Bollinger Bands for multiple stocks. These indicators are widely used in the analysis of financial markets to help identify market trends and potential trading opportunities.\n",
    "\n",
    "## Indicators\n",
    "\n",
    "### 1. Simple Moving Average (SMA)\n",
    "- **Purpose**: Provides a smooth data series by calculating the average of a specified number of past prices.\n",
    "- **Calculation**: The 30-day SMA is calculated, which represents the average of the last 30 closing prices.\n",
    "\n",
    "### 2. Relative Strength Index (RSI)\n",
    "- **Purpose**: Measures the velocity and magnitude of directional price movements. It provides signals about bullish or bearish price momentum.\n",
    "- **Calculation**: The 14-day RSI is computed, which involves:\n",
    "  - Identifying the magnitude of recent gains and losses.\n",
    "  - Calculating the average of these gains and losses over 14 days.\n",
    "  - Normalizing the result to an oscillator between 0 and 100.\n",
    "\n",
    "### 3. Moving Average Convergence Divergence (MACD)\n",
    "- **Purpose**: Tracks the relationship between two moving averages of a stock's price.\n",
    "- **Calculation**:\n",
    "  - The MACD line is calculated as the difference between the 12-day and 26-day exponential moving averages (EMA) of the closing prices.\n",
    "  - A signal line, which is the 9-day EMA of the MACD line itself, is also calculated. This acts as a trigger for buy and sell signals.\n",
    "\n",
    "### 4. Bollinger Bands\n",
    "- **Purpose**: Measures market volatility and provides insights into price levels relative to previous trades.\n",
    "- **Calculation**:\n",
    "  - A 20-day SMA of the closing prices forms the middle band.\n",
    "  - The upper and lower bands are then set two standard deviations above and below this middle band, respectively.\n",
    " \n",
    "### Dropping NaN Values in Financial Data\n",
    "\n",
    "'NaN' values in technical indicators like SMA, RSI, and Bollinger Bands typically occur because there isn't enough historical data to perform the calculation (e.g., not enough past days for a 30-day SMA at the start of a dataset). Removing these 'NaN' values is beneficial as it ensures that all analyses and models are based only on periods where full data is available, thereby enhancing the reliability and validity of financial assessments and predictions.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 157,
   "id": "96174e2a-2183-4ccd-ad7f-8276ea4f01a5",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Assuming 'data' is your DataFrame\n",
    "\n",
    "# Define the window for moving averages and other indicators\n",
    "window_sma = 30\n",
    "window_rsi = 14\n",
    "window_macd_short = 12\n",
    "window_macd_long = 26\n",
    "signal_line_window = 9\n",
    "window_boll = 20\n",
    "\n",
    "for ticker in tickers:\n",
    "    # Get the position of 'Adj Close' for each ticker to start inserting new features after it\n",
    "    position = data.columns.get_loc((ticker, 'Adj Close')) + 1\n",
    "\n",
    "    # Insert SMA\n",
    "    sma_series = data[(ticker, 'Adj Close')].rolling(window=window_sma).mean()\n",
    "    data.insert(loc=position, column=(ticker, 'SMA_30'), value=sma_series)\n",
    "\n",
    "    # Insert RSI\n",
    "    delta = data[(ticker, 'Adj Close')].diff(1)\n",
    "    gain = delta.where(delta > 0, 0).rolling(window=window_rsi).mean()\n",
    "    loss = -delta.where(delta < 0, 0).rolling(window=window_rsi).mean()\n",
    "    RS = gain / loss\n",
    "    rsi_series = 100 - (100 / (1 + RS))\n",
    "    data.insert(loc=position + 1, column=(ticker, 'RSI_14'), value=rsi_series)\n",
    "\n",
    "    # Insert MACD and Signal Line\n",
    "    exp1 = data[(ticker, 'Adj Close')].ewm(span=window_macd_short, adjust=False).mean()\n",
    "    exp2 = data[(ticker, 'Adj Close')].ewm(span=window_macd_long, adjust=False).mean()\n",
    "    macd = exp1 - exp2\n",
    "    data.insert(loc=position + 2, column=(ticker, 'MACD'), value=macd)\n",
    "    data.insert(loc=position + 3, column=(ticker, 'Signal_Line'), value=macd.ewm(span=signal_line_window, adjust=False).mean())\n",
    "\n",
    "    # Insert Bollinger Bands\n",
    "    sma = data[(ticker, 'Adj Close')].rolling(window=window_boll).mean()\n",
    "    rstd = data[(ticker, 'Adj Close')].rolling(window=window_boll).std()\n",
    "    data.insert(loc=position + 4, column=(ticker, 'Upper_Band'), value=sma + 2 * rstd)\n",
    "    data.insert(loc=position + 5, column=(ticker, 'Lower_Band'), value=sma - 2 * rstd)\n",
    "\n",
    "# Drop rows where any of the specified indicators are NaN\n",
    "data.dropna(subset=[\n",
    "    ('^GSPC', 'SMA_30'), \n",
    "    ('^GSPC', 'RSI_14'), \n",
    "    ('^GSPC', 'Upper_Band'), \n",
    "    ('^GSPC', 'Lower_Band'),\n",
    "    ('^IXIC', 'SMA_30'),\n",
    "    ('^IXIC', 'RSI_14'),\n",
    "    ('^IXIC', 'Upper_Band'),\n",
    "    ('^IXIC', 'Lower_Band'),\n",
    "    ('^N225', 'SMA_30'),\n",
    "    ('^N225', 'RSI_14'),\n",
    "    ('^N225', 'Upper_Band'),\n",
    "    ('^N225', 'Lower_Band')\n",
    "    ], inplace=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3b94b7ad-f3ab-4449-9848-a7c7d9c14df2",
   "metadata": {},
   "source": [
    "# Normalization\n",
    "\n",
    "## Overview\n",
    "Normalization is a critical preprocessing step in data analysis, and for this dataset, we use the **MinMaxScaler** from Python's scikit-learn library to adjust the scales of features. This type of normalization transforms features by scaling them to a given range, specifically between 0 and 1. This approach is particularly beneficial for financial datasets as it preserves the relationships among data points while standardizing the range. By normalizing features to a common scale, we prevent variables with larger scales from dominating the model's behavior, which is crucial for algorithms sensitive to input scale such as neural networks and distance-based algorithms like k-NN.\n",
    "\n",
    "\n",
    "## Features Normalized\n",
    "- **Price-related Features**: 'Open', 'High', 'Low', and 'Adj Close' are normalized because these features vary significantly in magnitude and can skew the performance of machine learning models.\n",
    "- **Volume**: Since trade volume can range over several orders of magnitude, normalizing this feature helps to maintain its proportional impact relative to price features.\n",
    "- **Derived Indicators (SMA, Bollinger Bands)**: These are based on price data and share its scale. Normalizing these alongside price data ensures consistency in scale across all price-related features.\n",
    "\n",
    "## Features Not Normalized\n",
    "- **RSI**: This indicator ranges from 0 to 100, representing overbought and oversold conditions. Normalizing RSI would strip it of its interpretative value.\n",
    "- **MACD**: Though derived from price, the MACD is a difference between two EMAs and its value (including its signal line) has meaning in its scale relative to zero. Normalizing these would disrupt their threshold-based interpretation.\n",
    "\n",
    "\n",
    "The decision to normalize certain features while excluding others is based on maintaining the utility and interpretability of each feature. By standardizing the scale of direct measurements and derived statistics based on those measurements, we enhance model reliability without compromising the data's inherent signals.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 160,
   "id": "d17b84e4-bfbe-40fd-83c6-62332ae81919",
   "metadata": {},
   "outputs": [],
   "source": [
    "def normalize_data(df, features):\n",
    "    scaler = MinMaxScaler()\n",
    "    # Normalize the specified features, ensuring they are referenced correctly\n",
    "    df.loc[:, features] = scaler.fit_transform(df[features])\n",
    "    return df\n",
    "\n",
    "# Apply normalization to each DataFrame, correctly referencing multi-level columns\n",
    "for ticker in tickers:\n",
    "    # Specify multi-level columns including the ticker to ensure correct reference\n",
    "    features_to_normalize = [(ticker, 'Open'), (ticker, 'High'), (ticker, 'Low'), \n",
    "                             (ticker, 'Close'), (ticker, 'Adj Close'), (ticker, 'Volume'),\n",
    "                             (ticker, 'SMA_30'), (ticker, 'Upper_Band'), (ticker, 'Lower_Band')]\n",
    "    # Filter to ensure only existing columns are selected and passed\n",
    "    existing_features = [feature for feature in features_to_normalize if feature in data.columns]\n",
    "\n",
    "    # Check if there are valid features to normalize\n",
    "    if existing_features:\n",
    "        # Normalize using the correct multi-level column references\n",
    "        data.loc[:, existing_features] = normalize_data(data.loc[:, existing_features], existing_features)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 162,
   "id": "a6d77ad2-f40c-40d9-8f29-6feea5901376",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Separate DataFrames for each ticker\n",
    "gspc_df = data['^GSPC']\n",
    "ixic_df = data['^IXIC']\n",
    "n225_df = data['^N225']"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4275bcd2-a024-4b21-b53a-200c2241f824",
   "metadata": {},
   "source": [
    "# 10-fold cross validation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 165,
   "id": "df2f40a1-a2d4-4b37-886e-cf7463e3527f",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Set up 10-fold cross-validation\n",
    "kf = KFold(n_splits=10, shuffle=True, random_state=42)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fe0a954b-ec72-4357-99a6-5a940ca630f0",
   "metadata": {},
   "source": [
    "# Random Forest Regressor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 168,
   "id": "903c611e-09f6-44cc-82a4-17d8db3bcafa",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create the Random Forest model\n",
    "rf_regressor = RandomForestRegressor(\n",
    "    n_estimators=100,             # Increased number of trees\n",
    "    # bootstrap=True,               # Use bootstrap samples\n",
    "    random_state=42               # For reproducibility\n",
    ")\n",
    "\n",
    "# Define custom scorers\n",
    "mse_scorer = make_scorer(mean_squared_error, greater_is_better=False)\n",
    "mae_scorer = make_scorer(mean_absolute_error, greater_is_better=False)\n",
    "r2_scorer = make_scorer(r2_score)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eb200cd1-9d45-4ff3-be74-5a3666958af8",
   "metadata": {},
   "source": [
    "## RFR on S&P Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 171,
   "id": "a0be1496-d956-4ed6-92b6-c4399ebc0335",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "MSE scores for S&P 500: [1.48063623e-05 1.22616510e-05 1.43478235e-05 9.12638331e-06\n",
      " 1.24074748e-05 2.26566585e-05 1.14766475e-05 9.35079006e-06\n",
      " 1.72476938e-05 1.01890656e-05]\n",
      "MAE scores for S&P 500: [0.00222635 0.00228805 0.00211298 0.00195971 0.00221731 0.00236401\n",
      " 0.00212015 0.00198033 0.00217172 0.00209919]\n",
      "R^2 scores for S&P 500: [0.99970896 0.99977146 0.99972915 0.99982934 0.99979964 0.9995718\n",
      " 0.99977977 0.99979428 0.99972099 0.9998046 ]\n"
     ]
    }
   ],
   "source": [
    "# We need to define the features and target. Let's assume 'Adj Close' is the target\n",
    "features_gspc = gspc_df.drop(columns=['Adj Close'])\n",
    "target_gspc = gspc_df['Adj Close']\n",
    "\n",
    "# Perform cross-validation and compute scores\n",
    "mse_scores_gspc = cross_val_score(rf_regressor, features_gspc, target_gspc, cv=kf, scoring=mse_scorer)\n",
    "mae_scores_gspc = cross_val_score(rf_regressor, features_gspc, target_gspc, cv=kf, scoring=mae_scorer)\n",
    "r2_scores_gspc = cross_val_score(rf_regressor, features_gspc, target_gspc, cv=kf, scoring=r2_scorer)\n",
    "\n",
    "# Output results\n",
    "print(\"MSE scores for S&P 500:\", -mse_scores_gspc)\n",
    "print(\"MAE scores for S&P 500:\", -mae_scores_gspc)\n",
    "print(\"R^2 scores for S&P 500:\", r2_scores_gspc)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d8cb1bd3-4d78-4a22-9f0e-e42622f3385f",
   "metadata": {},
   "source": [
    "## RFR on NASDAQ Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 173,
   "id": "96b4dfc0-a4bf-4304-b2cc-1e80400228c5",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "MSE scores for NASDAQ: [1.30549427e-05 1.07078392e-05 9.61035478e-06 7.73488707e-06\n",
      " 1.36980464e-05 1.62326134e-05 8.20011611e-06 1.05245182e-05\n",
      " 1.37602084e-05 1.01711786e-05]\n",
      "MAE scores for NASDAQ: [0.00202795 0.0020236  0.00182898 0.00180328 0.00205591 0.00214368\n",
      " 0.00183939 0.00187244 0.00210486 0.00196556]\n",
      "R^2 scores for NASDAQ: [0.99977421 0.99982029 0.99984222 0.99986999 0.99981194 0.99973113\n",
      " 0.99985787 0.99979224 0.99980864 0.99983219]\n"
     ]
    }
   ],
   "source": [
    "# We need to define the features and target. Let's assume 'Adj Close' is the target\n",
    "features_ixic = ixic_df.drop(columns=['Adj Close'])\n",
    "target_ixic = ixic_df['Adj Close']\n",
    "\n",
    "# Perform cross-validation and compute scores\n",
    "mse_scores_ixic = cross_val_score(rf_regressor, features_ixic, target_ixic, cv=kf, scoring=mse_scorer)\n",
    "mae_scores_ixic = cross_val_score(rf_regressor, features_ixic, target_ixic, cv=kf, scoring=mae_scorer)\n",
    "r2_scores_ixic = cross_val_score(rf_regressor, features_ixic, target_ixic, cv=kf, scoring=r2_scorer)\n",
    "\n",
    "# Output results\n",
    "print(\"MSE scores for NASDAQ:\", -mse_scores_ixic)\n",
    "print(\"MAE scores for NASDAQ:\", -mae_scores_ixic)\n",
    "print(\"R^2 scores for NASDAQ:\", r2_scores_ixic)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "14b82306-2a0c-4623-b8b3-2f696542a330",
   "metadata": {},
   "source": [
    "## RFR on NIKKEI Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 175,
   "id": "ef9dbed0-c958-408d-8b41-b679ab74bfd3",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "MSE scores for Nikkei 225: [1.80358245e-05 1.14148252e-05 1.43070882e-05 2.41586877e-05\n",
      " 2.14360950e-05 2.07625722e-05 1.59510879e-05 1.59991453e-05\n",
      " 1.71860492e-05 1.88384792e-05]\n",
      "MAE scores for Nikkei 225: [0.0028082  0.0024297  0.002736   0.00294752 0.0030012  0.00290305\n",
      " 0.00258643 0.00261396 0.00281675 0.00273367]\n",
      "R^2 scores for Nikkei 225: [0.99971652 0.99983588 0.99978559 0.99963689 0.99969986 0.99965926\n",
      " 0.99974395 0.99976225 0.99974763 0.99970701]\n"
     ]
    }
   ],
   "source": [
    "# We need to define the features and target. Let's assume 'Adj Close' is the target\n",
    "features_n225 = n225_df.drop(columns=['Adj Close'])\n",
    "target_n225 = n225_df['Adj Close']\n",
    "\n",
    "# Perform cross-validation and compute scores\n",
    "mse_scores_n225 = cross_val_score(rf_regressor, features_n225, target_n225, cv=kf, scoring=mse_scorer)\n",
    "mae_scores_n225 = cross_val_score(rf_regressor, features_n225, target_n225, cv=kf, scoring=mae_scorer)\n",
    "r2_scores_n225 = cross_val_score(rf_regressor, features_n225, target_n225, cv=kf, scoring=r2_scorer)\n",
    "\n",
    "# Output results\n",
    "print(\"MSE scores for Nikkei 225:\", -mse_scores_n225)\n",
    "print(\"MAE scores for Nikkei 225:\", -mae_scores_n225)\n",
    "print(\"R^2 scores for Nikkei 225:\", r2_scores_n225)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "12dd5eb4-f6f9-4258-ae01-ebe2c8f8d322",
   "metadata": {},
   "source": [
    "# Multilayer Perceptrons"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 177,
   "id": "2940e496-d3c4-4651-806b-816b9db4e826",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Define the MLP regressor\n",
    "\n",
    "# Initialize MLPRegressor with specific parameters\n",
    "mlp_regressor = MLPRegressor(\n",
    "    hidden_layer_sizes=(100, 50),  # Two layers with 100 and 50 neurons\n",
    "    activation='tanh',             # 'relu' or 'tanh'\n",
    "    solver='adam',                 # 'adam' or 'sgd'\n",
    "    alpha=0.001,                   # Regularization strength\n",
    "    learning_rate='constant',      # 'constant' or 'adaptive'\n",
    "    max_iter=500,                  # Number of iterations\n",
    "    random_state=42\n",
    ")\n",
    "\n",
    "# Applying normalization to the features for the MLP model as it is sensitive to the magnitude of input features\n",
    "scaler = StandardScaler()\n",
    "\n",
    "def train_and_evaluate_mlp(df, features, target):\n",
    "    # Extract features and target from the dataframe\n",
    "    X = df[features]\n",
    "    y = df[target]\n",
    "    \n",
    "    # Scale the features\n",
    "    X_scaled = scaler.fit_transform(X)\n",
    "    \n",
    "    # Perform 10-fold cross-validation\n",
    "    mse_scores = cross_val_score(mlp_regressor, X_scaled, y, cv=kf, scoring=mse_scorer)\n",
    "    mae_scores = cross_val_score(mlp_regressor, X_scaled, y, cv=kf, scoring=mae_scorer)\n",
    "    r2_scores = cross_val_score(mlp_regressor, X_scaled, y, cv=kf, scoring=r2_scorer)\n",
    "    \n",
    "    return -mse_scores, -mae_scores, r2_scores"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 178,
   "id": "54aa3ffd-2737-4ca0-b58c-0a0d2242bec8",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "MLP MSE scores for S&P 500: [0.00014822 0.00016408 0.00030474 0.00024013 0.00020544 0.00017993\n",
      " 0.00020225 0.00013605 0.00019689 0.00023026]\n",
      "MLP MAE scores for S&P 500: [0.00801552 0.00859964 0.01003888 0.00928213 0.00936249 0.0095505\n",
      " 0.00898456 0.0079799  0.00945016 0.01062256]\n",
      "MLP R^2 scores for S&P 500: [0.99708647 0.99694183 0.99424722 0.99550974 0.99668252 0.99659949\n",
      " 0.99611896 0.9970069  0.99681499 0.99558423]\n",
      "MLP MSE scores for NASDAQ: [0.00015141 0.00020089 0.00026981 0.00022333 0.00028393 0.00024853\n",
      " 0.00014929 0.00023402 0.00020774 0.00019419]\n",
      "MLP MAE scores for NASDAQ: [0.0088507  0.01014937 0.01159099 0.0092713  0.01213857 0.01084358\n",
      " 0.00862199 0.0096106  0.01034803 0.00991406]\n",
      "MLP R^2 scores for NASDAQ: [0.99738122 0.9966285  0.99557027 0.99624628 0.99610207 0.99588344\n",
      " 0.99741251 0.99538041 0.99711097 0.99679603]\n",
      "MLP MSE scores for Nikkei 225: [0.00022522 0.00018381 0.00016502 0.00016396 0.00019817 0.00018634\n",
      " 0.00020809 0.00016137 0.00017377 0.00017907]\n",
      "MLP MAE scores for Nikkei 225: [0.01042532 0.00976491 0.0097674  0.00964808 0.01023029 0.00976925\n",
      " 0.01093907 0.00955472 0.00950941 0.00977306]\n",
      "MLP R^2 scores for Nikkei 225: [0.99646016 0.99735714 0.99752693 0.99753565 0.99722531 0.99694191\n",
      " 0.99665971 0.99760201 0.9974483  0.99721499]\n"
     ]
    }
   ],
   "source": [
    "# S&P 500 MLP Regression Evaluation\n",
    "features_gspc = gspc_df.drop(columns=['Adj Close']).columns.tolist()\n",
    "mse_scores_gspc, mae_scores_gspc, r2_scores_gspc = train_and_evaluate_mlp(gspc_df, features_gspc, 'Adj Close')\n",
    "print(\"MLP MSE scores for S&P 500:\", mse_scores_gspc)\n",
    "print(\"MLP MAE scores for S&P 500:\", mae_scores_gspc)\n",
    "print(\"MLP R^2 scores for S&P 500:\", r2_scores_gspc)\n",
    "\n",
    "# NASDAQ MLP Regression Evaluation\n",
    "features_ixic = ixic_df.drop(columns=['Adj Close']).columns.tolist()\n",
    "mse_scores_ixic, mae_scores_ixic, r2_scores_ixic = train_and_evaluate_mlp(ixic_df, features_ixic, 'Adj Close')\n",
    "print(\"MLP MSE scores for NASDAQ:\", mse_scores_ixic)\n",
    "print(\"MLP MAE scores for NASDAQ:\", mae_scores_ixic)\n",
    "print(\"MLP R^2 scores for NASDAQ:\", r2_scores_ixic)\n",
    "\n",
    "# Nikkei 225 MLP Regression Evaluation\n",
    "features_n225 = n225_df.drop(columns=['Adj Close']).columns.tolist()\n",
    "mse_scores_n225, mae_scores_n225, r2_scores_n225 = train_and_evaluate_mlp(n225_df, features_n225, 'Adj Close')\n",
    "print(\"MLP MSE scores for Nikkei 225:\", mse_scores_n225)\n",
    "print(\"MLP MAE scores for Nikkei 225:\", mae_scores_n225)\n",
    "print(\"MLP R^2 scores for Nikkei 225:\", r2_scores_n225)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2718d58f-306e-4e60-a4e4-80606a723158",
   "metadata": {},
   "source": [
    "# Linear Regression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 180,
   "id": "26012161-8de4-423d-9a9f-58f02c1cab47",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Linear Regression MSE scores for S&P 500: [3.63331223e-06 5.31269452e-06 6.06246577e-06 3.88815077e-06\n",
      " 4.02913073e-06 6.61086759e-06 3.96639802e-06 3.86517215e-06\n",
      " 5.53912842e-06 3.89368101e-06]\n",
      "Linear Regression MAE scores for S&P 500: [0.00131566 0.00141875 0.00140675 0.00125644 0.00126327 0.0014685\n",
      " 0.0013337  0.00128903 0.00147208 0.00126058]\n",
      "Linear Regression R^2 scores for S&P 500: [0.99992858 0.99990098 0.99988556 0.99992729 0.99993494 0.99987506\n",
      " 0.99992389 0.99991496 0.9999104  0.99992533]\n",
      "Linear Regression MSE scores for NASDAQ: [4.15137538e-06 4.55613574e-06 7.78782162e-06 5.03217003e-06\n",
      " 4.05349458e-06 7.68477990e-06 4.01101693e-06 4.13099086e-06\n",
      " 5.29954407e-06 4.69011206e-06]\n",
      "Linear Regression MAE scores for NASDAQ: [0.00119725 0.00129575 0.00136868 0.00124515 0.00123328 0.00143361\n",
      " 0.00124203 0.00123148 0.00133737 0.00116923]\n",
      "Linear Regression R^2 scores for NASDAQ: [0.9999282  0.99992353 0.99987214 0.99991542 0.99994435 0.99987271\n",
      " 0.99993048 0.99991845 0.9999263  0.99992262]\n",
      "Linear Regression MSE scores for Nikkei 225: [6.85324773e-06 5.95683346e-06 7.01102184e-06 1.04638036e-05\n",
      " 1.00604316e-05 7.97728779e-06 6.28599422e-06 5.78882962e-06\n",
      " 7.34470835e-06 7.43865848e-06]\n",
      "Linear Regression MAE scores for Nikkei 225: [0.00189036 0.00175807 0.00184287 0.00200279 0.00207669 0.00183688\n",
      " 0.0017658  0.00173844 0.00196159 0.0020077 ]\n",
      "Linear Regression R^2 scores for Nikkei 225: [0.99989228 0.99991435 0.99989493 0.99984273 0.99985914 0.99986908\n",
      " 0.99989909 0.99991398 0.99989215 0.99988431]\n"
     ]
    }
   ],
   "source": [
    "# Initialize the Linear Regression model\n",
    "lr_regressor = LinearRegression()\n",
    "\n",
    "def train_and_evaluate_lr(df, features, target):\n",
    "    # Extract features and target from the dataframe\n",
    "    X = df[features]\n",
    "    y = df[target]\n",
    "    \n",
    "    # Perform 10-fold cross-validation\n",
    "    mse_scores = cross_val_score(lr_regressor, X, y, cv=kf, scoring=mse_scorer)\n",
    "    mae_scores = cross_val_score(lr_regressor, X, y, cv=kf, scoring=mae_scorer)\n",
    "    r2_scores = cross_val_score(lr_regressor, X, y, cv=kf, scoring=r2_scorer)\n",
    "    \n",
    "    return -mse_scores, -mae_scores, r2_scores\n",
    "\n",
    "# S&P 500 Linear Regression Evaluation\n",
    "features_gspc = gspc_df.drop(columns=['Adj Close']).columns.tolist()\n",
    "mse_scores_gspc, mae_scores_gspc, r2_scores_gspc = train_and_evaluate_lr(gspc_df, features_gspc, 'Adj Close')\n",
    "print(\"Linear Regression MSE scores for S&P 500:\", mse_scores_gspc)\n",
    "print(\"Linear Regression MAE scores for S&P 500:\", mae_scores_gspc)\n",
    "print(\"Linear Regression R^2 scores for S&P 500:\", r2_scores_gspc)\n",
    "\n",
    "# NASDAQ Linear Regression Evaluation\n",
    "features_ixic = ixic_df.drop(columns=['Adj Close']).columns.tolist()\n",
    "mse_scores_ixic, mae_scores_ixic, r2_scores_ixic = train_and_evaluate_lr(ixic_df, features_ixic, 'Adj Close')\n",
    "print(\"Linear Regression MSE scores for NASDAQ:\", mse_scores_ixic)\n",
    "print(\"Linear Regression MAE scores for NASDAQ:\", mae_scores_ixic)\n",
    "print(\"Linear Regression R^2 scores for NASDAQ:\", r2_scores_ixic)\n",
    "\n",
    "# Nikkei 225 Linear Regression Evaluation\n",
    "features_n225 = n225_df.drop(columns=['Adj Close']).columns.tolist()\n",
    "mse_scores_n225, mae_scores_n225, r2_scores_n225 = train_and_evaluate_lr(n225_df, features_n225, 'Adj Close')\n",
    "print(\"Linear Regression MSE scores for Nikkei 225:\", mse_scores_n225)\n",
    "print(\"Linear Regression MAE scores for Nikkei 225:\", mae_scores_n225)\n",
    "print(\"Linear Regression R^2 scores for Nikkei 225:\", r2_scores_n225)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8c92f78d-a052-41e8-80ac-6f4a59150778",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}