Unable to fetch data from YahooDownloader due a miss match in columns

[kuds]

Describe the bug
fetch_data() fails due to miss-match in the expected columns

ValueError: Length mismatch: Expected axis has 7 elements, new values have 8 elements

To Reproduce
Steps to reproduce the behavior:
When executing the stock fundamentals notebooks (link), the notebooks errors out when trying to rename the columns in the YahooDownloader.py file.

Expected behavior
To successfully download ticker data from Yahoo Finance

Screenshots

Google Colab Notebook Screenshot

YahooDownloader.py

Desktop (please complete the following information):
Google Colab
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=22.04
DISTRIB_CODENAME=jammy
DISTRIB_DESCRIPTION="Ubuntu 22.04.3 LTS"
PRETTY_NAME="Ubuntu 22.04.3 LTS"
NAME="Ubuntu"
VERSION_ID="22.04"
VERSION="22.04.3 LTS (Jammy Jellyfish)"
VERSION_CODENAME=jammy
ID=ubuntu
ID_LIKE=debian

Additional context
Python Version: 3.10.12
FinRL Version: 0.3.6
Is Cuda Available: False
Cuda Version: 12.4
Pandas Version: 2.2.3
Numpy Version: 1.26.4
Stable Baselines3 Version: 2.5.0a0
urllib3 Version: 1.26.20
yfinance Version: 0.2.51

[kuds]

One possible solution is to update the renaming columns to exclude the adjusted price column as that is not being returned in the data. Thoughts?

data_df.columns = [
  "date",
  "close",
  "high",
  "low",
  "open",
  "volumne",
  "tic",
]

[kuds]

After some further research of the yfinance library, it looks like the auto_adjust needs to be set to false to get the "Adj Close" column. Plus this returns the correct number of columns of 8. Let me know if you want me to create a pull request for this change.

[ZZZ-winner]

Hello, I also have this problem, which is the same every time I run the fetch_data() method. How did you solve it in the end? I tried the method you just sent, but it was still wrong

[kuds]

@ZZZ-winner

I had to update the fetch_data() method to include the auto_adjust parameter and rename the columns differently. Once those changes were in place, I re-installed the FinRL in Google Colab.

YahooDownloader.py

temp_df = yf.download(tic, start=self.start_date, end=self.end_date, proxy=proxy, auto_adjust=auto_adjust)

data_df.rename(columns={"Date": "date", 
                        "Adj Close": "adjcp",
                        "Close": "close",
                        "High": "high", 
                        "Low": "low", 
                        "Volume": "volumne", 
                        "Open": "open",
                        "tic":"tic"}, inplace=True)

After re-installing FinRL

See the associated pull request for my exact changes. Let me know if you have any further questions!