Import error when running recently added 'discrete_cql_online.py' from examples/cql

[spallaccini]

Hi, when attempting to run the example on the Discrete CQL algorithm (online) discrete_cql_online.py I experienced an import issue for the DiscreteCQLLoss class from torchrl/objectives/utils.py.

Issue can be reproduced both running directly the script or isolating the import in a jupyter notebook as below:
from torchrl.objectives import CQLLoss, DiscreteCQLLoss, SoftUpdate

ImportError Traceback (most recent call last)
Cell In[1], line 1
----> 1 from torchrl.objectives import CQLLoss, DiscreteCQLLoss, SoftUpdate

ImportError: cannot import name 'DiscreteCQLLoss' from 'torchrl.objectives'

I tried installing both torchrl and torchrl-nighly; other than this tiny issue the library is working perfectly for me.

The CQLLoss class is correctly imported.

I found that the Discrete CQL had been recently added:
#1666

Thank you!

[vmoens]

can you print this

$ pip list | grep torchrl

You may have both torchrl and torchrl-nightly in your environment

[spallaccini]

Thanks Vincent for your prompt reply! Printout of the command gives only: torchrl-nightly 2023.10.25 I was actually careful about this and the issue first occurred on the latest stable torchrl version some days ago when attempting the Discrete CQL example. Do you need detail of packages in my env?

[vmoens]

Second likely explanation: you're running the code from the torchrl root of the repo, but your git clone isn't updated, and when you run the script it tries to get a module that is not in the local directory.
If you're running it from /path/to/rl try to move somewhere else (eg, /path/to/rl/examples/cql)

I tried to reprod locally but couldn't. I also checked and the class is present and accessible in the nightlies on pypi...

[spallaccini]

Classification: Confidential Actually I downloaded both utils.py and discrete_cql_online.py from git online and put them in a local folder prior to attempt running the example. I did not clone the whole repo. Anyway I now tried to clone the rl repo and run the script as you instructed - keeping the pypi torchrl-nightly installed. Result is the same, here is the printout (just pruned some paths): ***@***.***:~/TORCHRL/GITCLONE/rl/examples/cql$ python discrete_cql_online.py UserWarning: Failed to import torchrl C++ binaries. Some modules (eg, prioritized replay buffers) may not work with your installation. If you installed TorchRL from PyPI, please report the bug on TorchRL github. If you installed TorchRL locally and/or in development mode, check that you have all the required compiling packages. warnings.warn(EXTENSION_WARNING) Traceback (most recent call last): File ".../TORCHRL/GITCLONE/rl/examples/cql/discrete_cql_online.py", line 25, in <module> from utils import ( File ".../TORCHRL/GITCLONE/rl/examples/cql/utils.py", line 35, in <module> from torchrl.objectives import CQLLoss, DiscreteCQLLoss, SoftUpdate ImportError: cannot import name 'DiscreteCQLLoss' from 'torchrl.objectives' (.../lib/python3.11/site-packages/torchrl/objectives/__init__.py) ***@***.***:~/TORCHRL/GITCLONE/rl/examples/cql The highlighted user warning was also there before From: Vincent Moens ***@***.***> Sent: venerdì 1 dicembre 2023 16:06 To: pytorch/rl ***@***.***> Cc: Paolo Spallaccini ***@***.***>; Author ***@***.***> Subject: Re: [pytorch/rl] Import error when running recently added 'discrete_cql_online.py' from examples/cql (Issue #1725) [CAUTION: This Email is from outside the Organization. Unless you trust the sender, Don't click links or open attachments as it may be a Phishing email, which can steal your Information and compromise your Computer.] Second likely explanation: you're running the code from the torchrl root of the repo, but your git clone isn't updated, and when you run the script it tries to get a module that is not in the local directory. If you're running it from /path/to/rl try to move somewhere else (eg, /path/to/rl/examples/cql) I tried to reprod locally but couldn't. I also checked and the class is present and accessible in the nightlies on pypi... - Reply to this email directly, view it on GitHub<#1725 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AENWEPWPZS6FJHHMXMJ3XNDYHHW3VAVCNFSM6AAAAABAC5P6BWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMZWGI3TEOJTG4>. You are receiving this because you authored the thread.Message ID: ***@***.***> ::DISCLAIMER::

…

________________________________ The contents of this e-mail and any attachment(s) are confidential and intended for the named recipient(s) only. E-mail transmission is not guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or may contain viruses in transmission. The e mail and its contents (with or without referred errors) shall therefore not attach any liability on the originator or HCL or its affiliates. Views or opinions, if any, presented in this email are solely those of the author and may not necessarily reflect the views or opinions of HCL or its affiliates. Any form of reproduction, dissemination, copying, disclosure, modification, distribution and / or publication of this message without the prior written consent of authorized representative of HCL is strictly prohibited. If you have received this email in error please delete it and notify the sender immediately. Before opening any email and/or attachments, please check them for viruses and other defects.

________________________________

[vmoens]

Alright I think I figured out what is happening.
When looking at the nightlies, there are some versions that are missing so you probably have installed the latest compatible version (smth like the 20th of September).
I suspect your distro isn't listed there:
https://pypi.org/project/torchrl-nightly/2023.12.1/#files

The immediate solution for you is to clone torchrl and do a python setup.py install within the repo (make sure you have the ninja lib in your env, pip install ninja).
I will investigate why some distros are missing from the nightlies!

[vmoens]

Wheels have been updated and will now be pushed for all platforms
Thanks for reporting!