Stop deepeval from storing sensitive information in dotenv files.

[thomend]

Is your feature request related to a problem? Please describe.
When using deepeval with Azure OpenAI:

deepeval set-azure-openai --openai-endpoint=<endpoint> \
    --openai-api-key=<api_key> \
    --deployment-name=<deployment_name> \
    --openai-api-version=<openai_api_version> \
    --model-version=<model_version>

credentials get stored in clear text as dotenv file (.deepeval). There are multiple problems with that:

• The user is not getting informed about this.
• Data is stored in clear text. There is a risk that the file get's accidentally committed to a remote repo. Also, the bigger risk is that sensitive information is stored in a file system instead of being simply in the memory of the current process.
• Data is not cleared when using deepeval unset-azure-openai

Describe the solution you'd like
One could you use a context manager approach where all calls to deepeval are made within it's context using Azure and once it exits the file gets cleaned up.
Alternative one could have the option to set it in the LLMTestCase or somewhere else as a attribute to a class or set it as a global variable which can be accessed by the API calls within deepeval.

Describe alternatives you've considered
A partial mitigation is of course to add .deepeval to the gitignore (for people using deepeval in a git repo) but in my opinion this just deals with the symptom rather than the underlying issue.

In case this feature makes sense I would also consider contributing to it directly if that helps :)

Best regards and many thanks
Thomas

[penguine-ip]

Hey @thomend would love a PR! It was written to file in the beginning because sometimes the variables are not properly preserved across sessions... would love a fix that could make the azure thing permanent while being secure

[thomend]

Hi @penguine-ip thanks for the quick reply. Interesting, do you have a reproducible example of this or was there a past issue open on this regarding not preserving across sessions? Also what exactly do you mean with a "session" - a http session?

I propose to keep the existing behavior (e.g. one can set it via the CLI + introduce the new option to set it as parameter/global variable in the current python process: Would you agree?

(asking those questions, so that I can get started on this issue ;) )

[penguine-ip]

Hey @thomend by session I mean for example quitting the virtual environment, restarting your laptop, and in some cases updating deepeval (although I'm not sure in which circumstances this occurs). So right now we're just writing to file to make it permanent. I think two options will be great - a default one (the current one) and the one you suggested which for more data sensitive users they will have the option to configure. Thanks!