Add is valid sql descriptor

[Sifr-un]

Implementing a SQL validator for LLM responses as a new descriptor.
Resolves #1321

[DimaAmega]

Hi @Rayryu

You have some CI failed jobs, please check it

[Sifr-un]

DimaAmega

I've pushed a typo fix.

[Sifr-un]

I see that the minimal requirements CI step has failed. I've now added sqlvalidator to the requirements file 👌

[DimaAmega]

Let's move this import to the beginning of the is_valid_sql function

[Sifr-un]

Changes made. What was causing the issue?

[DimaAmega]

Now all the UI checks are being passed in CI 😊

It's a bit tricky: other fix is to add llm extra to install step in these tests but llm extra at the same time is kinda overkill and unnecessary

[DimaAmega]

Let's consider the following sql

SELECT * FROM your_table WHERE your_column LIKE '%;%';

or

SELECT CONCAT('Hello', '; ', 'World') AS greeting_with_semicolon;

we will get invalid in your implementation but it is actually valid sql

Do we actually need to split by ;?

[Sifr-un]

The split by ; is a bit of a stretch since the column could contain multiple SQL queries. However, we can stick to the basic usage and treat the column values as single queries.
How do we view the scope of this feature?

[mike0sv]

Hi! To fix mypy you need to add ignore block to setup.cfg in repo root. You can check out mypy section for examples

[Sifr-un]

Hi! To fix mypy you need to add ignore block to setup.cfg in repo root. You can check out mypy section for examples

Thank you @mike0sv for chiming in. I've added a sqlvalidator block in setup.cfg.

[emeli-dral]

Hi @Rayryu,

It looks like we're almost there!

Could you please add the new descriptor to the list of descriptors under the TextEvals section on the documentation page? The file that needs to be updated is located here: all-metrics.md#text-evals.

Once that’s done, we’ll be all set to proceed with the merge!

[emeli-dral]

Hi @Sifr-un ,
do you need some help with adding IsValidSQL to the list of descriptors under the TextEvals section all-metrics.md#text-evals?