Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

In Power BI, filters for Cyrillic Unicode words are not working. #449

Open
luckyfs opened this issue Aug 23, 2024 · 1 comment
Open

In Power BI, filters for Cyrillic Unicode words are not working. #449

luckyfs opened this issue Aug 23, 2024 · 1 comment
Labels

Comments

@luckyfs
Copy link

luckyfs commented Aug 23, 2024

Prerequisites:

  1. Clickhouse on-premises, Ubuntu (project database or sandbox play.clickhouse.com (select * from actors))
  2. Power BI Desktop, Windows
  3. Clickhouse ODBC driver (Unicode)
  4. Clickhouse native driver for Power BI
  5. DirectQuery mode for the Power BI report

Steps to Reproduce:

  1. Open Power BI Desktop
  2. Connect to the Clickhouse database using the native connector
  3. Create a report that calculates a count of rows where the filter parameter is an attribute containing Unicode values (Cyrillic, Chinese, and others).
  4. Apply a filter with a Unicode value

Actual Result: Power BI Desktop throws an error: "OLE DB or ODBC error: [Expression.Error] We couldn't fold the expression to the data source. Please try a simpler expression."

Expected Result: The report successfully calculates the count of rows.

Log Analysis Findings:

  1. The Clickhouse native connector log does not register this error. It seems that the query does not reach the connector.
  2. The Power BI Desktop log shows that the query is generated. In the query, Unicode characters are encoded (example from the log, see the WHERE block): SELECT [t2].[SEGMENT_RENNA_ID] AS [c52],[t2].[SEGMENT_RENNA_TXT] AS [c53],[t2].[SEGMENT_RENNA_ID] AS [o0]#(lf)FROM [DICT_DIM_SKUMATCH] AS [t2]#(lf))#(lf) AS [t0]#(lf)WHERE #(lf)(#(lf)[c53] = N'\u041C\u0420\u0416'#(lf))#(lf)#(lf)GROUP BY [c52],[o0]#(lf)ORDER BY #(lf)
  3. ODBC trace logs show that when Unicode values are used, question marks appear, indicating that the ODBC driver cannot correctly interpret Unicode values.

Please help us resolve this issue, as this error essentially blocks the use of Power BI. I would like to remind you that 99% of reports in Power BI are created in Power BI Desktop and subsequently published to the server.

I have attached logs at all levels, screenshots, and a video of the system's behavior.

Some articles suggest checking the same connection to excel, but this is a completely different case, since excel always downloads all the data first, after which the user uses excel to filter the data. In other words, there are no direct requests from excel to the clickhouse database
Microsoft.Mashup.Container.NetFX45.22512.2024-08-15T15-48-23-346359.log
msmdsrv.26716.2024-08-15T15-48-23-324476.log
logs.txt
SQL.LOG
Microsoft.Mashup.Container.NetFX45.16788.2024-08-15T15-48-23-399625.log
Microsoft.Mashup.Container.NetFX45.29208.2024-08-15T15-48-23-339172.log
PBIDesktop.26112.2024-08-15T15-48-16-135078.log

Video
Screenshot_1

@luckyfs
Copy link
Author

luckyfs commented Aug 23, 2024

A fragment from SQL.LOG :
Microsoft.Mashu 924c-8eec EXIT SQLGetData with return code 0 (SQL_SUCCESS)
HSTMT 0x000001578954C750
UWORD 1
SWORD -8 <SQL_C_WCHAR>
PTR 0x000001578C172588 [ 32] "AS-(101) ???????"
SQLLEN 2048
SQLLEN * 0x0000002C309FD518 (32)

Pay attention to "?????"

@mshustov mshustov added the bug label Sep 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants