Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Aggregate version declaration to root pom #28837

Merged
merged 5 commits into from
Oct 23, 2023
Merged

Conversation

terrymanu
Copy link
Member

No description provided.

@terrymanu terrymanu added this to the 5.4.2 milestone Oct 22, 2023
@terrymanu terrymanu added the type: dependencies Pull requests that update a dependency file label Oct 22, 2023
@terrymanu terrymanu changed the title Aggregate version to root pom Aggregate version declaration to root pom Oct 22, 2023
@terrymanu terrymanu marked this pull request as draft October 22, 2023 13:11
@sandynz
Copy link
Contributor

sandynz commented Oct 23, 2023

Pipeline MySQL E2E error investigation:

Problem

MySQLMigrationGeneralE2EIT as an example, it need to create ShardingSphereDataSource for avoiding some issue of proxy, then it'll execute sql EXPORT DATABASE CONFIGURATION; in proxy. And error occur from MySQL JDBC driver:

Caused by: java.sql.SQLException: Statement.executeQuery() cannot issue statements that do not produce result sets.
	at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:129)
	at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:97)
	at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:89)
	at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:63)
	at com.mysql.cj.jdbc.StatementImpl.executeQuery(StatementImpl.java:1134)
	at com.zaxxer.hikari.pool.ProxyStatement.executeQuery(ProxyStatement.java:110)
	at com.zaxxer.hikari.pool.HikariProxyStatement.executeQuery(HikariProxyStatement.java)
	at org.apache.shardingsphere.test.e2e.data.pipeline.cases.PipelineContainerComposer.queryForListWithLog(PipelineContainerComposer.java:424)

It exists from mysql-connector-java 8.0.27:

When Statement.executeQuery() was called, Connector/J's check for whether a statement would return results was inadequate, so that sometimes appropriate statements were rejected (for examples, SELECT statements starting with a WITH clause, statements preceded by consecutive comments, and so on) and, at other times, inappropriate statements were executed (for example, DO statements), resulting in various kinds of errors. With this fix, Connector/J performs more accurate checks by looking at the statement keywords and the context, as well as handling properly different corner cases.

Related commit.

It means extended query SQL in proxy could not be executed via mysql-connector-java new version any more.

Workaround

Workaround 1: Still use version 5.1.49 of mysql-connector-java in shardingsphere-test-e2e-operation, just in E2E module. It doesn't affect project's dependency and it's easy to do.

Workaround 2: Read related nodes value from ZooKeeper and combine them as dataSources and rules. It need much code to do it, construct DatabaseRulePersistService, DataSourceUnitPersistService, PersistRepository and etc.

@sandynz
Copy link
Contributor

sandynz commented Oct 23, 2023

mysql-connection-j 8.0.33 QueryInfo.java
https://github.com/mysql/mysql-connector-j/blob/8.0.33/src/main/core-api/java/com/mysql/cj/QueryInfo.java#L668

    /**
     * Returns the type of return that can be expected from executing the given query. This operation does not take into consideration the multiplicity
     * of queries in the specified SQL.
     * 
     * @param sql
     *            the query to check
     * @param noBackslashEscapes
     *            whether backslash escapes are disabled or not
     * @return the return type that can be expected from the given query, one of the elements of {@link QueryReturnType}.
     */
    public static QueryReturnType getQueryReturnType(String sql, boolean noBackslashEscapes) {
        /*
         * Statements that return results:
         * - ANALYZE; CHECK/CHECKSUM; DESC/DESCRIBE; EXPLAIN; HELP; OPTIMIZE; REPAIR; SELECT; SHOW; TABLE; VALUES; WITH ... SELECT|TABLE|VALUES ...; XA RECOVER;
         * 
         * Statements that may return results:
         * - CALL; EXECUTE;
         * 
         * Statements that do not return results:
         * - ALTER; BINLOG; CACHE; CHANGE; CLONE; COMMIT; CREATE; DEALLOCATE; DELETE; DO; DROP; FLUSH; GET; GRANT; HANDLER; IMPORT; INSERT; INSTALL; KILL; LOAD;
         * - LOCK; PREPARE; PURGE; RELEASE; RENAME; REPLACE; RESET; RESIGNAL; RESTART; REVOKE; ROLLBACK; SAVEPOINT; SET; SHUTDOWN; SIGNAL; START; STOP;
         * - TRUNCATE; UNINSTALL; UNLOCK; UPDATE; USE; WITH ... DELETE|UPDATE ...; XA [!RECOVER];
         */
        int statementKeywordPos = indexOfStatementKeyword(sql, noBackslashEscapes);
        if (statementKeywordPos == -1) {
            return QueryReturnType.NONE;
        }
        char firstStatementChar = Character.toUpperCase(sql.charAt(statementKeywordPos));
        if (firstStatementChar == 'A' && StringUtils.startsWithIgnoreCaseAndWs(sql, "ANALYZE", statementKeywordPos)) {
            return QueryReturnType.PRODUCES_RESULT_SET;
        } else if (firstStatementChar == 'C' && StringUtils.startsWithIgnoreCaseAndWs(sql, "CALL", statementKeywordPos)) {
            return QueryReturnType.MAY_PRODUCE_RESULT_SET;
        } else if (firstStatementChar == 'C' && StringUtils.startsWithIgnoreCaseAndWs(sql, "CHECK", statementKeywordPos)) { // Also matches "CHECKSUM".
            return QueryReturnType.PRODUCES_RESULT_SET;
        } else if (firstStatementChar == 'D' && StringUtils.startsWithIgnoreCaseAndWs(sql, "DESC", statementKeywordPos)) { // Also matches "DESCRIBE".
            return QueryReturnType.PRODUCES_RESULT_SET;
        } else if (firstStatementChar == 'E' && StringUtils.startsWithIgnoreCaseAndWs(sql, "EXPLAIN", statementKeywordPos)) {
            return QueryReturnType.PRODUCES_RESULT_SET;
        } else if (firstStatementChar == 'E' && StringUtils.startsWithIgnoreCaseAndWs(sql, "EXECUTE", statementKeywordPos)) {
            return QueryReturnType.MAY_PRODUCE_RESULT_SET;
        } else if (firstStatementChar == 'H' && StringUtils.startsWithIgnoreCaseAndWs(sql, "HELP", statementKeywordPos)) {
            return QueryReturnType.PRODUCES_RESULT_SET;
        } else if (firstStatementChar == 'O' && StringUtils.startsWithIgnoreCaseAndWs(sql, "OPTIMIZE", statementKeywordPos)) {
            return QueryReturnType.PRODUCES_RESULT_SET;
        } else if (firstStatementChar == 'R' && StringUtils.startsWithIgnoreCaseAndWs(sql, "REPAIR", statementKeywordPos)) {
            return QueryReturnType.PRODUCES_RESULT_SET;
        } else if (firstStatementChar == 'S' && (StringUtils.startsWithIgnoreCaseAndWs(sql, "SELECT", statementKeywordPos)
                || StringUtils.startsWithIgnoreCaseAndWs(sql, "SHOW", statementKeywordPos))) {
            return QueryReturnType.PRODUCES_RESULT_SET;
        } else if (firstStatementChar == 'T' && StringUtils.startsWithIgnoreCaseAndWs(sql, "TABLE", statementKeywordPos)) {
            return QueryReturnType.PRODUCES_RESULT_SET;
        } else if (firstStatementChar == 'V' && StringUtils.startsWithIgnoreCaseAndWs(sql, "VALUES", statementKeywordPos)) {
            return QueryReturnType.PRODUCES_RESULT_SET;
        } else if (firstStatementChar == 'W' && StringUtils.startsWithIgnoreCaseAndWs(sql, "WITH", statementKeywordPos)) {
            String context = getContextForWithStatement(sql, noBackslashEscapes);
            if ("SELECT".equalsIgnoreCase(context) || "TABLE".equalsIgnoreCase(context) || "VALUES".equalsIgnoreCase(context)) {
                return QueryReturnType.PRODUCES_RESULT_SET;
            } else if ("UPDATE".equalsIgnoreCase(context) || "DELETE".equalsIgnoreCase(context)) {
                return QueryReturnType.DOES_NOT_PRODUCE_RESULT_SET;
            } else {
                return QueryReturnType.MAY_PRODUCE_RESULT_SET;
            }
        } else if (firstStatementChar == 'X' && StringUtils.indexOfIgnoreCase(statementKeywordPos, sql, new String[] { "XA", "RECOVER" }, OPENING_MARKERS,
                CLOSING_MARKERS, noBackslashEscapes ? SearchMode.__MRK_COM_MYM_HNT_WS : SearchMode.__FULL) == statementKeywordPos) {
            return QueryReturnType.PRODUCES_RESULT_SET;
        }
        return QueryReturnType.DOES_NOT_PRODUCE_RESULT_SET;
    }

@terrymanu terrymanu marked this pull request as ready for review October 23, 2023 11:07
@sandynz sandynz merged commit 9b80669 into apache:master Oct 23, 2023
25 of 26 checks passed
@terrymanu terrymanu deleted the dev branch October 24, 2023 01:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type: dependencies Pull requests that update a dependency file
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants