Skip to content

Commit

Permalink
Expose all data from the API. Follow the protocol doc.
Browse files Browse the repository at this point in the history
  • Loading branch information
@bolner
bolner committed Aug 13, 2020
1 parent 4fea06a commit 08b8311
Show file tree
Hide file tree
Showing 5 changed files with 251 additions and 66 deletions.
19 changes: 13 additions & 6 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -329,8 +329,8 @@ $result3 = $connection3->Query("...");
| --- | --- |
| <strong>__construct</strong> | Create a new connection to a MonetDB database. <br><br><strong>@param</strong> <em>string</em> <strong>$host</strong> : The host of the database. Use '127.0.0.1' if the DB is on the same machine.<br><strong>@param</strong> <em>int</em> <strong>$port</strong> : The port of the database. For MonetDB this is usually 50000.<br><strong>@param</strong> <em>string</em> <strong>$user</strong> : The user name.<br><strong>@param</strong> <em>string</em> <strong>$password</strong> : The password of the user.<br><strong>@param</strong> <em>string</em> <strong>$database</strong> : The name of the database to connect to. Don't forget to release and start it.<br><strong>@param</strong> <em>string</em> <strong>$saltedHashAlgo</strong> <em>= "SHA1"</em> : Optional. The preferred hash algorithm to be used for exchanging the password. It has to be supported by both the server and PHP. This is only used for the salted hashing. Another stronger algorithm is used first (usually SHA512).<br><strong>@param</strong> <em>bool</em> <strong>$syncTimeZone</strong> <em>= true</em> : If true, then tells the clients time zone offset to the server, which will convert all timestamps is case there's a difference. If false, then the timestamps will end up on the server unmodified.<br><strong>@param</strong> <em>int</em> <strong>$maxReplySize</strong> <em>= 200</em> : The maximal number of tuples returned in a response. A higher value results in smaller number of memory allocations and string operations, but also in higher memory footprint. |
| <strong>Close</strong> | Close the connection |
| <strong>Query</strong> | Execute an SQL query and return its response. For 'select' queries the response can be iterated using a 'foreach' statement. You can pass an array as second parameter to execute the query as prepared statement, where the array contains the parameter values. SECURITY WARNING: For prepared statements in MonetDB, the parameter values are passed in a regular 'EXECUTE' command, using escaping. Therefore the same security considerations apply here as for using the Connection->Escape(...) method. Please read the comments for that method. <br><br><strong>@param</strong> <em>string</em> <strong>$sql</strong><br><strong>@param</strong> <em>array</em> <strong>$params</strong> <em>= null</em> : An optional array for prepared statement parameters. If not provided (or null), then a normal query is executed, instead of a prepared statement. The parameter values will retain their PHP type if possible. The following values won't be converted to string: null, true, false and numeric values.<br><strong>@return</strong> <em>Response</em> |
| <strong>QueryFirst</strong> | Execute an SQL query and return only the first row as an associative array. If there is more data on the stream, then discard all. Returns null if the query has empty result. You can pass an array as second parameter to execute the query as prepared statement, where the array contains the parameter values. <br><br><strong>@param</strong> <em>string</em> <strong>$sql</strong><br><strong>@param</strong> <em>array</em> <strong>$params</strong> <em>= null</em> : An optional array for prepared statement parameters. If not provided (or null), then a normal query is executed, instead of a prepared statement. See the 'Query' method for more information about the parameter values.<br><strong>@return</strong> <em>string[] -or- null</em> |
| <strong>Query</strong> | Execute an SQL query and return its response. For 'select' queries the response can be iterated using a 'foreach' statement. You can pass an array as second parameter to execute the query as prepared statement, where the array contains the parameter values. SECURITY WARNING: For prepared statements in MonetDB, the parameter values are passed in a regular 'EXECUTE' command, using escaping. Therefore the same security considerations apply here as for using the Connection->Escape(...) method. Please read the comments for that method. <br><br><strong>@param</strong> <em>string</em> <strong>$sql</strong><br><strong>@param</strong> <em>array</em> <strong>$params</strong> <em>= null</em> : An optional array for prepared statement parameters. If not provided (or null), then a normal query is executed instead of a prepared statement. The parameter values will retain their PHP type if possible. The following values won't be converted to string: null, true, false and numeric values.<br><strong>@return</strong> <em>Response</em> |
| <strong>QueryFirst</strong> | Execute an SQL query and return only the first row as an associative array. If there is more data on the stream, then discard all. Returns null if the query has empty result. You can pass an array as second parameter to execute the query as prepared statement, where the array contains the parameter values. <br><br><strong>@param</strong> <em>string</em> <strong>$sql</strong><br><strong>@param</strong> <em>array</em> <strong>$params</strong> <em>= null</em> : An optional array for prepared statement parameters. If not provided (or null), then a normal query is executed instead of a prepared statement. See the 'Query' method for more information about the parameter values.<br><strong>@return</strong> <em>string[] -or- null</em> |
| <strong>Command</strong> | Send a 'command' to MonetDB. Commands are used for configuring the database, for example setting the maximal response size, or for requesting unread parts of a query response ('export').<br><br><strong>@param</strong> <em>string</em> <strong>$command</strong><br><strong>@param</strong> <em>bool</em> <strong>$noResponse</strong> <em>= true</em> : If true, then returns NULL and makes no read to the underlying socket.<br><strong>@return</strong> <em>Response -or- null</em> |
| <strong>Escape</strong> | Escape a string value, to be inserted into a query, inside single quotes. The following characters are escaped by this method: backslash, single quote, carriage return, line feed, tabulator, null character, CTRL+Z. As a security measure this library forces the use of multi-byte support and UTF-8 encoding, which is also used by MonetDB, avoiding the SQL-injection attacks, which play with differences between character encodings. <br><br><strong>@param</strong> <em>string</em> <strong>$value</strong><br><strong>@return</strong> <em>string</em> |
| <strong>ClearPsCache</strong> | Clears the in-memory cache of prepared statements. This is called automatically when an error is received from MonetDB, because that also purges the prepared statements and all session state in this case. |
Expand Down Expand Up @@ -361,13 +361,20 @@ $result3 = $connection3->Query("...");
| --- | --- |
| <strong>GetQueryType</strong> | Returns a short string which identifies the type of the query.<br><br><strong>@return</strong> <em>string</em> |
| <strong>GetDescription</strong> | Returns a user-friendly text which describes the effect of the query.<br><br><strong>@return</strong> <em>string</em> |
| <strong>GetExecutionTime</strong> | The time the server spent on executing the query. In milliseconds.<br><br><strong>@return</strong> <em>float -or- null</em> |
| <strong>GetQueryParsingTime</strong> | The time it took to parse and optimize the query. In milliseconds.<br><br><strong>@return</strong> <em>float -or- null</em> |
| <strong>GetQueryTime</strong> | The time the server spent on executing the query. In milliseconds.<br><br><strong>@return</strong> <em>float -or- null</em> |
| <strong>GetSqlOptimizerTime</strong> | SQL optimizer time in milliseconds.<br><br><strong>@return</strong> <em>float -or- null</em> |
| <strong>GetMalOptimizerTime</strong> | MAL optimizer time in milliseconds.<br><br><strong>@return</strong> <em>float -or- null</em> |
| <strong>GetAffectedRows</strong> | The number of rows updated or inserted.<br><br><strong>@return</strong> <em>integer -or- null</em> |
| <strong>GetRowCount</strong> | The number of rows in the response.<br><br><strong>@return</strong> <em>integer -or- null</em> |
| <strong>GetTotalRowCount</strong> | The total number of rows in the result set. This includes those rows too, which are not in the current response.<br><br><strong>@return</strong> <em>integer -or- null</em> |
| <strong>GetAsText</strong> | Get a description of the status response in a human-readable format.<br><br><strong>@return</strong> <em>string</em> |
| <strong>GetPreparedStatementID</strong> | Get the ID of a created prepared statement. This ID can be used in an 'EXECUTE' statement, but only in the same session.<br><br><strong>@return</strong> <em>integer -or- null</em> |
| <strong>GetQueryID</strong> | Returns the ID of the query response that is returned in the result set.<br><br><strong>@return</strong> <em>integer -or- null</em> |
| <strong>GetResultID</strong> | Returns the ID of the result set that is returned for a query. It is stored on the server for this session, and parts of it can be queried using the "export" command.<br><br><strong>@return</strong> <em>integer -or- null</em> |
| <strong>GetAutoCommitState</strong> | Available after "start transaction", "commit" or "rollback". Tells whether the current session is in auto-commit mode or not.<br><br><strong>@return</strong> <em>boolean -or- null</em> |
| <strong>GetRowCount</strong> | The number of rows (tuples) in the current response only.<br><br><strong>@return</strong> <em>integer -or- null</em> |
| <strong>GetColumnCount</strong> | Column count. If the response contains tabular data, then this tells the number of columns.<br><br><strong>@return</strong> <em>integer -or- null</em> |
| <strong>GetQueryID</strong> | Query ID. A global ID which is also used in functions such as sys.querylog_catalog().<br><br><strong>@return</strong> <em>integer -or- null</em> |
| <strong>GetLastInsertID</strong> | The last automatically generated ID by an insert statement. (Usually auto_increment) NULL if none.<br><br><strong>@return</strong> <em>integer -or- null</em> |
| <strong>GetExportOffset</strong> | The index (offset) of the first row in a block response. (For an "export" command.)<br><br><strong>@return</strong> <em>integer -or- null</em> |

<hr><br>

Expand Down
35 changes: 24 additions & 11 deletions protocol_doc/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -151,9 +151,11 @@ password hashing, then the formula for getting the hash string is the following:
Where the hash functions output hexadecimal values. After the client calculated the hash,
it sends it in a message like the following:

LIT:monetdb:{SHA1}b8cb82cca07f379e25e99262e3b4b70054546136:sql:myDatabase:
LIT:monetdb:{SHA1}b8cb82cca07f379e25e99262e3b4b70054546136:sql:myDatabase:\n

Which is also separated by colons and the meanings of the values are:
The `\n` at the end means a line feed character. (It seems to work well without that too,
but mclient puts a newline there.) The line consists of colon-separated
values with the following meanings:

| Value | Description |
| --- | --- |
Expand All @@ -170,7 +172,7 @@ Optionally the client can allow the server the send/receive file
transfer requests to/from the client, by adding a `FILETRANS` at the
end, the following way:

LIT:monetdb:{SHA1}b8cb82cca07f379e25e99262e3b4b70054546136:sql:myDatabase:FILETRANS:
LIT:monetdb:{SHA1}b8cb82cca07f379e25e99262e3b4b70054546136:sql:myDatabase:FILETRANS:\n

## 3.1. Possible responses to an authentication request

Expand All @@ -194,8 +196,7 @@ After the client has sent the hashed password to the server, it can receive 3 ki
## 3.2. The Merovingian redirect

The `Merovingian redirect` can be a real redirect, when the client is asked for connecting
to a new host and port, but this case is not well documented and depend of special server
configurations, therefore it will be ignored in this document.
to a new host and port. For this see an example in section [Redirect](#51-redirect---).

A more common case of the `Merovingian redirect` is a request for the repetition of the authentication
process. It happens in the existing TCP connection. No new connections are created. This repetition is
Expand Down Expand Up @@ -289,12 +290,24 @@ if the field count is less than expected.

## 5.1. Redirect - **^**

This response has been discussed already in chapter [The Merovingian redirect](#32-the-merovingian-redirect).
Redirect messages always start with the `^` (caret) character. An example response:
Redirect messages always start with the `^` (caret) character.
This can be a real redirect, which instructs the client to close the current
connection and open another one on a specific host/port. Example:

^mapi:merovingian://proxy?database=myDatabase
^mapi:monetdb://localhost:50001/test?lang=sql&user=monetdb

| Sample value | Description |
| --- | --- |
| ^mapi:monetdb:// | This prefix identifies the redirect type |
| localhost | Host name or IP address. (It can be IPv6) |
| 50001 | Port. |
| test | Database name. |
| sql | Query language to request during the authentication. |
| monetdb | User name to specify during the authentication. |

?? Investigate if there are different kinds of redirects.
Or it can mean a [Merovingian redirect](#32-the-merovingian-redirect). Example:

^mapi:merovingian://proxy?database=myDatabase

## 5.2. Query response - **&**

Expand Down Expand Up @@ -425,9 +438,9 @@ The first line of the response consists of 5 space-separated values:
| --- | --- | --- |
| 0 | &5 | Identifies the response type. (Prepared statement creation) |
| 1 | 15 | The ID of the created prepared statement. This can be used in an `EXECUTE` statement. |
| 2 | 4 | Row count |
| 2 | 4 | Total row count in the result set. |
| 3 | 6 | Column count |
| 4 | 4 | Tuple count |
| 4 | 4 | Row count in current response only. |

The original query requested only 3 columns, but this response returned 4 data rows, as the
last one is for the `?` placeholder. The additional type information of the
Expand Down
11 changes: 3 additions & 8 deletions src/Connection.php
View file
Original file line number Diff line number Diff line change
Expand Up @@ -207,19 +207,14 @@ private function Authenticate() {
$pwHash = $challenge->HashPassword($this->password, $this->saltedHashAlgo);
$upperSaltHash = strtoupper($this->saltedHashAlgo);

$this->Write("LIT:{$this->user}:{{$upperSaltHash}}{$pwHash}:sql:{$this->database}:");
$this->Write("LIT:{$this->user}:{{$upperSaltHash}}{$pwHash}:sql:{$this->database}:\n");

$this->inputStream->LoadNextResponse();
$inputStream = $this->inputStream->ReadNextLine();
if (InputStream::IsResponse($inputStream, InputStream::MSG_REDIRECT, "mapi:merovingian:")) {
/*
Only the main process of MonetDB opens ports to listen on, and it spawns
separate sub-processes for each database.
The main process acts as a proxy and it forwards the queries to the
processes of the databases. For security reasons the user has to
authenticate not just at the main process, but also at the
sub-process too. This repetition of the authentication process is
called a "Merovingian redirect".
See doc:
https://github.com/MonetDB/MonetDB-PHP/tree/master/protocol_doc#32-the-merovingian-redirect
*/
continue;
}
Expand Down
8 changes: 4 additions & 4 deletions src/Response.php
View file
Original file line number Diff line number Diff line change
Expand Up @@ -192,8 +192,8 @@ private function Parse() {
If not all rows did fit into the window,
request the next batch.
*/
$diff = $this->queryStatusRecord->GetRowCount() - $this->rowCount;
$queryID = $this->queryStatusRecord->GetQueryID();
$diff = $this->queryStatusRecord->GetTotalRowCount() - $this->rowCount;
$queryID = $this->queryStatusRecord->GetResultID();

if ($diff > 0) {
$size = min($diff, $this->connection->GetMaxReplySize());
Expand Down Expand Up @@ -237,7 +237,7 @@ private function Parse() {
responses just get ignored.
*/
if ($this->queryStatusRecord !== null) {
if ($this->queryStatusRecord->GetQueryID() != $status->GetQueryID()) {
if ($this->queryStatusRecord->GetResultID() != $status->GetResultID()) {
$this->ignoreTuples = true;
}
}
Expand Down Expand Up @@ -291,7 +291,7 @@ private function Parse() {
continue;
}

if ($this->queryStatusRecord->GetQueryID() != $status->GetQueryID()) {
if ($this->queryStatusRecord->GetResultID() != $status->GetResultID()) {
$this->ignoreTuples = true;
continue;
}
Expand Down
Loading

0 comments on commit 08b8311

Please sign in to comment.