Document fulltext search (#484)

* Document fulltext search * Review updates --------- Co-authored-by: James Bayly <[email protected]>
subquery · Mar 7, 2024 · 265dd90 · 265dd90
1 parent 497dd5a
commit 265dd90
Show file tree

Hide file tree

Showing 3 changed files with 87 additions and 38 deletions.
diff --git a/docs/build/graph-migration.md b/docs/build/graph-migration.md
@@ -47,7 +47,7 @@ This is the recommended proccess that we use at SubQuery whenever we migrate pro
 
 ## GraphQL Schema
 
-Both SubGraphs and SubQuery projects use the same `schema.graphql` to define entities.
+Both SubGraphs and SubQuery projects use the same `schema.graphql` to define entities and includes both similar [scalar types](./graphql.md#supported-scalar-types) as well as [full text search](./graphql.md#full-text-search).
 
 Visit this [full documentation for `schema.graphql`](./graphql.md). **You can copy this file from your SubGraph to your SubQuery project in most cases.**
 
@@ -56,7 +56,6 @@ Notable differences include:
 - SubQuery does not have support for `Bytes` (use `String` instead) and `BigDecimal` (use `Float` instead).
 - SubQuery has the additional scalar types of `Float`, `Date`, and `JSON` (see [JSON type](./graphql.md#json-type)).
 - Comments are added to SubQuery Project GraphQL files using hashes (`#`).
-- SubQuery does not yet support full-text search.
 
 ## Manifest File
 
@@ -518,6 +517,7 @@ SubQuery does not support historical metadata querying. However `deployments` wi
 - SubQuery has a larger support for query pagination. You have the options of using `first` and `offset`, or `cursors` on `edges`.
   - Note that [cursor-based pagination](https://graphql.org/learn/pagination/#pagination-and-edges) is far more efficient compared to `first`/`offset`/`after` pagination
 - SubQuery supports [advanced aggregate functions](../run_publish/query/aggregate.md) to allow you to perform a calculation on a set of values during your query.
+- Full text search is implemented slightly differently, please review the [full text query docs](../run_publish/query/graphql.md#full-text-search).
 
 ## What's Next?
 

diff --git a/docs/build/graphql.md b/docs/build/graphql.md
@@ -77,7 +77,7 @@ All entites can be imported from the following directory after codegen:
 import { GraphQLEntity1, GraphQLEntity2 } from "../types";
 ```
 
-### Entities
+### IDs
 
 Each entity must define its required fields `id` with the type of `ID!`. It is used as the primary key and unique among all entities of the same type.
 
@@ -91,6 +91,32 @@ type Example @entity {
 }
 ```
 
+### Supported scalar types
+
+We currently support the following scalar types:
+
+- `ID` (supports [full text search](#full-text-search))
+- `Int`
+- `String` (supports [full text search](#full-text-search))
+- `BigInt`
+- `Float`
+- `Date`
+- `Boolean`
+- `<EntityName>` for nested relationship entities, you might use the defined entity's name as one of the fields. Please see in [Entity Relationships](graphql.md#entity-relationships).
+- `JSON` can alternatively store structured data, please see [JSON type](graphql.md#json-type)
+- `<EnumName>` types are a special kind of enumerated scalar that is restricted to a particular set of allowed values. Please see [Graphql Enum](https://graphql.org/learn/schema/#enumeration-types)
+
+### Naming Constraints
+
+You'll need to shorten the entity's name if you get an error such as the following when running the app:
+
+```shell
+subquery-notifications-postgres-1        | 2022-08-26 14:18:12.355 UTC [1922] ERROR:  constraint "bank_msg_multi_send_input_coins_bank_msg_multi_send_input_id_fk" for table "bank_msg_multi_send_input_coins" does not exist
+subquery-notifications-postgres-1        | 2022-08-26 14:18:12.355 UTC [1922] STATEMENT:  COMMENT ON CONSTRAINT bank_msg_multi_send_input_coins_bank_msg_multi_send_input_id_fkey ON "app"."bank_msg_multi_send_input_coins" IS E'@foreignFieldName coins'
+```
+
+SubQuery automatically generates Postgres identifiers for your entities. For example, if you have an entity named `BankMsgMultiSendInputCoins`, then an identifier `bank_msg_multi_send_input_coins_bank_msg_multi_send_input_id_fkey` will be automatically generated in Postgres. However, this identifier is 65 bytes, and Postgres doesn't support identifiers larger than 63 bytes. In this example, shortening the entity's name to `BankMultiSendInputCoins` will resolve the issue.
+
 ### Adding Documentation Strings to GraphQL Schema
 
 Documentation strings (also known as "doc strings") can be added to a GraphQL schema to provide human-readable descriptions of the schema's types and fields. This can be particularly useful for maintaining and understanding the schema over time, and for auto-generating API documentation.
@@ -124,61 +150,37 @@ In addition, when using GraphQL query Playground, these doc strings will automat
 
 ![image](../.vuepress/public/assets/img/build/schema_docstring.png)
 
-### Naming Constraints
-
-You'll need to shorten the entity's name if you get an error such as the following when running the app:
-
-```shell
-subquery-notifications-postgres-1        | 2022-08-26 14:18:12.355 UTC [1922] ERROR:  constraint "bank_msg_multi_send_input_coins_bank_msg_multi_send_input_id_fk" for table "bank_msg_multi_send_input_coins" does not exist
-subquery-notifications-postgres-1        | 2022-08-26 14:18:12.355 UTC [1922] STATEMENT:  COMMENT ON CONSTRAINT bank_msg_multi_send_input_coins_bank_msg_multi_send_input_id_fkey ON "app"."bank_msg_multi_send_input_coins" IS E'@foreignFieldName coins'
-```
-
-SubQuery automatically generates Postgres identifiers for your entities. For example, if you have an entity named `BankMsgMultiSendInputCoins`, then an identifier `bank_msg_multi_send_input_coins_bank_msg_multi_send_input_id_fkey` will be automatically generated in Postgres. However, this identifier is 65 bytes, and Postgres doesn't support identifiers larger than 63 bytes. In this example, shortening the entity's name to `BankMultiSendInputCoins` will resolve the issue.
-
-### Supported scalar types
-
-We currently support the following scalar types:
-
-- `ID`
-- `Int`
-- `String`
-- `BigInt`
-- `Float`
-- `Date`
-- `Boolean`
-- `<EntityName>` for nested relationship entities, you might use the defined entity's name as one of the fields. Please see in [Entity Relationships](graphql.md#entity-relationships).
-- `JSON` can alternatively store structured data, please see [JSON type](graphql.md#json-type)
-- `<EnumName>` types are a special kind of enumerated scalar that is restricted to a particular set of allowed values. Please see [Graphql Enum](https://graphql.org/learn/schema/#enumeration-types)
-
-## Indexing by non-primary-key field
+## Indexing
 
 To improve query performance, index an entity field simply by implementing the `@index` annotation on a non-primary-key field (you can also use [composite indexes](#composite-index)).
 
 `@index` annotations are not supported on any [JSON](graphql.md#json-type) object or a boolean field.
 
 By default, indexes are automatically added to foreign keys and for JSON fields in the database, but only to enhance query service performance.
 
+### Standard Indexes
+
+Assuming we knew this user's `email`, but we don't know the exact `id` value, rather than extract all users and then filtering by `email` we can add `@index` behind the `email` field. This makes querying much faster and we can additionally pass the `unique: true` to ensure uniqueness.
+
 Here is an example.
 
 ```graphql
 type User @entity {
   id: ID!
-  name: String! @index(unique: true) # unique can be set to true or false
+  email: String! @index(unique: true) # unique can be set to true or false
+  name: String!
   title: Title! # Indexes are automatically added to foreign key field
 }
 
 type Title @entity {
   id: ID!
-  name: String! @index(unique: true)
+  title: String! @index(unique: true)
 }
 ```
 
-Assuming we knew this user's name, but we don't know the exact id value, rather than extract all users and then filtering by name we can add `@index` behind the name field. This makes querying much faster and we can additionally pass the `unique: true` to ensure uniqueness.
-
 **If a field is not unique, the maximum result set size is 100**
 
-When code generation is run, this will automatically create a `getByName` under the `User` model, and the foreign key field `title` will create a `getByTitleId` method,
-which both can directly be accessed in the mapping function.
+When code generation is run, this will automatically create a `getByEmail` under the `User` model, and the foreign key field `title` will create a `getByTitleId` method, which both can directly be accessed in the mapping function.
 
 ```sql
 /* Prepare a record for title entity */
@@ -190,14 +192,14 @@ INSERT INTO titles (id, name) VALUES ('id_1', 'Captain')
 import { User } from "../types/models/User";
 import { Title } from "../types/models/Title";
 
-const jack = await User.getByName("Jack Sparrow");
+const jack = await User.getByEmail("[email protected]");
 
 const captainTitle = await Title.getByName("Captain");
 
 const pirateLords = await User.getByTitleId(captainTitle.id); // List of all Captains
 ```
 
-### Composite Index
+### Composite Indexes
 
 Composite indexes work just like regular indexes, except they provide even faster access to data by utilising multiple columns to create the index.
 
@@ -362,6 +364,30 @@ type Transfer @entity {
 }
 ```
 
+## Full Text Search
+
+We support a fast, efficient way to perform full text search across multiple fields in entities.
+
+::: warning Note
+
+This will create a new generated column and index in your Database. Adding full text search to an existing project via [project upgrade](./project-upgrades.md) or on an existing dataset might result in some performance issues when initially building these indexes.
+
+:::
+
+To add support for full text search on an entity field, add the `fullText` directive to an entity. This directive requires the fields that wish to be searchable. These fields must be either `ID`, `String`, or a foreign key. A language option is also required to provide optimal search functionality, supported languages can be found at [https://stackoverflow.com/a/39752553](https://stackoverflow.com/a/39752553).
+
+```graphql
+type NFT
+  @entity
+  @fullText(fields: ["name", "description"], language: "english") {
+  id: ID!
+  name: String!
+  description: String!
+}
+```
+
+To read about how to query full text fields, see [querying with full text search](../run_publish/query/graphql.md#full-text-search).
+
 ## JSON type
 
 We are supporting saving data as a JSON type, which is a fast way to store structured data. We'll automatically generate corresponding JSON interfaces for querying this data and save you time defining and managing entities.

diff --git a/docs/run_publish/query/graphql.md b/docs/run_publish/query/graphql.md
@@ -16,3 +16,26 @@ You can follow the [official GraphQL guide here](https://graphql.org/learn/) to
 ![SubQuery Project](/assets/img/run_publish/query.png)
 
 On the top right of the playground, you'll find a _Docs_ button that will open a documentation draw. This documentation is automatically generated and helps you find what entities and methods you can query. On our [Managed Service Explorer](https://explorer.subquery.network/), you will also find a query constructor.
+
+### Full Text Search
+
+The result of the directive will provide new connections to the graphql schema allowing you to search. They follow the pattern `search<EntityName>` and take a `search` parameter.
+
+The search parameter allows for more than just searching for strings, you can do AND (`&`), OR (`|`) , NOT (`!`, `-`), begins with (`<Text>:*`, `<Text>*`>) and follows (`>`, `<->`).
+
+For more details on these operations please see [pg-tsquery](https://github.com/caub/pg-tsquery) for sanitised operations and [Postgres tsquery](https://www.postgresql.org/docs/current/textsearch-controls.html) for the underlying DB implementation.
+
+```graphql
+
+# Search for all NFTs with either "blue" or "red" in the name or description
+{
+  searchNFTs(search: "blue|red") {
+    nodes: {
+      id
+      name
+      description
+    }
+  }
+}
+
+```