Prompt my Docs

Talk with your docs and GPT at the same time

Have you ever found a new library and wanted to ask questions about it? Or stumbled upon some undocumented code and have no idea what happens there? Then look no further, as you can put any data inside of docs and start prompting.

---

• In Action
• Setup
• Bring your docs
• Start prompt-my-docs
• Under the hood
• Config
  • Change the config
  • Disabling a file type like markdown
  • Update ignorePaths to remove / add paths
  • Handle larger documents with maxDocs and chunkSize
  • Make the search more/less strict with docSearchDistance
  • Find more/less docs with docSearchDistance
  • Have more/less variations in the output with temperature
  • Spend more/less tokens for talking with GPT with maxNewTokens
• Do I have to use Next.js?

---

In Action

We cloned hyv into the docs folder and used the following prompt to generate a guide on how to get started with hyv:

Getting started guide for hyv with a config for GPT-4, so that I can use GPT-4 to get the answer to my question “what is the meaning of life” and see the result printed to the console. Please use ESM syntax.

Setup

• Clone the project or download the ZIP
• Install dependencies: npm i
• Register for a Weaviate sandbox and obtain the host of your sandbox and the API Key
• Create an OpenAI account and create an API Key
  • 🚨 It is recommended to have access to GPT-4, as this is what we used during development of Prompt my Docs. If you don't have access yet, see How can I access GPT-4
• Create a .env based on .env.example and put the API keys (both Weaviate & OpenAI) + Weaviate host inside

Bring your docs

Add all the files that you want to search into the docs folder (currently only all dataType (like js, ts or md) that are defined in the config, please open an issue for more!)

For example you can clone a repo that you would love to learn about into the docs folder, like hyv.

When you have prepared your data, you can add it into the vector database:

# Populate the database. This only needs to be done once for a new database.
# Run this if new pages have been added or content has been changed.
npm run update-database

Start prompt-my-docs

When the vector database is ready, you can start the web app:

# Run the development server.
npm run dev

Open the web app via localhost:3000 (or similar based on your setup).

Under the hood

• We read all your data from the "docs" folder, currently only all dataType (like js, ts or md) that are defined in the config
• Your data will be converted into a vector and saved into the vector database (e.g. weaviate)
  • This needs to be done before you run the project for the first time or when your data is changing
• You can then run the project and open the web app
• You can then ask your question and we will use the data from the vector database that is very similar to your prompt to populate the context when we interact with GPT
  • This makes sure that GPT knows about your specific data and can answer questions related to your data

Config

The default config looks like this:

{
  "vectorDatabase": {
    "maxDocs": 6,
    "docSearchDistance": 0.24,
    "answerSearchDistance": 0.24
  },
  "gpt": {
    "temperature": 0.2,
    "maxNewTokens": 3048
  },
  "dataType": {
    "markdown": {
      "enabled": true,
      "extensions": [".md", ".mdx"],
      "ignorePaths": ["node_modules", "dist", ".github"],
      "chunkSize": 1000,
      "chunkOverlap": 0
    },
    "js": {
      "enabled": true,
      "extensions": [".js"],
      "ignorePaths": ["node_modules", "dist", ".github"],
      "chunkSize": 1000,
      "chunkOverlap": 0
    },
    "ts": {
      "enabled": true,
      "extensions": [".ts"],
      "ignorePaths": ["node_modules", "dist", ".github", ".d.ts"],
      "chunkSize": 1000,
      "chunkOverlap": 0
    }
  }
}

Change the config

You can override the default configuration by creating a config.json file at the root of the project. This comes in handy if the default settings make no sense for the docs that you want to use. For example, it might be, that the code files are large, so it's better to have bigger chunks and fewer files.

We have covered some of the default cases, so feel free to change the config accordingly to your use case.

Disabling a file type like markdown

This configuration disables the markdown file type.

{
  "dataType": {
    "markdown": {
      "enabled": false
    }
  }
}

Update ignorePaths to remove / add paths

This configuration changes the ignore paths for markdown files, removing dist and .github from the ignore list and adding src.

{
  "dataType": {
    "markdown": {
      "ignorePaths": ["node_modules", "src"]
    }
  }
}

Handle larger documents with maxDocs and chunkSize

This configuration reduces the maximum number of documents retrieved from the vector database to 4 and increases the chunkSize for markdown files to 1500. After changing the configuration, run npm run update-database to update the database.

{
  "vectorDatabase": {
    "maxDocs": 4
  },
  "dataType": {
    "markdown": {
      "chunkSize": 1500
    }
  }
}

Make the search more/less strict with docSearchDistance

This configuration reduces the docSearchDistance to 0.1, making searching documents in the vector database more strict. This means that returned documents will be more closely related to the search vector prompt.

{
  "vectorDatabase": {
    "docSearchDistance": 0.1
  }
}

Find more/less docs with docSearchDistance

This configuration increases the docSearchDistance to 0.3. This change makes the search for documents in the vector database less strict. As a result, more documents are found, but they may be less closely related to the search vector prompt.

{
  "vectorDatabase": {
    "docSearchDistance": 0.3
  }
}

Have more/less variations in the output with temperature

The temperature parameter controls the randomness of the GPT's output. A higher temperature value results in more random outputs, while a lower temperature value makes the outputs more deterministic and focused. You might want to adjust this parameter to fine-tune the balance between randomness and determinism in the GPT's responses.

{
  "gpt": {
    "temperature": 0.6
  }
}

Spend more/less tokens for talking with GPT with maxNewTokens

The maxNewTokens parameter controls the maximum length of the output from GPT. By adjusting this value, you can control how much text the GPT generates in response to an input. If you're finding that the responses are too short or too long, you can tweak this setting.

The context of GPT is defined by combining the tokens of the input (like the user prompt) with the tokens needed to respond (in this case controlled by maxNewTokens). For GPT-4 8k, the sum of both values can't exceed 8096 tokens. You will run into an error if you request a bigger context size.

{
  "gpt": {
    "maxNewTokens": 1024
  }
}

Do I have to use Next.js?

You don't need Next.js to use the prompt-my-docs, we just use it, as it's a nice way to have a web app running. You can also just extract the parts that you need and use them without Next.js on your website. We just hadn't the requirement yet. But if you need help here, please open an issue and we are happy to extract the parts that are needed, so you can use them in any environment.

We are using an API route from Next.js, so that the request to GPT happens on the server and we don't have to expose the secrets to the client. But you could write your own API and use everything else without Next.js at all.