Merge branch 'main' into deep-cody-docs

README.md

@@ -1,6 +1,6 @@
 # Sourcegraph Docs
 
-<!-- Working branch for Nov 25th 2024 Release -->
+<!-- Working branch for Dec 2024 Release -->
 
 Welcome to the Sourcegraph documentation! We're excited to have you contribute to our docs. We've recently rearchitectured our docs tech stack — powered by Next.js, TailwindCSS and deployed on Vercel. This guide will walk you through the process of contributing to our documentation using the new tech stack.
 

baseai/memory/docs/index.ts

@@ -1,14 +1,24 @@
 import { MemoryI } from '@baseai/core';
-import path from 'path';
 
 const memoryDocs = (): MemoryI => ({
   name: 'docs',
   description: 'Docs folder of sourcegraph docs repository as an auto-synced memory',
-  config: {
-    useGitRepo: true,
-    dirToTrack: path.posix.join('.', 'docs'),
-    extToTrack: [".md", ".mdx"]
-  }
+	git: {
+		enabled: true,
+		include: ['**/*'],
+		gitignore: true,
+		deployedAt: '',
+		embeddedAt: '',
+	},
+  documents: {
+		meta: doc => {
+			const url = `https://sourcegraph.com/docs/${doc.path}`;
+			return {
+				url,
+				name: doc.name,
+			};
+		},
+	},
 });
 
 export default memoryDocs;

docs.config.js

@@ -1,5 +1,5 @@
 const config = {
-	DOCS_LATEST_VERSION: '5.10'
+	DOCS_LATEST_VERSION: '5.11'
 };
 
 module.exports = config;

docs/admin/audit_log.mdx

@@ -51,9 +51,9 @@ Here's a word-by-word breakout to demonstrate how the captured entry aligns with
 
 ### What is audited?
 
-- [Security events](./security_event_logs)
-- [Gitserver access](https://sourcegraph.com/github.com/sourcegraph/sourcegraph/-/blob/cmd/gitserver/internal/accesslog/accesslog.go?L100-104)
-- [GraphQL requests](https://sourcegraph.com/github.com/sourcegraph/sourcegraph/-/blob/cmd/frontend/internal/httpapi/graphql.go?L226-244)
+- **Security Events** - [A list of events](./security_event_logs) such as logins, site configuration updates, code host configuration updates (_These logs can be sent to the database for easier collection as well as to the standard output_).
+- **Gitserver Access** - Requests to the [`gitserver`](./deploy/scale.mdx#gitserver) component that is responsible for brokering transactions between the git repos on disk and all other Sourcegraph components.
+- **GraphQL Requests** - Requests To The Sourcegraph [`frontend`](./deploy/scale.mdx#frontend) component that is usually named `sourcegraph-frontend-{DYNAMICALLY CREATED ALPHANUMERIC}` 
 
 This list is expected to grow in the future.
 
@@ -89,6 +89,9 @@ Audit logs are structured logs. As long as one can ingest logs, we assume one ca
 
 ### On Premises
 
+All logs mentioned are delivered to the standard output (stdout) for each individual [component](./deploy/scale.mdx#core-components). 
+
+
 There are two easy approaches to filtering the audit logs:
 
 - JSON-based: look for the presence of the `Attributes.audit` node. Do not depend on the log level, as it can change based on `SRC_LOG_LEVEL`.

docs/admin/config/index.mdx

@@ -33,3 +33,4 @@ This page documents how to configure a Sourcegraph instance. For deployment conf
 - [Restore postgres database from snapshot](/admin/config/restore/)
 - [Enabling database encryption for sensitive data](/admin/config/encryption)
 - [Configuring Sourcegraph in private networks](/admin/config/private-network)
+- [Restricting outgoing connections](/admin/config/network-filtering)

docs/admin/config/network-filtering.mdx

@@ -0,0 +1,32 @@
+# Outoing Connection Filtering
+Sourcegraph supports outbound connection filtering. Both for regular external connections and so-called "untrusted" connections, where a regular user can provide a URL to make an outbound connection to.
+
+The allow- and denylist support a comma separated list of IP ranges, hostnames and keywords. To block or allow all the internal connections use the “private” keyword, this would block all RFC 1918 (10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16) and RFC 4193 (FC00::/7) IP addresses. Keywords can be combined with ranges and IP addresses so it's very customizable.
+
+## Trusted External Connections
+It’s possible for the Sourcegraph instance to deny access to external hosts by setting the environment variable `EXTERNAL_DENY_LIST` on the deployment.
+
+The default denylist is set up to only block localhost and the Cloud metadata service IP address. Expanding the denylist could interfere with internal authentication providers, and they might need to be excluded from the denylist.
+
+### Example Configuration
+
+Adding a denylist can be done by setting the environment variable `EXTERNAL_DENY_LIST` on the deployment.
+
+```
+EXTERNAL_DENY_LIST="private,github.com"
+```
+
+This would deny all connections to hosts in the private network and github.com.
+
+## Untrusted External Connections
+Codemonitors, webhooks and Cody URL context are limited to only be able to access public IP addresses by default. This behavior can be changed with the `UNTRUSTED_EXTERNAL_ALLOW_LIST` environment variable, which configures the allowlist.
+
+### Example Configuration
+If you want Cody to use context from an internal server in addition to internet access, you can add the internal server's IP address to the allowlist:
+
+```
+UNTRUSTED_EXTERNAL_ALLOW_LIST="external,192.168.1.53"
+```
+
+## Implementation Details
+To achieve this, we use [gitea's hostmatcher](https://github.com/go-gitea/gitea/blob/v1.22.6/modules/hostmatcher/hostmatcher.go#L39). This is configured by default for the `ExternalClient`, which is used for all external requests. The common options and configuration can be found [here](https://github.com/sourcegraph/sourcegraph-public-snapshot/blob/main/internal/httpcli/client.go#L406C1-L423C2).

docs/admin/config/private-network.mdx

@@ -1,9 +1,11 @@
 # Private network configuration
+
 A **private network** refers to a secure network environment segregated from the public internet, designed to facilitate internal communications and operations within an organization. This network setup restricts external access, enhancing security and control over data flow by limiting exposure to external threats and unauthorized access.
 
-When deploying self-hosted Sourcegraph instances in private networks with specific compliance and policy requirements, additional configuration may be required to ensure all networking features function correctly. The reasons for applying the following configuration options depend on the specific functionality of the Sourcegraph service and the unique network and infrastructure requirements of the organization. 
+When deploying self-hosted Sourcegraph instances in private networks with specific compliance and policy requirements, additional configuration may be required to ensure all networking features function correctly. The reasons for applying the following configuration options depend on the specific functionality of the Sourcegraph service and the unique network and infrastructure requirements of the organization.
 
 The following is a list of Sourcegraph services and how and when each initiates outbound connections to external services:
+
 - **executor**: Sourcegraph [Executor](../executors) batch change or precise indexing jobs may need to connect to services hosted within an organization's private network
 - **frontend**: The frontend service communicates externally when connecting to external [auth providers](../auth), sending [telemetry data](../pings), testing code host connections, and connecting to [externally hosted](../external_services) Sourcegraph services
 - **gitserver**: Executes git commands against externally hosted [code hosts](../external_service)
@@ -12,15 +14,17 @@ The following is a list of Sourcegraph services and how and when each initiates
 - **worker**: Sourcegraph [Worker](../workers) run various background jobs that may require establishing connections to services hosted within an organization's private network
 
 ## HTTP proxy configuration
+
 All Sourcegraph services respect the conventional `HTTP_PROXY`, `HTTPS_PROXY`, and `NO_PROXY` environment variables for routing Sourcegraph client application HTTP traffic through a proxy server. The steps for configuring proxy environment variables will depend on your Sourcegraph deployment method.
 
 ### Kubernetes Helm
+
 Add the proxy environment variables to your Sourcegraph Helm chart [override file](https://github.com/sourcegraph/deploy-sourcegraph-helm/blob/main/charts/sourcegraph/values.yaml):
 
 ```yaml
 executor|frontend|gitserver|migrator|repo-updater|worker:
   env:
-  - name: HTTP_PROXY 
+  - name: HTTP_PROXY
     value: http://proxy.example.com:8080
   - name: HTTPS_PROXY
     value: http://proxy.example.com:8080
@@ -33,7 +37,7 @@ executor|frontend|gitserver|migrator|repo-updater|worker:
 ## Using private CA root certificates
 Some organizations maintain a private Certificate Authority (CA) for issuing certificates within their private network. When Sourcegraph connects to TLS encrypted service using a self-signed certificate that it does not trust, you will observe an `x509: certificate signed by unknown authority` error message in logs.
 
-In order for Sourcegraph to respect an organization's self-signed certificates, the private CA root certificate(s) will need to be appended to Sourcegraph's trusted CA root certificate list in `/etc/ssl/certs/ca-certificates.crt`. 
+In order for Sourcegraph to respect an organization's self-signed certificates, the private CA root certificate(s) will need to be appended to Sourcegraph's trusted CA root certificate list in `/etc/ssl/certs/ca-certificates.crt`.
 
 ### Configuring sourcegraph-frontend to recognize private CA root certificates
 The following details the process for setting up the sourcegraph-frontend to acknowledge and trust a private CA root certificate for Sourcegraph instances deployed using [Helm](../deploy/kubernetes/helm). For any other Sourcegraph service that needs to trust an organization's private CA root certificate (including gitserver, repo-updater, or migrator), similar steps will need to be followed.

docs/admin/config/site_config.mdx

@@ -268,7 +268,7 @@ All site configuration options and their default values are shown below.
 	// - {
 	//     "debug": true,
 	//     "sampling": "all",
-	//     "type": "jaeger",
+	//     "type": "opentelemetry", // Jaeger now uses the OpenTelemetry format, the old jaeger format is deprecated
 	//     "urlTemplate": "{{ .ExternalURL }}/-/debug/jaeger/trace/{{ .TraceID }}"
 	//   }
 
@@ -304,9 +304,12 @@ All site configuration options and their default values are shown below.
 	// Don't sync a user's permissions if they have synced within the last n seconds.
 	"permissions.syncUsersBackoffSeconds": 60,
 
-	// The maximum number of user-centric permissions syncing jobs that can be spawned concurrently. Service restart is required to take effect for changes.
+	// The maximum number of user-centric permissions syncing jobs that can be spawned concurrently. Server restart is required for changes to take effect.
 	"permissions.syncUsersMaxConcurrency": 1,
 
+	// The maximum number of repo-centric permissions syncing jobs that can be spawned concurrently. Server restart is required for changes to take effect.
+	"permissions.syncReposMaxConcurrency": 5,
+
 	"rateLimits": null,
 
 	// Enables redacting sensitive information from outbound requests. Important: We only respect this setting in development environments. In production, we always redact outbound requests.