Skip to content

Commit

Permalink
docs: fix references to github repo
Browse files Browse the repository at this point in the history 
This relies on the myst parser's (new in v0.19) feature of customizing
external URL schemes with the "myst_url_schemes" configuration option.
We specify new URL schemes "blob" and "tree" which are supposed to be
used for references to blobs (files) and trees (directories) in the git
repo.

This also drops sphinx event hooks ("doctree-resolved" and
"missing-reference"), simplifying the conf.py considerably. They were
useless anyway as they were totally broken and didn't actually fix any
references to repo files.

The downside of this is that the affected references are not shown when
browsing .md files directly from github (as github dosn't handle our
custom 'tree:' and 'blob:' url schemes. However, it should be preferable
to have the cross-references working in the official documentation.

Signed-off-by: Markus Lehtonen <[email protected]>
  • Loading branch information
@marquiz @klihub
marquiz authored and klihub committed Jun 22, 2023
1 parent e2c45a4 commit b4308f1
Show file tree
Hide file tree
Showing 2 changed files with 20 additions and 161 deletions.
159 changes: 9 additions & 150 deletions docs/conf.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -32,16 +32,6 @@
#
# This section determines the behavior of links to local items in .md files.
#
# if useGitHubURL == True:
#
# links to local files and directories will be turned into github URLs
# using either the baseBranch defined here or using the commit SHA.
#
# if useGitHubURL == False:
#
# local files will be moved to the website directory structure when built
# local directories will still be links to github URLs
#
# if built with GitHub workflows:
#
# the GitHub URLs will use the commit SHA (GITHUB_SHA environment variable
Expand All @@ -50,7 +40,6 @@
##############################################################################

baseBranch = "main"
useGitHubURL = True
commitSHA = getenv('GITHUB_SHA')
githubServerURL = getenv('GITHUB_SERVER_URL')
githubRepository = getenv('GITHUB_REPOSITORY')
Expand Down Expand Up @@ -123,6 +112,15 @@ def gomod_versions(modules):
}
myst_heading_anchors = 3

myst_url_schemes = {
"http": None,
"https": None,
"ftp": None,
"mailto": None,
"blob": githubFileURL + "{{path}}",
"tree": githubDirURL + "{{path}}",
}

# Add any paths that contain templates here, relative to this directory.
templates_path = ['_templates']

Expand All @@ -141,142 +139,3 @@ def gomod_versions(modules):
html_theme_options = {
'display_version': True,
}

# Add any paths that contain custom static files (such as style sheets) here,
# relative to this directory. They are copied after the builtin static files,
# so a file named "default.css" will overwrite the builtin "default.css".
#html_static_path = ['_static']

def setup(app):
app.connect('doctree-resolved',fixLocalMDAnchors)
app.connect('missing-reference',fixRSTLinkInMD)

###############################################################################
#
# This section defines callbacks that make markdown specific tweaks to
# either:
#
# 1. Fix something that recommonmark does wrong.
# 2. Provide support for .md files that are written as READMEs in a GitHub
# repo.
#
# Only use these changes if using the extension ``recommonmark``.
#
###############################################################################

def isHTTPLink(uri):
return uri.startswith('http://') or uri.startswith('https://')

def isMDFileLink(uri):
return uri.endswith('.md') or '.md#' in uri

def isRSTFileLink(uri):
return uri.endswith('.rst')

# Callback registerd with 'missing-reference'.
def fixRSTLinkInMD(app, env, node, contnode):
refTarget = node.get('reftarget')

if isHTTPLink(refTarget):
return

if isRSTFileLink(refTarget) and not isHTTPLink(refTarget):
# This occurs when a .rst file is referenced from a .md file
# Currently unable to check if file exists as no file
# context is provided and links are relative.
#
# Example: [Application examples](examples/readme.rst)
#
contnode['refuri'] = contnode['refuri'].replace('.rst','.html')
contnode['internal'] = "True"
return contnode
elif refTarget.startswith("/"):
# This occurs when a file is referenced for download from an .md file.
# Construct a list of them and short-circuit the warning. The files
# are moved later (need file location context). To avoid warnings,
# write .md files, make the links absolute. This only marks them fixed
# if it can verify that they exist.
#
# Example: [Makefile](/Makefile)
#
filePath = refTarget.lstrip("/")
if isfile(filePath) or isdir(filePath):
return contnode


def normalizePath(docPath,uriPath):
if uriPath == "":
return uriPath
if "#" in uriPath:
# Strip out anchors
uriPath = uriPath.split("#")[0]
if uriPath.startswith("/"):
# It's an absolute path
return uriPath.lstrip("/") #path to file from project directory
else:
# It's a relative path
docDir = dirname(docPath)
return join(docDir,uriPath) #path to file from referencing file


# Callback registerd with 'doctree-resolved'.
def fixLocalMDAnchors(app, doctree, docname):
for node in doctree.traverse(nodes.reference):
uri = node.get('refuri')

if uri == None or isHTTPLink(uri):
continue

filePath = normalizePath(docname,uri)

if isfile(filePath):
# Only do this if the file exists.
#
# TODO: Pop a warning if the file doesn't exist.
#
if isMDFileLink(uri) and not isHTTPLink(uri):
# Make sure .md file links that weren't caught are converted.
# These occur when creating an explicit link to an .md file
# from an .rst file. By default these are not validated by Sphinx
# or recommonmark. Only toctree references are validated. recommonmark
# also fails to convert links to local Markdown files that include
# anchors. This fixes that as well.
#
# Only include this code if .md files are being converted to html
#
# Example: `Google Cloud Engine <gce.md>`__
# [configuration options](autotest.md#configuration-options)
#
node['refuri'] = node['refuri'].replace('.md','.html')
else:
# Handle the case where markdown is referencing local files in the repo
#
# Example: [Makefile](/Makefile)
#
if useGitHubURL:
# Replace references to local files with links to the GitHub repo
#
newURI = join(githubFileURL, filePath)
print("new url: ", newURI)
node['refuri']=newURI
else:
# If there are links to local files other than .md (.rst files are caught
# when warnings are fired), move the files into the Sphinx project, so
# they can be accessed.
newFileDir = join(app.outdir,dirname(filePath)) # where to move the file in Sphinx output.
newFilePath = join(app.outdir,filePath)
newURI = uri # if the path is relative no need to change it.
if uri.startswith("/"):
# It's an absolute path. Need to make it relative.
uri = uri.lstrip("/")
docDirDepth = len(docname.split("/")) - 1
newURI = "../"*docDirDepth + uri
if not isdir(newFileDir):
makedirs(newFileDir)
copyfile(filePath,newFilePath)
node['refuri'] = newURI
elif "#" not in uri: # ignore anchors
# turn links to directories into links to the repo
if isdir(filePath):
newURI = join(githubDirURL, filePath)
node['refuri']=newURI
22 changes: 11 additions & 11 deletions docs/resource-policy/developers-guide/architecture.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ Kubernetes CRDs and ConfigMaps.

## Components

### [Node Agent](/pkg/resmgr/agent/)
### [Node Agent](tree:/pkg/resmgr/agent)

The node agent is a component internal to NRI-RP itself. All interactions
by NRI-RP with the Kubernetes Control Plane go through the node agent with
Expand All @@ -50,7 +50,7 @@ The agent interface implements the following functionality:
The config interface is defined and has its gRPC server running in
NRI-RP. The agent acts as a gRPC client for this interface. The low-level
cluster interface is defined and has its gRPC server running in the agent,
with the [convenience layer](/pkg/resmgr/agent) defined in NRI-RP.
with the [convenience layer](tree:/pkg/resmgr/agent) defined in NRI-RP.
NRI-RP acts as a gRPC client for the low-level plumbing interface.

Additionally, the stock node agent that comes with NRI-RP implements schemes
Expand All @@ -59,7 +59,7 @@ for:
- management of dynamic adjustments to container resource assignments


### [Resource Manager](/pkg/resmgr/)
### [Resource Manager](tree:/pkg/resmgr/)

NRI-RP implements an event processing pipeline. In addition to NRI events,
it processes a set of other events that are not directly related to or the
Expand Down Expand Up @@ -127,7 +127,7 @@ following, based on the event type:
4. Release the pipeline lock.


### [Cache](/pkg/resmgr/cache/)
### [Cache](tree:/pkg/resmgr/cache/)

The cache is a shared internal storage location within NRI-RP. It tracks the
runtime state of pods and containers known to NRI-RP, as well as the state
Expand Down Expand Up @@ -161,22 +161,22 @@ the policy's event handler with the injected event as an argument and with
the cache properly locked.


### [Generic Policy Layer](/pkg/resmgr/policy/policy.go)
### [Generic Policy Layer](blob:/pkg/resmgr/policy/policy.go)

The generic policy layer defines the abstract interface the rest of NRI-RP
uses to interact with policy implementations and takes care of the details
of activating and dispatching calls through to the configured active policy.


### [Generic Resource Controller Layer](/pkg/resmgr/control/control.go)
### [Generic Resource Controller Layer](blob:/pkg/resmgr/control/control.go)

The generic resource controller layer defines the abstract interface the rest
of NRI-RP uses to interact with resource controller implementations and takes
care of the details of dispatching calls to the controller implementations
for post-policy enforcment of decisions.


### [Metrics Collector](/pkg/metrics/)
### [Metrics Collector](tree:/pkg/metrics/)

The metrics collector gathers a set of runtime metrics about the containers
running on the node. NRI-RP can be configured to periodically evaluate this
Expand All @@ -185,19 +185,19 @@ resources is and to attempt a rebalancing/reallocation if it is deemed
both possible and necessary.


### [Policy Implementations](/cmd/)
### [Policy Implementations](tree:/cmd/)

#### [Topology Aware](/cmd/topology-aware/)
#### [Topology Aware](tree:/cmd/topology-aware/)

A topology-aware policy capable of handling multiple tiers/types of memory,
typically a DRAM/PMEM combination configured in 2-layer memory mode.

#### [Balloons](/cmd/balloons/)
#### [Balloons](tree:/cmd/balloons/)

A balloons policy allows user to define fine grained control how the
computer resources are distributed to workloads.

#### [Template](/cmd/template/)
#### [Template](tree:/cmd/template/)

The template policy can be used as a base for developing new policies.
It provides hooks that the policy developer can fill to define fine grained
Expand Down

0 comments on commit b4308f1

Please sign in to comment.