Skip to content

buildbarn/bb-remote-asset

This branch is 10 commits ahead of, 11 commits behind master.

Folders and files

NameName
Last commit message
Last commit date

Latest commit

ec2ae0e · Feb 12, 2025
Feb 12, 2025
Sep 11, 2024
Sep 11, 2024
Jan 22, 2025
Feb 12, 2025
Dec 20, 2024
Nov 5, 2023
Jan 22, 2025
Aug 27, 2020
Aug 27, 2020
Aug 27, 2020
Sep 11, 2024
Aug 27, 2020
Feb 12, 2025
Feb 12, 2025
Sep 11, 2024
Feb 12, 2025
Feb 12, 2025
Sep 10, 2023
Sep 10, 2023

Repository files navigation

Buildbarn Remote Asset Build status Go Report CardDocker Pulls

This repository provides a service for the remote asset protocol. This protocol is used by tools such as bazel / buildstream to provide a mapping between URIs and qualifiers to digests which can be used by the remote execution (REv2) protocol.

The remote asset daemon can be configured with bb-storage blobstore backends to enable a scalable remote asset service which can be integrated with any REv2 compatible GRPC cache.

Setting up the Remote Asset daemon

With Action Cache

$ cat config/bb_remote_asset.jsonnet
{
  contentAddressableStorage: common.blobstore.contentAddressableStorage,
  assetCache: {
    actionCache: common.blobstore.actionCache,
  },
  fetcher: {
    http: {},
  },
  global: common.global,
  grpcServers: [{
    listenAddresses: [':8981'],
    authenticationPolicy: { allow: {} },
  }],
  allowUpdatesForInstances: ['foo'],
  maximumMessageSizeBytes: 16 * 1024 * 1024 * 1024,
  fetchAuthorizer: { allow: {} },
  pushAuthorizer: { allow: {} },
}

With Blob Access cache

$ cat config/bb_remote_asset.jsonnet
{
  contentAddressableStorage: common.blobstore.contentAddressableStorage,
  assetCache: {
    blobAccess: {
      'local': {
        keyLocationMapOnBlockDevice: {
          file: {
            path: '/storage/key_location_map',
            sizeBytes: 1024 * 1024,
          },
        },
        keyLocationMapMaximumGetAttempts: 8,
        keyLocationMapMaximumPutAttempts: 32,
        oldBlocks: 8,
        currentBlocks: 24,
        newBlocks: 1,
        blocksOnBlockDevice: {
          source: {
            file: {
              path: '/storage/blocks',
              sizeBytes: 100 * 1024 * 1024,
            },
          },
          spareBlocks: 3,
        },
        persistent: {
          stateDirectoryPath: '/storage/persistent_state',
          minimumEpochInterval: '300s',
        },
      },
    },
  },
  fetcher: {
    http: {},
  },
  global: common.global,
  grpcServers: [{
    listenAddresses: [':8981'],
    authenticationPolicy: { allow: {} },
  }],
  allowUpdatesForInstances: ['foo'],
  maximumMessageSizeBytes: 16 * 1024 * 1024 * 1024,
  fetchAuthorizer: { allow: {} },
  pushAuthorizer: { allow: {} },
}

Both of the above configs rely on there being a common.libsonnet file containing a definition of the bb-storage blobstore.

$ docker run \
    -p 8981:8981 \
    -v $(pwd)/config:/config \
    -v $(pwd)/storage-asset:/storage-asset \
    bazel/cmd/bb_remote_asset:bb_remote_asset_container \
    /config/bb_remote_asset.jsonnet

In the example above, the daemon is configured to store asset references within a disk backed circular storage backend. The fetcher is configured to support fetching via HTTP when a reference is not found matching the request URI/Qualifier criteria, these fetched blobs are placed into a REv2 compatible GRPC cache and the digest returned to the remote asset client. HTTP Fetched blobs are configured to be cached references to newly fetched blobs in the asset store for future fetches.

Bazel can be configured to use this service as a remote uploader as follows:

$ bazel build --remote_cache=grpc://<cache_address>:<cache grpc port> --remote_instance_name=foo --experimental_remote_downloader="grpc://localhost:8981" //...