Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Nhse d34 nhskv.i33 token #35

Open
wants to merge 37 commits into
base: nhse-develop-3.4
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 32 commits
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
79a0e55
Initial support for OTP26
martinsumner May 2, 2024
246774a
Remove tracers
martinsumner May 2, 2024
6c46b9d
Update rebar.config
martinsumner May 11, 2024
1d04e59
Update rebar.config
martinsumner May 14, 2024
38907f2
Token Manager
martinsumner May 29, 2024
4e3489d
Add strong conditional check to PB API
martinsumner May 29, 2024
b615d2e
Reset state on PB connection after completion
martinsumner May 29, 2024
cb53d98
Add downstream recording of state
martinsumner May 30, 2024
deb3ce3
Fix uniquenes of node list
martinsumner May 31, 2024
6854977
Refactor riak_kv_pb_object
martinsumner May 31, 2024
47bd106
Move strong conditional check to FSM (#34)
martinsumner May 31, 2024
39a3e87
Add support to HTTP API
martinsumner May 31, 2024
09fcda2
Conditional PUT to require GET not HEAD
martinsumner Jun 5, 2024
56b6833
Add configuration
martinsumner Jun 5, 2024
a0b027b
Add profile function
martinsumner Jun 5, 2024
ef61d8b
Add profiler
martinsumner Jun 6, 2024
a11d4d2
Merge branch 'nhse-d34-otp26' into nhse-d34-nhskv.i30-profiler
martinsumner Jun 6, 2024
664b6e5
Type fix
martinsumner Jun 6, 2024
5b8cb9f
Add to extending list of defaults
martinsumner Jun 7, 2024
346f276
Merge branch 'nhse-d34-nhskv.i30-profiler' into nhse-d34-nhskv.i33-token
martinsumner Jun 12, 2024
87cf5b1
Initial write-up of riak_kv_token_manager
martinsumner Jun 12, 2024
5c18813
Refactor to use monitoring
martinsumner Jun 13, 2024
665d467
Change to config parameters
martinsumner Jun 13, 2024
2f95b40
Remove duplicate
martinsumner Jun 13, 2024
55615f7
Rename (again), make messages async
martinsumner Jun 16, 2024
30a871b
Clarify erpc errors
martinsumner Jun 17, 2024
7758686
Use token_manager to control downstream messages
martinsumner Jun 18, 2024
ce10992
Add GC process
martinsumner Jun 19, 2024
140f883
Avoid re-fetching objects (#36)
martinsumner Jun 19, 2024
c4a033f
Merge branch 'nhse-d34-nhskv.i33-token' of https://github.com/nhs-ria…
martinsumner Jun 19, 2024
09ad183
Extend eunit test of token_manager
martinsumner Jun 19, 2024
10de288
Try and keep github formatter happy
martinsumner Jun 19, 2024
11caa89
Use monitor rather than spoof 'DOWN'
martinsumner Jun 19, 2024
6ba1736
Use uniq code in OTP 24
martinsumner Jun 19, 2024
8f9b5e0
Update comments for perceived clarity
martinsumner Jun 21, 2024
904fd69
Merge branch 'nhse-develop-3.4' into nhse-d34-nhskv.i33-token
martinsumner Sep 23, 2024
3a102b3
Remove double-definition on merge
martinsumner Sep 23, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 10 additions & 17 deletions .github/workflows/erlang.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,15 @@ name: Erlang CI

on:
push:
branches: [ nhse-develop ]
branches:
- nhse-develop
- nhse-develop-3.2
- nhse-develop-3.4
pull_request:
branches: [ nhse-develop ]
branches:
- nhse-develop
- nhse-develop-3.2
- nhse-develop-3.4


jobs:
Expand All @@ -17,25 +23,12 @@ jobs:
strategy:
fail-fast: false
matrix:
otp: [22, 24, 25]
otp: [24, 26]
os: [ubuntu-latest]
# OTP lower than 23 does not run on ubuntu-latest (22.04), see
# https://github.com/erlef/setup-beam#compatibility-between-operating-system-and-erlangotp
exclude:
- otp: 22
os: ubuntu-latest
include:
- otp: 22
os: ubuntu-20.04

steps:
- uses: lukka/get-cmake@latest
- uses: actions/checkout@v2
- name: Install dependencies (Ubuntu)
if: ${{ startsWith(matrix.os, 'ubuntu') }}
run: |
sudo apt-get -qq update
sudo apt-get -qq install libsnappy-dev libc6-dev
- uses: actions/checkout@v4
- name: Install Erlang/OTP
uses: erlef/setup-beam@v1
with:
Expand Down
4 changes: 3 additions & 1 deletion eqc/ec_eqc.erl
Original file line number Diff line number Diff line change
Expand Up @@ -671,13 +671,15 @@ get_fsm_proc(ReqId, #params{n = N, r = R}) ->
DeletedVclock = true,
ExpectedVclock = false,
NodeConfirms = 0,
ReturnBody = true,
GetCore = riak_kv_get_core:init(N, R,
0, %% SLF hack
FailThreshold,
NotFoundOk, AllowMult, DeletedVclock,
[{Idx, primary} || Idx <- lists:seq(1, N)], %% SLF hack
ExpectedVclock,
NodeConfirms
NodeConfirms,
ReturnBody
),
#proc{name = {get_fsm, ReqId}, handler = get_fsm,
procst = #getfsmst{getcore = GetCore}}.
Expand Down
68 changes: 68 additions & 0 deletions priv/riak_kv.schema
Original file line number Diff line number Diff line change
Expand Up @@ -540,6 +540,74 @@
hidden
]}.

%% @doc Mode for handling conditional checks on PUTs
%% Handling if-not-modified (vclock-based) and if-none-match conditionals on
%% PUTS, there are three possible modes:
%% - api_only (default)
%% - prefer_token
%% - mandate_token (not currently implemented)
%%
%% In the api_only mode, a read will be done before the write within the API,
%% and if the read passes the condition the PUT will be allowed (even though
%% a parallel conditional PUT may be in-flight).
%% In the prefer_token mode a token must be requested for the key to be
%% updated, and the read and the write will be managed within the token
%% session. Only a single update will normally have access to the token,
%% requests will queue for use of the token. When a token cannot be secured
%% within a timeout, then the api_only method will be used.
%%
%% Future releases may support a mandate_token mode which will error on failure
%% to get a token, rather than proceed and accept eventual consistency (as with
%% prefer_token).
%%
%% Only conditional PUTs are impacted by this setting, non-conditional PUTs
%% being sent in parallel to conditional PUTs may cause siblings. The
%% HTTP standard headers of If-Unmodified-Since and If-Match, are always
%% applied as api_only checks. The riak-specific vector-clock based
%% if-not-modified header, and the HTTP-default if-none-match header are the
%% only conditional PUTs that will be subject to stronger, token-based
%% restrictions.
{mapping, "conditional_put_mode", "riak_kv.conditional_put_mode", [
{default, api_only},
{datatype, {enum, [api_only, prefer_token]}}
]}.

%% @doc Set the level of verification required on token access
%% When requesting access to a token, this can be done in three different
%% verification modes:
%% - head_only
%% - basic_consensus
%% - primary_consensus (default)
%%
%% The head_only mode will make the node currently at the head of each preflist
%% responsible for granting tokens in isolation. This is intended to meet
%% constraints only in healthy clusters (or single-node clusters).
%%
%% In a consensus mode, the token will be granted by the node at the head of
%% the preflist, and the issuance will be validated by up to two "downstream"
%% nodes. This means that when a node recovers from failure, and becomes head
%% of the preflist it is prevented from making grants which are duplicates of
%% ones made by a downstream node during the failure.
%%
%% There are two forms of consensus - basic and primary. With basic
%% consensus, any avaliable unique nodes in the preflist (either primary or
%% fallback) can be used for consensus. With primary consensus, the nodes
%% must be 3 of 5 primary nodes (and hence a target_n_val of at least 5 is
%% required in this mode).
%%
%% With basic_consensus, tokens can still be granted in a wide range of failure
%% scenarios, but with a risk of duplicate grants, in particular should a
%% cluster be partitioned.
%%
%% No mode provides strict guarantees, including primary_consensus, especially
%% in complex partition scenarios where different nodes have alternative views
%% of node reachability.
{mapping, "token_request_mode", "riak_kv.token_request_mode", [
{default, primary_consensus},
{datatype, {enum, [head_only, basic_consensus, primary_consensus]}}
]}.


%% @doc Controls which binary representation of a riak value is stored
%% on disk.
%% * 0: Original erlang:term_to_binary format. Higher space overhead.
Expand Down
79 changes: 0 additions & 79 deletions priv/tracers/tracer_accumulating_time.erl

This file was deleted.

Loading