Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: expand the reference documentation #169

Merged
merged 18 commits into from
Aug 30, 2024
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions docs/capture_events.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
scenario.capture_events
=======================

.. automodule:: scenario.capture_events
4 changes: 4 additions & 0 deletions docs/consistency_checker.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
scenario.consistency_checker
============================

.. automodule:: scenario.consistency_checker
5 changes: 5 additions & 0 deletions docs/context.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
scenario.Context
================

.. automodule:: scenario.context
:members: DEFAULT_JUJU_VERSION, InvalidEventError, ContextSetupError, AlreadyEmittedError
4 changes: 2 additions & 2 deletions docs/custom_conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -306,10 +306,10 @@ def _compute_navigation_tree(context):
# ('envvar', 'LD_LIBRARY_PATH').
nitpick_ignore = [
# Please keep this list sorted alphabetically.
('py:class', 'AnyJson'),
('py:class', '_CharmSpec'),
('py:class', '_Event'),
('py:class', 'scenario.state._DCBase'),
('py:class', '_EntityStatus'),
('py:class', 'ModelError'),
('py:class', 'scenario.state._EntityStatus'),
('py:class', 'scenario.state._Event'),
('py:class', 'scenario.state._max_posargs.<locals>._MaxPositionalArgs'),
Expand Down
29 changes: 9 additions & 20 deletions docs/index.rst
Original file line number Diff line number Diff line change
@@ -1,32 +1,21 @@

Scenario API reference
======================

.. toctree::
:maxdepth: 2
:maxdepth: 1
:caption: Contents:

scenario.State
==============

.. automodule:: scenario.state


scenario.Context
================

.. automodule:: scenario.context

scenario.consistency_checker
============================

.. automodule:: scenario.consistency_checker
context
state
capture_events
consistency_checker


scenario.capture_events
=======================
scenario
========

.. automodule:: scenario.capture_events
.. automodule:: scenario
:special-members: __call__


Indices
Expand Down
2 changes: 1 addition & 1 deletion docs/requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ mdurl==0.1.2
# via markdown-it-py
myst-parser==2.0.0
# via ops-scenario (pyproject.toml)
ops==2.12.0
ops==2.15.0
# via ops-scenario (pyproject.toml)
packaging==24.0
# via sphinx
Expand Down
5 changes: 5 additions & 0 deletions docs/state.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
scenario.State
==============

.. automodule:: scenario.state
:members: ATTACH_ALL_STORAGES, CREATE_ALL_RELATIONS, BREAK_ALL_RELATIONS, DETACH_ALL_STORAGES, ACTION_EVENT_SUFFIX, BUILTIN_EVENTS, FRAMEWORK_EVENTS, PEBBLE_READY_EVENT_SUFFIX, PEBBLE_CUSTOM_NOTICE_EVENT_SUFFIX, PEBBLE_CHECK_FAILED_EVENT_SUFFIX, PEBBLE_CHECK_RECOVERED_EVENT_SUFFIX, RELATION_EVENTS_SUFFIX, STORAGE_EVENTS_SUFFIX, SECRET_EVENTS, META_EVENTS, DEFAULT_JUJU_DATABAG, AnyJson, AnyRelation, CharmType, JujuLogLine, MetadataNotFoundError, PathLike, Port, RawDataBagContents, RawSecretRevisionContents, RelationBase, UnitID, next_action_id, next_notice_id, next_relation_id, next_storage_index
56 changes: 54 additions & 2 deletions scenario/__init__.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,60 @@
#!/usr/bin/env python3
# Copyright 2023 Canonical Ltd.
# See LICENSE file for licensing details.

"""Charm state-transition testing SDK for Operator Framework charms.

tonyandrewmeyer marked this conversation as resolved.
Show resolved Hide resolved
Write unit tests that declaratively define the Juju state all at once, define
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So we promote scenario as another way to do unit tests, then? We should sync with Charmcraft to make sure their profiles create a directory structure for tests that is aligned.

Copy link
Collaborator Author

@tonyandrewmeyer tonyandrewmeyer Aug 29, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We're (imminently) going to be promoting Scenario as the preferred way to do unit tests. I also tried to have these work if you were reading them when you'd done pip install ops-scenario but also if you had done pip install ops[testing] and were getting all the classes from the ops.testing namespace. This is also why all the references to the name "scenario" in the docs are gone.

I do have a ticket for updating the charmcraft profiles. I need to get all the bits in places first, but it's definitely in my definition-of-done.

the Juju context against which to test the charm, and fire a single event on the
charm to execute its logic. The tests can then assert that the Juju state has
changed as expected.

These tests are somewhere in between unit and integration tests: they could be
tonyandrewmeyer marked this conversation as resolved.
Show resolved Hide resolved
called 'functional' or 'contract', and most properly are 'state-transition'.
However, for simplicity, we refer to them as 'unit' tests in the charm context.

tonyandrewmeyer marked this conversation as resolved.
Show resolved Hide resolved
Writing these tests should nudge you into thinking of a charm as a black-box
input->output function. The input is the union of an `Event` (why am I, charm,
being executed), a `State` (am I leader? what is my relation data? what is my
config?...) and the charm's execution `Context` (what relations can I have? what
containers can I have?...). The output is another `State`: the state after the
charm has had a chance to interact with the mocked Juju model and affect the
state.

.. image:: https://raw.githubusercontent.com/canonical/ops-scenario/main/resources/state-transition-model.png
:alt: Transition diagram, with the input state and event on the left, the context including the charm in the centre, and the state out on the right

Writing unit tests for a charm, then, means verifying that:

- the charm does not raise uncaught exceptions while handling the event
- the output state (as compared with the input state) is as expected.

A test consists of three broad steps:

- **Arrange**:
- declare the context
- declare the input state
- **Act**:
- select an event to fire
- run the context (i.e. obtain the output state, given the input state and the event)
- **Assert**:
- verify that the output state (as compared with the input state) is how you expect it to be
- verify that the charm has seen a certain sequence of statuses, events, and `juju-log` calls
- optionally, you can use a context manager to get a hold of the charm instance and run
assertions on APIs and state internal to it.

The most basic scenario is one in which all is defaulted and barely any data is
available. The charm has no config, no relations, no leadership, and its status
tonyandrewmeyer marked this conversation as resolved.
Show resolved Hide resolved
is `unknown`. With that, we can write the simplest possible test:

.. code-block:: python

def test_base():
ctx = Context(MyCharm)
out = ctx.run(ctx.on.start(), State())
assert out.unit_status == UnknownStatus()
"""

from scenario.context import Context, Manager
from scenario.state import (
ActionFailed,
Expand All @@ -22,7 +76,6 @@
Network,
Notice,
PeerRelation,
Port,
Relation,
Resource,
Secret,
Expand Down Expand Up @@ -58,7 +111,6 @@
"Address",
"BindAddress",
"Network",
"Port",
"ICMPPort",
"TCPPort",
"UDPPort",
Expand Down
30 changes: 15 additions & 15 deletions scenario/capture_events.py
Original file line number Diff line number Diff line change
Expand Up @@ -25,23 +25,23 @@ def capture_events(
include_framework=False,
include_deferred=True,
):
"""Capture all events of type `*types` (using instance checks).

Arguments exposed so that you can define your own fixtures if you want to.
"""Capture all events of type ``*types`` (using instance checks).

Example::
>>> from ops.charm import StartEvent
>>> from scenario import Event, State
>>> from charm import MyCustomEvent, MyCharm # noqa
>>>
>>> def test_my_event():
>>> with capture_events(StartEvent, MyCustomEvent) as captured:
>>> trigger(State(), ("start", MyCharm, meta=MyCharm.META)
>>>
>>> assert len(captured) == 2
>>> e1, e2 = captured
>>> assert isinstance(e2, MyCustomEvent)
>>> assert e2.custom_attr == 'foo'

from ops.charm import StartEvent
from scenario import Event, State
from charm import MyCustomEvent, MyCharm # noqa

def test_my_event():
ctx = Context(MyCharm, meta=MyCharm.META)
with capture_events(StartEvent, MyCustomEvent) as captured:
ctx.run(ctx.on.start(), State())

assert len(captured) == 2
e1, e2 = captured
assert isinstance(e2, MyCustomEvent)
assert e2.custom_attr == 'foo'
"""
allowed_types = types or (EventBase,)

Expand Down
Loading
Loading