Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Initial draft. #1

Merged
merged 4 commits into from
May 13, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file added _static/test_cycle.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
92 changes: 88 additions & 4 deletions index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,93 @@ Night Planning Tool

.. abstract::

This technote describes the design of an integrated night planning tool. This is a tools that allows our team to collect a group of Jira tickets that describes a set of tests, allows us to organize these into a coherent list of tests to execute, that can later be loaded and executed at the summit.
This technote describes the design of an integrated night planning tool.
This is a tools that allows our team to collect a group of Jira tickets describing a set of tests/observations, allows us to organize these into a coherent list that can later be loaded and executed at the summit.
Copy link

@b1quint b1quint May 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typo

"This is a tool that allows ..."

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

... "tests/observations, allows us to organize" ...

Consider splitting the sentence here.


Add content here
================
Introduction
============

Test planning and execution for commissioning the Vera Rubin observatory has revolved around using Jira and Confluence to generate a comprehensive plan that can be followed by the night crew.
This process involves a combination of Jira projects:

- SITCOM are used to capture high level tests/verification activities.
They are usually connected to one or more requirement or some other commissioning activity.
For example, SITCOM-1099 was created to capture the tests required to validate the operation of the system with the TMA dynamic settings going from 5% all the way to 100% of its requirements.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Weak suggestion - convert SITCOM-1099 into a link.


- DM tickets are used to capture work needed either by a regular contruction deliverable or a commissioning activity.
DM tickets can be linked to SITCOM tickets when there is work required to accomplish a particular test, but they can also be standalone ticket that are just broadly required.
For instance, in the example above of SITCOM-1099, we have DM-39533 linked as a related ticket.
It is not strickly necessary to have this relantionship as DM-39533 represents a broader system functionality, however the issues are linked to demonstrate the relantionship.
Sometimes a SITCOM ticket might trigger the development of systewide functionality and the relantionship is used to indicate it.

- BLOCK tickets are used to design, plan, execute and track activities at the summit.
Usually a BLOCK ticket will be linked to a SITCOM ticket.

Up to now, planning activities at the summit have revolved around collecting the BLOCK tickets that will be executed at night and sorting them out in a table with some information about priority and order of execution.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe add a link to one of the night plans/logs for a practical example? Here is one for convenience:
https://confluence.lsstcorp.org/display/LSSTCOM/2024-04-23+Simonyi+Night+Log

Later, the summit crew adds information to these tables about the execution and are also required to do more detailed logging in LOVE/OLE.
Gathering information later about task execution and performing traceability under these conditions is taxing.
Users have to peruse the night logs for detailed information and also check the confluence page with the night plan.

In order to improve overal user experience on both sides of the isle; creating and executing the plan, and also to improve metadata extraction for traceability, we propose to introduce a set of new procedures and tools for planning and execution of the summit activities.

Planning with Zephyr
====================

We will adopt Jira's Zephyr application to create test cases, test cycles and test plans to organize how tests will be executed and planned.

- BLOCK tickets will be used to describe what the block does, link it to a particular SITCOM ticket and will also aggregate the block configuration.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we adopt Zephyr, I recommend that we limit BLOCK tickets for those who will have JSON files or scheduler configurations. I don't see why we need to create a BLOCK ticket and a test case if that does not favor automatization/scripting.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure about this. For example, see the doc page that Erik is working on describing the BLOCK project workflow. I think there is some value to having a BLOCK ticket. However, this might be a good discussion to have with the rest of the team.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we have a couple of examples here? I believe, the way this is now, it is too abstract for a discussion.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think Erik's document would be a better place for this information. I am going to link Erik's document here and will add an item in the FAQ to talk about this.


- Test Cases will have a step by step description of how to execute a particular BLOCK.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Including instructions to setup/wrap-up.


- Test Cycles will be used to plan a particular night.
They contain a list of Test Cases, ordered in the same way they should be executed.
The Environment field is used to separate the Test Cases that will be executed at different parts of the day; Daytime, Afternoon, Early Night, Late Night, etc.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The time periods are something that I would like to agree on and have it set in stone.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

agree... let's discuss that in the next meeting.


- Test Execution are created for each Test Case to track a particular execution.

- Test Plans contains a collection of Test Cycles and are used for long term planning.

.. figure:: /_static/test_cycle.png
:name: fig-test-cycle
:target: ../images/test_cycle.png
:alt: Test Cycle

An example Test Cycle showing the planned Test Executions for the Test Cases scheduled for a particular night.
The Test Executions are sorted in the order they should be executed, sub-divided in two different parts of the day; Late Afternoon and Early Night.

Integration with LOVE
=====================

As an initial fallback mode, we expect the summit crew would be able to interact directly with the Test Cycle in Jira to execute a night plan.
However we have the potential to considerably improve the user experience by integrating the Test Cycle with LOVE.

The idea would be to provide an interface in LOVE similar to that of the Jira Test Player.
Through this interface users would be able to:

- Select the plan for the night.

This could be a drop down menu that shows the list of Test Cycles from Jira.

Uppon selection of a particular Test Cycle, users would be presented with the list of Test Executions, ordered like they appear in the Test Cycle.

- Select a Test Execution from the night plan.

This would present the users with the Test Player interface, showing information about the particular tests and the steps that need to be executed.

- Start and run a Test Execution.

If the test steps are correctly configured with the required script information, users will be able to send these to the ScriptQueue directly from the test step.
The test step will also monitor the relevant information and back feed that into the Test Execution.
This means that, if a block fails to execute, the test step will be marked as failed.

- Log information from a step execution and or the Test Execution.

Each step in the Test Execution will also have an input text box that can be used to log entries in OLE.
The system should use metadata provided in the Test Execution to augment the narrative log entry with additional labels, etc.

The high level Test Execution will also have an input text box to create log entries.

- Feed the information back to Jira.

All metadata generated at the summit by this tool will be fed back to jira to allow for better traceability.

See the `Documenteer documentation <https://documenteer.lsst.io/technotes/index.html>`_ for tips on how to write and configure your new technote.
Loading