Writing Competency Questions

Competency Questions

THIS PAGE IS WORK IN PROGRESS

The value of an ontology is carried by its axioms. They concretize its terms, definitions and relationships. So, in order to objectively assert the quality of an ontology, a systematic and objective assessment of the existence (completeness) and consistency (soundness) of these axioms is necessary. Competency Questions(CQ) offer a framework to define the requirements of an ontology AND the means of evaluating them against these requirements. Through this guide we will learn how to write a CQ that is useful for the quality assessment of the OEO.

Types of Competency Questions

In our context we will separate the competency questions in two kinds: Completeness competency questions and Soundness competency questions¹. The former comprises the majority of the CQs, for an ontology to be complete it has to address all the axioms within its requirements. These requirements are concretized through the questions and formalized through the ontology. The latter kind consist of CQs that ensure the consistency of the ontology. The main characteristic of soundness questions is that they are to be answered negatively. For example, the question "Is coal renewable?" should have a negative answer. One could go different ways to evaluate these assertions, like inverting the expression to "Is coal not renewable?" but since the objective of these questions is to catch contradictions sticking to positive assertions can make it easier add bugs reactively.

Steps to Write a Competency Question

1. Natural Language Question

The first appearance of a competency question should be as a natural language query and encompass the requirement of the ontology one wants to challenge. For example "Is natural gas a renewable fuel?" should have an answer within the ontology as it queries an axiom likely to be part of its requirements. This initial question does not imply any kind of ontological commitment ². This means that if we ask "Does a combined cycle gas turbine consume Hydrogen?" it does not mean that the ontology needs to have the concept or property for consume in the first place, there is probably a different property that has the same meaning, but this should not stop you from asking the question.

2. Identify Existing Ontology Entities

From now on your CQ implementation can go three ways. Either the terms are already completely in the ontology, they are not there or they are only partially defined. Depending on what is the case you will need to use place-holders to define your new terms before implementing them.

3a. Concretize the Question in OWL

3b. Write the Question in OWL with Placeholders

4. Implement

Inclusion Approaches

No approach should be preferred, they work in conjunction.

Proactive Inclusion

Proactive inclusion consist in actively looking for existing holes in the testing framework and fix them.

Reactive Inclusion

Reactive inclusion consists in the inclusion of questions in response to arising issues/bugs.

¹ I have found no sources for this division, so if you have one please refer it to me: @areleu .

² The ontological commitments of a theory are the entities or kinds of entity that must exist in order for the theory to be true.