Ontology Metrics

Ontology Metrics in BioPortal

This page describes the metrics that BioPortal calculates for the ontologies in its repository. In general, BioPortal will calculate the metrics when the ontology is uploaded and will store it as part of the Ontology Metadata. There are two groups of metrics:

• Statistical metrics
• Quality-control and quality-assurance metrics

Each ontology may have all, some, or no values filled in for its metrics. Some metrics are meaningful only for ontologies in a specific representation language (e.g., number of axioms matters only for OWL ontologies; there are no individuals to count in the ontologies in OBO format).

Statistical Metrics

We calculate the following statistics on each ontology in BioPortal:

• number of classes: the number of classes in the ontology. For OWL ontologies
  • does not include imported classes
  • includes only named classes (no anonymous classes in this count)
• number of properties: the number of properties or slots in the ontology.
• number of individuals: the number of individuals in the ontology. Note:
  • Individuals exist only in OWL and Protege ontologies.
  • An individual in OWL is a resource that is neither a class, nor a property.
• number of axioms: the number of axioms in an OWL ontology.
• maximum depth: the maximum depth of the hierarchy tree.
  • For OWL, RDFS, Protege, and RRF ontologies, we consider only the is-a relationship as a hierarchical relationship
  • For OBO format ontologies, we consider the following relationships as hierarchical relationships: is-a, has-part, inverse of develops-from
• average number of siblings: the average number of siblings at one level in the tree.
• maximum number of siblings: the maximum number of siblings in the ontology.

Quality-Control and Quality-Assurance Metrics

We calculate the following metrics that may give some indication of the quality of the ontology and help ontology authors improve the quality.

• classes with only one subclass: a list of classes that have only one subclass in the is-a hierarchy. While technically there is no problem in having only one subclass, this situation often indicates that either the hierarchy is under-specified, or the distinction between the class and the subclass is not appropriate.
• classes with too many subclasses: a list of classes that have more direct subclasses than a certain threshold. The specific threshold is an internal parameter in BioPortal (currently set to 25). This situation may be an indication that additional distinctions and categorization are needed.
• classes with no definition: a list of classes that have no value for the definition property. For ontologies in OBO and RRF formats, the property for definition is part of the language. For OWL ontologies, the authors specify this property as part of the ontology metadata (the default is skos:definition). If the ontology author has not specified any property as a definition property, we do not fill out this value.
• univocality: the consistency of expression of concepts in an ontology (developed by Karin Verspoor∗, Daniel Dvorkin, K. Bretonnel Cohen, and Lawrence E. Hunter, University of Colorado)

Auditing and other information

• auditing information: a set of <author, classes> pairs that indicates which lists the classes that each author contributed to the ontology. We use the property that the ontology developers specify in their metadata to determine who is the author of the class. If the ontology author has not specified any property as a definition property, we do not fill out this value.

Please, contact us at support@bioontology.org, if there are additional metrics of any kind that you would like to see.

Generating metrics

At the moment, we do not generate metrics as part of the ontology load process, but we will shortly. The process will be as follows: When BioPortal loads and parses a version of an ontology, it then starts a process that calculates the metrics and stores the values as part of the metadata about that ontology version.

Metrics computation and ontology metadata

Metric generation relies on the following information being specified in the ontology metadata:

For OBO and RRF ontologies:

• property used to specify the authors of a class

For OWL ontologies:

• property used to specify preferred name (default rdfs:label, and if not specified, then rdf:ID)
• property used to specify synonyms (default skos:altLabel)
• property used to specify the definition of a class (default skos:definition)
• property used to specify the author of a class (default dc:creator)

Representing metrics in BioPortal

The metrics in BioPortal are stored as part of the ontology metadata. Specifically, each instance of the OMV:Ontology class, which describes the metadata about a single ontology version, has a number of properties that store the metrics. Some of these properties are imported from the Ontology Metadata Vocabulary, OMV (the omv: prefix) and the rest are defined locally in the BioPortal Metadata Ontology (the metrics: prefix). These properties include:

• OMV:numberOfClasses
• OMV:numberOfProperties
• OMV:numberOfAxioms
• OMV:numberOfIndividuals
• metrics:averageNumberOfSiblings
• metrics:maximumNumberOfSiblings
• metrics:classesWithNoDocumentation
• metrics:maximumDepth
• metrics:preferredMaximumSubclassLimit
• metrics:classesWithSingleSubclass
• metrics:classesWithNoAuthor
• metrics:classesWithMoreThanXSubclasses
• metrics:classesWithMoreThanOnePropertyValueForPropertyWithUniqueValue

Accessing metrics in BioPortal

BioPortal users will be able to access ontology metrics in two ways:

• through the BioPortal user interface, as part of the Ontology Details tab (not implemented yet)
• through a dedicated REST service that returns a MetricsBean

REST service access

Applications can use the following REST service to access metrics information for a specific ontology version. Note: there is no version of this service that uses virtual ontology id yet.

• Signature: ./ontologies/metrics/{ontologyVersionId}?email={email_address}
• Example: http://rest.bioontology.org/bioportal/ontologies/metrics/40616?email=example@example.org

The service returns a ontologyMetricsBean that contains the version id for the ontology and the values for the metrics that have some values computed for them:

<success>
  <accessedResource>/bioportal/ontologies/metrics/40616</accessedResource>
  <accessDate>2009-10-15 17:43:25.978 PDT</accessDate>
  <data>
    <ontologyMetricsBean>
      <id>40616</id>
      <numberOfClasses>1577</numberOfClasses>
      <numberOfIndividuals>74</numberOfIndividuals>
      <numberOfProperties>72</numberOfProperties>
    </ontologyMetricsBean>
  </data>
</success>