Schema language

The NOMAD Metainfo is the schema language used to define all data in NOMAD. This page is the authoritative reference for the language itself: the attributes you can put on a definition, the quantity types you can declare, and the conventions definitions should follow.

For task-oriented instructions, see How-to guides > Work with schemas. For the concepts behind the language, see Explanation > Data structure.

Quantity types

The type of a quantity determines what values it accepts, how they are validated and converted, and how they are serialized. Type names are case insensitive.

YAML spelling	Python spelling	Runtime type	Serialized form	Notes
string, str	str	str	JSON string
boolean, bool	bool	bool	JSON boolean
int	int	32-bit integer	JSON number	Bare int is 32-bit. Use np.int64 for wider values.
float	float	64-bit float	JSON number
complex	complex	complex	{"re": ..., "im": ...}
np.int8, np.int16, np.int32, np.int64	same	NumPy integer	JSON number or nested array
np.float16, np.float32, np.float64	same	NumPy float	JSON number or nested array
np.complex128	same	NumPy complex	{"re": ..., "im": ...}
Datetime	Datetime	datetime.datetime (UTC)	ISO 8601 string
{type_kind: Enum, type_data: [...]}	MEnum('a', 'b')	str	JSON string	Use type_data to list the allowed values.
JSON	JSON	dict	JSON object	Arbitrary nested JSON.
Bytes	Bytes	bytes	base64 string	The Python bytes type itself is not accepted; use the name.
URL	URL	str	JSON string
File	File	str	JSON string	A path relative to the upload.
Capitalized	Capitalized	str	JSON string	Capitalizes the first letter on assignment.
Any	Any	any	as-is	No validation.
User	User	NOMAD user	user id string
Author	Author	Author	JSON object
*<section name>*	MySection	section instance or MProxy	reference URL	See Work with schemas > Link data with references.
{type_kind: quantity_reference, type_data: <Section>/<quantity>}	MySection.my_quantity	the target quantity's value	reference URL ending in the quantity name	Assign the section that holds the quantity, not the value.

What you can write in type

Types are resolved by nomad.metainfo.data_type.normalize_type(), which accepts:

• Strings — 'string', 'boolean', 'json', 'datetime', 'url', 'file', 'any', 'capitalized', 'bytes', 'user', 'author', or any builtin name such as 'int' and 'float'.
• NumPy names — any np.<dtype> or numpy.<dtype> string, e.g. 'np.float64'.
• Python types — str, bool, int, float, complex, datetime.datetime. Note that the Python bytes type is not accepted, only the string 'bytes'.
• NumPy types — np.int16, np.float32, np.complex128, np.bool_, and np.dtype(...) instances.
• Section definitions — a section class or its m_def, making a section reference.
• Quantity definitions — MySection.my_quantity, making a quantity reference.
• Dictionaries — the {'type_kind': ..., 'type_data': ...} form produced when a definition is serialized. This is how MEnum and custom types round-trip, and also how references are spelled: {'type_kind': 'reference'} for a section and {'type_kind': 'quantity_reference'} for a quantity. The dictionary is the only way to write a quantity reference outside Python, because a bare string is always resolved as a section reference.

To add a type that is not in this list, see How-to guides > Develop the core software > Add a new type.

Shapes

The shape of a quantity is a list of dimensions. An empty list (or no shape) is a scalar, one dimension is a vector, two a matrix, and so on. Each dimension may be:

• an integer, for a fixed size — a 3x3 matrix is [3, 3];
• the string '*', for an arbitrary size — a list is ['*'];
• the name of a sibling quantity with an integer type, e.g. ['number_of_atoms', 3].

Note

shape works the same way for every quantity type, including reference types. For a reference type it describes the dimensionality of the references, not of the referenced data.

Value coercion

Values are converted to the declared type and unit on assignment, using NumPy and Pint. Assigning a Pint quantity extracts the magnitude and converts it to the declared unit.

The rule for array quantities is:

• if the type is a NumPy type, such as np.int32, the value becomes a NumPy array;
• if the type is a Python type, such as float or int, the value becomes a (nested) Python list.

Units are given as strings parsed by Pint — simple units or expressions, e.g. m, meter, mm, m/s, m/s**2. Uploaded schemas may use any unit; the built-in NOMAD schema uses SI units only. See Work with schemas > Work with units.

Reference forms

A reference quantity is serialized as a URL. An inter-entry reference has two parts, <entry>#<path>: a path or URL denoting the target entry, and a path within that entry's subsection hierarchy. The path usually ends at a section; for a quantity reference it ends at the target quantity's name. The host and path parts correspond to the NOMAD API.

Example reference	Meaning
#/data/processes/0	A section within the same archive.
#/data/instruments/0/name	A quantity within the same archive, i.e. a quantity reference.
/run/0/calculation/1	A section within the same archive (legacy form).
Instrument	A section definition in the same archive. Targets section definitions only.
nomad.datamodel.metainfo.workflow	A section definition written in Python as part of the NOMAD code. Targets section definitions only.
../upload/raw/data.archive.yaml#/data	A section in a different .archive.yaml file of the same upload.
../upload/archive/mainfile/data.archive.yaml#/data	A section in a processed archive, given by the entry mainfile.
../upload/archive/zxhS43h2kqHsVDqMboiP9cULrS_v#/data	A section in a processed archive, given by entry id.
../uploads/zxhS43h2kqHsVDqMboiP9cULrS_v/raw/data.archive.yaml#/data	A section in an entry of a different upload.
/entries/{entry_id}/archive#/data/processes/0	A section in a different entry on the same NOMAD installation.
/uploads/{upload_id}/archive/{entry_id}#/data/processes/0	The same, addressed by upload.
https://mylab.eu/oasis/api/v1/uploads/{upload_id}/raw/data.archive.yaml#/data	A section in a different NOMAD installation.

Writing type: Instrument or section: Instrument in a schema is itself a reference — a convenience form standing in for the otherwise cryptic #/definitions/sections/0. This is also what m_def is for: whenever the section definition cannot be worked out from the subsection that contains it, m_def names it explicitly.

Naming conventions

• Section definitions use UpperCamelCase, e.g. MySection, PvdEvaporation.
• Quantities and subsections use lower_snake_case, e.g. chamber_pressure, data_file.
• Prefer subsections over inheritance when adding specific quantities to a general section. For example, the workflow section contains a geometry_optimization subsection for the quantities that only apply to geometry optimizations.

These conventions apply equally to YAML and Python schemas. In Python they are not optional: a definition takes its name from the Python class or attribute name, so it must be a valid identifier.

Where definitions live

The nomad-lab package defines the Metainfo in three modules:

• nomad.metainfo — the schema language itself, including its self-referencing schema. This is the package rendered below.
• nomad.datamodel — the root section EntryArchive and the metadata section holding administrative metadata.
• nomad.datamodel.metainfo — the central, method-specific (but not parser-specific) definitions that are shared across parsers.

Serialization options

m_to_dict() converts a section instance to a plain dictionary. The options you are most likely to need:

Option	Effect
with_meta	Include m_def, and m_parent_index/m_parent_sub_section where applicable.
with_def_id	Include the definition id of each definition.
include_defaults	Include quantities that still hold their default value.
include_derived	Include derived quantities.
resolve_references	Replace references with what they point to — the target section, or the target value for a quantity reference — instead of reference URLs.
categories	Only include definitions in the given categories.
include, exclude	Filter properties by a predicate.
transform	Apply a function to every serialized value.

m_from_dict() performs the inverse, reconstructing a section tree from a dictionary and validating it against the schema. It is the mechanism behind the archive REST API.

Definition reference

The following is generated from the nomad.metainfo package, which defines the schema language in terms of itself. Section, Quantity and SubSection are the definitions you use to write a schema; Definition holds the attributes they all share.

Note

The more attribute is not listed below. It collects any additional keyword arguments passed to a definition, so that schemas can carry custom metadata. Category is retained for backwards compatibility but is deprecated — use annotations instead.

Definition

description: :class:Definition is the common base class for all metainfo definitions. All metainfo definitions (sections, quantities, subsections, packages, ...) share some common properties.

properties:

name	type
name	str	Each definition has a name. Names have to be valid Python identifier. They can contain letters, numbers and _, but must not start with a number. This also qualifies them as identifier in most storage formats, databases, makes them URL safe, etc. Names must be unique within the :class:Package or :class:Section that this definition is part of. By convention, we use capitalized CamelCase identifier to refer to sections definitions (i.e. section definitions are represented by Python classes), lower case snake_case identifier for variables that hold sections, and for properties (i.e. fields in a Python class) we typically use lower case snake_case identifier. Subsections are often prefixed with section_ to clearly separate subsections from quantities. Generally, you do not have to set this attribute manually, it will be derived from Python identifiers automatically.
label	str	Each definition can have an optional label. Label are like names, but do not have to adhere to the Python identifier syntax.
description	str	The description can be an arbitrary human-readable text that explains what a definition is about. For section definitions you do not have to set this manually as it will be derived from the classes doc string. Quantity and subsection descriptions can also be taken from the containing section class' doc-string Attributes: section.
links	str	Each definition can be accompanied by a list of URLs. These should point to resources that further explain the definition. shape=['0..*']
categories	Category	All metainfo definitions can be put into one or more categories. Categories allow to organize the definitions themselves. It is different from sections, which organize the data (e.g. quantity values) and not the definitions of data (e.g. quantities definitions). See :ref:metainfo-categories for more details. shape=['0..*'], default=[]
deprecated	str	If set this definition is marked deprecated. The value should be a string that describes how to replace the deprecated definition.
aliases	str	A list of alternative names. For quantities and subsections these can be used to access the respective property with a different name from its containing section. Package aliases will be considered when resolving Python references, e.g. in m_def. shape=['0..*'], default=[]
variable	bool	A boolean that indicates this property has variable parts in its name. If this is set to true, all capital letters in the name can be replaced with arbitrary strings. However, variable names work similar to aliases and can be considered on-demand aliases. Other aliases and the defined name will work as well. Thus, variable names are only resolved at runtime by the Python interface and are not directly serialized. However, the variable name is set in a meta attribute m_source_name automatically for properties (but not attributes). Variable names are only implemented for Quantity, SubSection, Attribute. default=False
more	nomad.metainfo.data_type.JSON	A dictionary that contains additional definition properties that are not part of the metainfo. Those can be passed as additional kwargs to definition constructors. The values must be JSON serializable. default=Complex object, default value not displayed.
all_attributes	nomad.metainfo.data_type.Any	A virtual convenient property that provides all attributes as a dictionary from attribute name to attribute. This includes meta attributes (starting with m_) that are defined for all properties of the same kind (sub_section or quantity).
attributes	Attribute	The attributes that can further qualify property values. sub-section, repeats

Attribute

description: Attributes can be used to qualify all properties (subsections and quantities) with simple primitive values.

inherits from: Definition

properties:

name	type
type	nomad.metainfo.metainfo.QuantityType	The type of the attribute. Needs to be a primitive type that is a subclass of Datatype.
shape	nomad.metainfo.data_type.Dimension	The shape of the attribute. Need to be a list, similar to the shape of a quantity. shape=['0..*'], default=[]

Property

description: A common base-class for section properties: subsections and quantities.

inherits from: Definition

Section

description: Instances of the class :class:Section are created by writing Python classes that extend :class:MSection like this:

.. code-block:: python

class SectionName(BaseSection):
    ''' Section description '''
    m_def = Section(**section_attributes)

    quantity_name = Quantity(**quantity_attributes)
    sub_section_name = SubSection(**sub_section_attributes)

We call such classes section classes. They are not the section definition, but just representation of it in Python syntax. The section definition (in instance of :class:Section) will be created for each of these classes and stored in the m_def property. See :ref:metainfo-reflection for more details.

Most of the attributes for a :class:Section instance will be set automatically from the section class:

inherits from: Definition

properties:

name	type
base_sections	Section	shape=['0..*'], default=[]
extending_sections	Section	shape=['0..*'], default=[]
extends_base_section	bool	default=False
inheriting_sections	Section	shape=['0..*'], default=[]
constraints	str	shape=['0..*'], default=[]
event_handlers	nomad.metainfo.data_type.Callable	Event handler are functions that get called when the section data is changed. There are two types of events: set and add_sub_section. The handler type is determined by the handler (i.e. function) name: on_set and on_add_sub_section. The handler arguments correspond to meth:MSection.m_set (section, quantity_def, value) and meth:MSection.m_add_sub_section (section, sub_section_def, sub_section). Handler are called after the respective action was performed. This quantity is automatically populated with handler from the section classes methods. If there is a method on_set or on_add_sub_section, it will be added as handler. shape=['0..*'], default=[]
inherited_sections	nomad.metainfo.data_type.Any	A helper attribute that gives direct and indirect base sections and extending sections including this section. These are all sections that this sections gets its properties from.
all_base_sections	nomad.metainfo.data_type.Any	A helper attribute that gives direct and indirect base sections.
all_inheriting_sections	nomad.metainfo.data_type.Any	A helper attribute that gives direct and indirect inheriting sections.
all_properties	nomad.metainfo.data_type.Any	A helper attribute that gives all properties (subsection and quantity) definitions including inherited properties and properties from extending sections as a dictionary with names and definitions.
all_quantities	nomad.metainfo.data_type.Any	A helper attribute that gives all quantity definition including inherited ones and ones from extending sections as a dictionary that maps names (strings) to :class:Quantity.
all_sub_sections	nomad.metainfo.data_type.Any	A helper attribute that gives all subsection definition including inherited ones and ones from extending sections as a dictionary that maps names (strings) to :class:SubSection.
all_sub_sections_by_section	nomad.metainfo.data_type.Any	A helper attribute that gives all subsection definition including inherited ones and ones from extending sections as a dictionary that maps section classes (i.e. Python class objects) to lists of :class:SubSection.
all_aliases	nomad.metainfo.data_type.Any	A helper attribute that gives all aliases for all properties including inherited properties and properties form extending sections as a dictionary with aliases and the definitions.
all_inner_section_definitions	nomad.metainfo.data_type.Any	A helper attribute that gives all inner_section_definitions including their aliases by name.
has_variable_names	nomad.metainfo.data_type.Any
path	nomad.metainfo.data_type.Any	Shortest path from a root section to this section. This is not the path in the metainfo schema (m_path) but an archive path in potential data.
quantities	Quantity	sub-section, repeats
sub_sections	SubSection	sub-section, repeats
inner_section_definitions	Section	sub-section, repeats

Package

description: Packages organize metainfo definitions alongside Python modules Each Python module with metainfo Definition (explicitly or implicitly) has a member m_package with an instance of this class. Definitions (categories, sections) in Python modules are automatically added to the module's :class:Package. Packages are not nested and rather have the fully qualified Python module name as name.

This allows to inspect all definitions in a Python module and automatically puts module name and docstring as :class:Package name and description.

Besides the regular :class:Definition attributes, packages can have the following attributes:

inherits from: Definition

properties:

name	type
all_definitions	nomad.metainfo.data_type.Any	A helper attribute that provides all section and category definitions by name and aliases.
dependencies	nomad.metainfo.data_type.Any	All packages that provide definitions needed by this package. Being 'needed' includes categories, base sections, and referenced definitions.
section_definitions	Section	All section definitions in this package as :class:Section objects. sub-section, repeats
category_definitions	Category	All category definitions in this package as :class:Category objects. sub-section, repeats

normalization without further documentation

Category

description: Categories allow to organize metainfo definitions (not metainfo data like sections do) Each definition, including categories themselves, can belong to a set of categories. Categories therefore form a hierarchy of concepts that definitions can belong to, i.e. they form a is a relationship.

inherits from: Definition

Quantity

description: To define quantities, instantiate :class:Quantity as a class attribute values in a section classes. The name of a quantity is automatically taken from its section class attribute. You can provide all other attributes to the constructor with keyword arguments

See :ref:metainfo-sections to learn about section classes. In Python terms, Quantity is a descriptor. Descriptors define how to get and set attributes in a Python object. This allows us to use sections like regular Python objects and quantity like regular Python attributes.

Each quantity must define a basic data type and a shape. The values of a quantity must fulfil the given type. The default shape is a single value. Quantities can also have physical units. Units are applied to all values.

inherits from: Property

properties:

name	type
type	nomad.metainfo.metainfo.QuantityType
shape	nomad.metainfo.data_type.Dimension	shape=['0..*'], default=[]
unit	nomad.metainfo.data_type.Unit
dimensionality	str
default	nomad.metainfo.data_type.Any
derived	nomad.metainfo.data_type.Callable	A Python callable that takes the containing section as input and outputs the value for this quantity. This quantity cannot be set directly, its value is only derived by the given callable. The callable is executed when this quantity is get. Derived quantities are always virtual.
cached	bool	A bool indicating that derived values should be cached unless the underlying section has changed. default=False
virtual	bool	A boolean that determines if this quantity is virtual. Virtual quantities can be got/set like regular quantities, but their values are not (de-)serialized, hence never permanently stored. default=False
is_scalar	bool
use_full_storage	bool	A derived boolean that indicates if this quantity should be stored in full storage mode. It will be set to True if flexible_unit is True, or variable is True, or it has attributes.
flexible_unit	bool	A boolean to indicate if this quantity may have a unit that is not the default unit. In this case, the quantity will be stored in full storage mode as a MQuantity. default=False

SubSection

description: Like quantities, subsections are defined in a section class as attributes of this class. Unlike quantities, each subsection definition becomes a property of the corresponding section definition (parent). A subsection definition references another section definition as the subsection (child). As a consequence, parent section instances can contain child section instances as subsections.

Contrary to the old NOMAD metainfo, we distinguish between subsection the section and subsection the property. This allows to use on child section definition as subsection of many parent section definitions.

inherits from: Property

properties:

name	type
sub_section	Section	A :class:Section or Python class object for a section class. This will be the child section definition. The defining section the child section definition.
repeats	bool	A boolean that determines whether this subsection can appear multiple times in the parent section. default=False
key_quantity	str