Tutorial: OEMetadata - What is it and how do I use it?
Intoduction to OpenEnergyMetadata (OEM)Permalink
The development of tutorials for the Open Energy Family takes place publicly in a dedicated tutorial repository. Please report bugs and suggestions as new issues. If Jupyter Notebooks are new to you and you'd like to get an introduction, have a look at this less than 10 minute introduction video. Official installation instructions are available on jupyter's readthedocs page.
license: GNU Affero General Public License Version 3 (AGPL-3.0)
copyright: Reiner Lemoine Institut
- Understand what OEMetadata is and where it is applied
- How to use OEMetadata.
- Find helpful resources.
- Where to start contributing as a developer.
OpenenEnergyMetadata(OEM) is a standardised metadata format. OEM are used to provide relevant information about a dataset on the OEP "It´s data explaining the data :)". OEM are mandatory for oficially publishing data on the OEP, because this will clarify any licensing issues in advance and also provide context for the review that is part of every publishing process. You can find more information about publishing your data in thepublishing guidebook. There are descriptions on how to upload OEM on the Open Energy Platform itself with a wizard and a more technical description) for using the API in scrips. If a dataset has already been uploaded to the OEP, then the metadata can be viewed via the detailed view of a table. You just scroll down to the "Meta Information" section and there they are. The OEP also offers functions for creating, editing and downloading metadata. These functions can also be found on the detail view. Make sure to sign in to the OEP in advance to get writing access.
Compatible with frictionless dataPermalink
OEMetadata development is partially based on the specification of frictionless data. The advantage is that OEM is compatible with frictionless datapackage format and thus a solid basis for applicability is ensured. In the easiest case, a frictionless datapackage is a JSON-file that is placed directly next to the actual data - the former describing the latter.
The oemetadata is developed in JSON format and is published on GitHub under the MIT License. The current version of the JSON string can be found in the oemetadata repository. As an open source project, suggestions for improvement and forking are welcome and encouraged. Please take note of our development philosophy when you start contributing.
Structure of the OEMetadataPermalink
The OEMetadata are, generally speaking, divided into five sections:
- gernal information
- Describes the dataset and links to related data table on the OEP and potential further documentation.
- Documentation of all sources used with attribution and licenses.
- The license for the data, here an open license must be available.
- Information on who created/edited the metadata.
- data model
- Structure of the table in which the data is stored, also technical documentation of the table, requires basic database (SQL) knowledge.
There is a detailed overview in which all fields that occur in the metadata are described. You will also find sample values there.
Suggested workflow to get startedPermalink
There are several ways to create the OEMetadata. One way is to download the template JSON file and fill it manually in your text editor of choice. Another option is to use a tool on the OEP website called wizard. There, a table can be created, data can be uploaded from an excel or CSV file and the metadata can be filled in. The wizard is much more user friendly for beginners, so we recommend using the tool. Once the metadata has been created, in order to go into review it should be downloaded and and put in an inssue on github to start the review process. If you are unable to do this, create an issue and get in touch with a reviewer who will help you with those steps.
The metadata can be filled in to the best of your knowledge in a first step. If there are any difficulties, it is sufficient to create an issue on github to get in contact with the OEP developers. The issue template can be used for this purpose. Unfortunately at this stage of development a GitHub account is still required for this.