Obelisk Core

Obelisk is a service platform for building scalable data-driven applications and services, connecting data producers and consumers with powerful APIs for ingesting, querying and streaming large amounts of data in an organized and secure way.

Metadata model

Namespaces

On the top level of the Obelisk metadata structure, there are namespaces. Each user has his/her own private namespace by default. You can always create more namespaces though. They can be used to conceptually bundle datasets, for instance for an organization or a project. Namespaces can be nested, thus acting a bit like folders in a filesystem.

Datasets

Inside a namespace you can create datasets. A dataset can hold multiple series of data. When data is ingested to the system, it is placed under the given (namespace and) dataset. When that same data is queried, it is also queried from this dataset.

Series

Multiple series are present under a dataset. A series is defined by every label permutation. Data ingested to a dataset can result in multiple series being created/available to query.

Labels

A label is a key:value pair, that can be added to the data that is ingested. These labels are treated as a sort of index, so they are important for efficient retrieval while querying. You can filter/sort/target specific labels values.

Membership

Users can be granted membership of namespaces and/or datasets. Membership of a namespace implies membership of all datasets and memberships below that namespace.

Data model

Data must be ingested as JSON Arrays of JSON objects. These JSON objects must follow the following format.

Timestamp

Optionally you can add the timestamp representing this measurement point. This is done in as a string following the ISO 8601 datetime format. If you omit this field, the timestamp recorded will default to the server time at the moment of ingesting the data.

MetricName

The metric name suffix (part behind the ::) must be of type number, bool, String, JSON or number[].

Value type

Make sure the value is written as an instance of the type that is noted in your MetricName's suffix!

LabelMap

You can add as many labels if you want, but do mind the cardinality of your dataset! If the ingested data requires no labels, you can omit this key and its value.

Location

Optionally a location can be added to each point. A location is a JSON object with at least a latitutde and longitude key-value pair. Optionally an elevation key-value pair can also be added. If the ingested data requires no location, you can omit this key and its value.