Concepts
Get a greater understanding of everything OpenDataDSL
This section helps you to understand the basic concepts of what OpenDataDSL is, how it works and how it can help you.
What is OpenDataDSL?
OpenDataDSL is a concatenation of the 3 fundamental aspects of what the platform is:
- Open
OpenDataDSL is an open and transparent data management platform
- Data
Data management is the primary focus of the platform
- DSL
Domain Specific Language - ODSL is our 4GL language which is used throughout the platform
Architecture
The diagram below shows the functional architecture of the platform.
Architecture notes
- All the layers that not 'ON PREM' are based in the Microsoft Azure Cloud.
- All access to the platform is secured using your own Azure Active Directory
- The process layer is handled by a scalable cluster of servers using Azure Batch
Data Location
All data is stored in MongoDB Atlas - a cloud based data service.
There are 2 (sometimes more) sources for data:
- Public
This is data that is freely distributable and accessible by everyone who uses the platform
- Private
This is your own private data which is only accessible to users on your Azure AD tenant (your company)
Your proprietary private data is stored in your own dedicated database.
Components and Terminology
This section gives a brief summary of all the components and related terminology within OpenDataDSL.
Master Data or Objects in OpenDataDSL are documents that contain descriptive information and properties which represent a specific product, entity or resource.
- An FX currency pair from a specific data provider
- A futures product on a commodity exchange
- A physical power generation plant
- A weather station
A timeseries is attached to master data and represents a value or metric which is recorded or observed at a point-in-time. Timeseries usually have a calendar which defines the interval size of the timeseries.
- The daily FX rates for an FX currency pair
- The temperature reading at a certain location
- The GDP of a country
- Price ticks from an exchange
A curve is a structure that represents values at a point-in-time that reference a future time period. Those values can either be:
- The actual value today for a future delivery period
- A forecast value made today for a future period
- Futures on a commodity exchange - forward curve
- Weather forecast
A matrix is a table of values that relate 2 items together. The value in a matrix is usually a calculation or statistic.
- A correlation matrix
- A covariance matrix
- A set of statistics (min/max/mean/stdev etc.) about various products
Events are individual observations for a single time-point. An event has a reference time and a start and end time as well as values and properties.
- FX currency pairs for a single day
- All prices for a single futures contract delivery for 1 day
- An individual trade (price and volume)
- A recorded metric such as temperature
A report can be any data in any structure that is generated and stored for a specific date. A report can also have a template which defines the way the report is displayed using charts, tables or representation as a specific file format (xml,csv,json etc.)
- A price/volume report
- An aggregated curve
- A newsletter or documentation
- An file to be exported to another system in csv
You use subscriptions to trigger an action to be performed after something has been updated. Subscriptions are the mechanism that makes OpenDataDSL event-driven.
- Build a smart curve
- Build a report
- Send an email
- Send the data to a message queue
A queue is a conduit for messages which are sent from the platform. A queue can be listened to by an internal application which can retrieve the data and perform tasks with the data.
- Push input data in order to build a curve using your own language
- Push data to be fed into an ETRM system