This post defines what both the Common Data Service (CDS) and the Common Data Model (CDM) are. The Common Data Service and the Common Data Model are core concepts in Microsoft Business Applications and the whole PowerPlatform, so it’s important to know what they are if your planning on or already in this area. This post aims to do this in a simple and easy to understand way.
The Common Data Service aims to be a single place for data used in interactive applications across an organisation. It achieves this in a few different ways. Read on to find out how!
Relational Databases are a very common type of storage which is used across organisations. A relational database holds ‘Tables’ of data. A table contains multiple columns and rows. A table has a primary key. The key is unique across ‘rows’ within the table, which is a record within a table, containing data related to the table’s columns. A ‘column’ is a defined attribute on the table. An example can be a table called Cars. The table contains columns ‘ID’ ‘Colour’, ‘Make’ and ‘Model’. There would be rows of data with different values in the columns for that row, depending on the data of the car.
Relationships can occur between the table using foreign keys. Foreign Keys are links from one table to another. Relationships are links between tables, and one thing that relational databases do, is they maintain the integrity of those relationships through the operational processing of the data.
The schema of the relational database has to be pre-defined before data can start to be added to it. It is then fixed and cannot be changed without effort and potential data loss.
Examples include MySQL and Azure SQL.
So… non-relational databases?
Databases don’t have to be relational, using the table and column standard. They can also contain simple flat files, key-value pairs, document type storage (JSON!) and more. Non-relational databases are more specific in what they store i.e. JSON and how that data is accessed, meaning it can be optimised for that specific type. The data stored in these databases can be XML, JSON, or another format. They offer a flexible approach for storing data which doesn’t have to be related, it could be related, and it doesn’t have to store it in a structured, pre-formatted way that relational databases do.
To confuse things a little, there are types of service, such as Azure CosmosDB which can appear to be both relational and non-relational, even though they are fundamentally non-relational. CosmosDB can be queried using SQL, which is a structured relational language, normally used for relational databases, but services like CosmosDB allow data to be retrieved using familiar languages like SQL with the data still not having to form a relational design and being stored in a non-relational way.
So what’s the point in both types? Can we not just have one type?
Both types of databases have positive and negative points. As a database grows, it grows in size due to the data it is holding. For relational databases, that means the number of relationships grow, and thus the operational processes about ensuring the integrity of those relationships can mean additional time for actions performing on that data (such as retrieval or a write).
Non-relational databases don’t have a lot of the heavy baggage that comes with relational databases, including such operational processes around relationships, therefore improving speed and performance in doing so. The downside of non-relational databases is you have to go ahead and get the data and store the data in specific formats and present it to the user in an easy to read format, for example, not JSON.
Business Applications are applications that deal with life. (Have a bank anyone? Did you go to university? Have you been to a hospital? Those are all Business Applications). Life, however, isn’t relational. So this means in Business Applications, things are not exactly always clear cut and as black and white as ‘related’ or ‘not related’. In addition, the scale of the life of customers and their ‘lifespan’ is extensive, centuries even, and so that data, even regarding a couple of hundred ‘customers’ can be extensive.
So, we have a challenge. What type of storage should organisations use?
We have some core related data, such as ‘Contact’ and ‘Account’, though not everything can be, or should be related to another. There will be inferences of data from other data, where the extent is unknown and cannot be pre-defined.
If Business Applications used only just relational data type stores, it means everything has to be related. In the same way, if they only used non-related data stores, it means it would be expensive and a lot of work creating custom operations and actions to display and write data back and forth and even do basic analytics.
Enter the Common Data Service.
So what actually is the Common Data Service?
The CDS is hosted on Azure and licenced through PowerApps (and Dynamics 365 CE). The Common Data Service contains six core features:
- ALM Capabilities
- Business Logic
This post will be specifically referring to the interface. The interface has specifically a relational type structure, coming pre-shipped with a set of schemas common to 80% of organisations, such as the ‘Account’ and ‘Contact’. It’s easy to create custom schema’s (i.e. Entities) via this interface, such as ‘Card’ in a loyalty management scenario.
There is a single CDS API. This API communicates to and manages every API request for both internal and external services for the Common Data Service. These ‘Other Services’ include managing data storage. It does this dynamically, depending on what type of data it reads it as. This allows organisations to manage the different storage types of relational and non-relational, without having to do really… anything.
A small fraction of this segmentation can be seen within the administrative dashboard found at admin.powerplatform.com when you see the split between ‘Database’ ‘Files’ and ‘Log’ storage.
Based on the data being stored, it would read it as a database, file or log storage type and go ahead and use what storage it would see as the most appropriate.
Looking at the screenshot below, we have a small subset of public data on Microsoft as an Organisation.
That data above is a combination of different types of data. Audit, Image and Relationship Data.
Based on the category of that data, it goes in different types of storage, behind the scenes. Those who utilise the Common Data Service do not have to pay for each of these storage options separately, and instead manage their data in the Common Data Service, including other actions such as the security model for that data. They don’t have to manage security, users or anything else inside CosmosDB, Azure SQL, or Azure Storage, only ever the Common Data Service.
The Common Data Service is not designed, and can’t be possible at the time of writing, to go ahead and access or choose where your data comes from and goes to. The idea is that you trust Microsoft to store the data in the best and secure possible way, so you don’t have to do it.
It does this using the endpoint for the Common Data Service.
So yes, technically using the Common Data Service should be faster than using just an Azure SQL storage if you’re using it for file storage. Also, it means you shouldn’t have to go ahead and do this yourself in an organisation, have all these different subscriptions and it means you should be able to have a single subscription and place.
So what is exactly is the Common Data Model within the CDS?
The Common Data Model is the relational schema defined by Microsoft which ships with the Common Data Service as standard. It has actually been open-sourced (accessed here) and whilst it has underpinned Dynamics or ‘Microsoft CRM’ for as long as it’s been around, it’s now become the adaptable base that is being extended upon by Partners, ISVs and customers to address the needs of vertical markets. This can be seen with the Non for Profit, Healthcare and Banking Accelerator solutions.
The concept of CDM allows interconnectivity between multiple systems, and by it being open source, allows companies to build on this model and decreases time to market and increases value for customers of 3rd party solutions.
Can I create my own Architecture rather than just use the CDS?
You might be asking the following or making the following statements at this point:
- The CDS ‘doesn’t work’ in enterprise scenarios
- The CDS isn’t scalable
The core takeaway is that we, and businesses, are used to having a single place for analytical data where we connect to applications such as PowerBI. The Business Applications world is still coming to terms with what the main purpose of what the CDS aims to achieve. This purpose is to be a single place for data used in interactive apps.
Historically it’s been Dynamics 365 Data, SharePoint Data, Finance Data, Azure Data etc.
The principle of CDS does not stop organisations exporting data from one environment to another for analytical purposes for the consumption of services like PowerBI or Machine Learning environments in a separate data warehouse. Go right ahead and create those integrations. CDS aims to be the ‘Home of Data’ for business applications, where apps – whether that is the app your doctor uses to review your medical records, the portal you log in to book your own next doctor appointment and see your patient history or the application the receptionist at the surgery uses to register you have arrived for an appointment and subsequently dispenses your prescriptions from. All these apps are hosted in a single environment, using a single store of shared data – giving businesses a bigger view of their customer’s worlds, quickly and by abstracting the heavy lifting, thus providing much quicker time to market and implementation time.
So I hope that has demystified some concepts for you, the purpose of the Common Data Service and really, why it’s amazing.