Virtual Entities


Interact with data from external systems using the new virtual entities

Starting with the July 2017 Update for Dynamics 365 (online), virtual entities enable the integration of data residing in external systems by seamlessly representing that data as entities in Dynamics 365, without replication of data and often without custom coding.

Virtual entities replace previous client-side and server-side approaches to integrating external data, which required customized code and suffered from numerous limitations, including imperfect integration, data duplication, or extensive commitment of development resources. In addition, for administrators and system customizers, the use of virtual entities greatly simplifies administration and configuration

A virtual entity is a definition of an entity in the Dynamics 365 platform metadata without the associated physical tables for entity instances created in the Dynamics 365 database. Instead during runtime, when an entity instance is required, its state is dynamically retrieved from the associated external system. Each virtual entity type is associated with a virtual entity data provider and (optionally) some configuration information from an associated virtual entity data source.

Records based on virtual entities are available from all Dynamics 365 (online) clients, including custom applications developed using the Dynamics 365 SDK.

Virtual entities provide these benefits:

  • End users work with the records created by the virtual entity to view the data in fields, grids, search results, and Fetch XML-based reports and dashboards.
  • System customizers can configure the data source record and create virtual entities without writing any code.
  • Developers can implement plugins to read external data using the Dynamics 365 SDK and Dynamics 365 (online) Plug-in Registration tool.

Considerations

In this release, virtual entities have a couple of restrictions:

  • Data is read-only
  • Only organization-owned entities are supported
  • Field-level security is not supported
  • It must be possible to model the external data as a Dynamics 365 entity. This means:
    • All entities in the external data source must have an associated GUID primary key
    • All entity properties must be represented as Dynamics 365 attributes - you can use simple types representing text, numbers, optionsets, dates, images, and lookups
    • You must be able to model any entity relationships in Dynamics 365

Example

In this blog post I'll walk you through a simple way of creating a virtual entity.

What you need to test this out:

  1. Access to a service exposing data in OData v4
  2. An entity in this external data source with an associated GUID primary key
  3. Access to the latest preview of Dynamics 365

On the odata.org site you will find a service you can use for testing. Just type services.odata.org/V4/OData/OData.svc/$metadata in your browser to see a list of entities

Fig. 1

If you expand a given entity branch, eg Product, you will need to check if the entity has an associated key of type GUID (requirement #2 above). Please note that this is NOT the fact for the Product entity (ref picture below). So this entity is not one we can bring in.

Fig. 2

However expanding the entity Advertisement branch, we see that this entity HAS got an associated key of type GUID, so we will use that. Note the ID is spelled "ID" (all capital letters), and also note the Name and AirDate properties - we will get back to those

Fig. 3

To see which records this data set returns we will need the collection (ie plural) name of the entity - for that we type services.odata.org/V4/OData/OData.svc in the browser, and see the collection name is Advertisements

Fig. 4

Using this information its a matter of typing services.odata.org/V4/OData/OData.svc/Advertisements in the brower to learn that the data set holds two records (these are the two external records we will surface in Dynamics 365 using the new Virtual Entity capability).

Note: You can use this OData API Explorer for a more visual representation of the data set (in its v3 edition though)

Fig. 5

 

Create a Virtual Entity Data Source

With the data source identified we can proceed to the next step - creating a Virtual Entity Data Source. In Dynamics 365 click Settings -> Administration (1) and then Virtual Entity Data Sources (2) to open the Data Sources grid.

Fig. 6

In the Data Sources grid click NEW (1) to open the Select Data Provider dialog

Fig. 7

In the Select Data Provider dialog select OData v4 Data Provider (the only option) and then click OK to open the New OData v4 Data Source dialog

Fig. 8

Fig. 9

In the New OData v4 Data Source dialog fill out the three to fields in the General section

  1. In the Name text box type a name of your choice for the data source (I'll use "Public Service")
  2. In the URL text box type or paste the URL from above (fig 4) services.odata.org/V4/OData/OData.svc
  3. In the Timeout text box (optional) type the number of seconds to wait for a response from the web service before quitting a data operation

And then click OK (4) to return to the Data Sources grid

Fig. 10

Fig. 11

 

Create a Virtual Entity

Last thing is to create a Virtual Entity to bring in data from the OData source. Click Settings -> Administration -> Customize the System 

Fig. 12

Create a virtual entity like any custom entity, and then select the Virtual Entity check box (1) - see below

Selecting the check box displays additional information requirements for the data source (2), as well as the External Name and External Collection Name values (3) for the entity definition.

So in this example the

  1. name of the new Virtual Entity is called "Advertisement" (a name I typed)
  2. name of the Data Source is "Public Service" (as per fig. 10 above)
  3. External Name = "Advertisement" and External Collection Name = "Advertisements" (as per fig. 3 and 4 above)

Fig. 13

Once the new Virtual Entity is created a couple of important things to ensure/edit in order for the mapping to work.

The system automatically creates two fields, one for the id (1) and one for the name (2). You will need to map those to the external data source names.

Fig. 14

So for the ID field (the primary key) ensure that the name in the External Name text box (1) matches the property name in the data source (2) as per figure 3 above. Observe case sensitivity.

Fig. 15

For the Name field ensure that the name in the External Name text box (1) and the Data Type (2) matches the property name and type in the data source (3)  as per figure 3 above. Observe case sensitivity and Data Type.

Fig. 16

Optionally you can create a third field to bring in the AirDate field from the data source (Date type)  as per figure 3 above. Observe case sensitivity and Data Type. If the property has Nullable = "false" like seen for the AirDate property (3), then you must set Field Requirement = Business Required (4)

Fig. 17

Create a form with the desired fields

Fig. 18

Create a view with the desired columns

Fig. 19

Publish customizations

Fig. 20

 

You now have a list with the two records from fig. 5 above

Fig. 21

Opening a record will display the form you defined

Fig. 22

I hope you will enjoy this new elegant way of surfacing data from external data sources in Dynamics 365 without the data residing in Dynamics 365. Data are mastered outside of Dynamics 365, yet available to work with in Dynamics 365 like any other entity.

Enjoy.

See Also

  • Virtual Entities Demo Solutions (download) - link
  • Virtual Entities - Relationships  - link
Comments (33)

  1. Priyesh Wagh says:

    Superb article! Thanks a lot.

    1. Priyesh Wagh says:

      For some reason, this is failing for me in the Entity. Generic Error. The one that is embedded in the main window and not the one we get to Download the Log File from.
      I did all steps correctly. (Just that I have a publisher and hence a different prefix to my entity. I don’t think this should be a problem)

      Any ideas?

      1. Yes Even I am facing the same issue as Priyesh, Any idea, why this would happen ? I followed the exact same steps as the blog, but it is failing.

      2. Lewis Harris-Manley says:

        We are experiencing same as Priyesh and Somesh – followed steps to the letter but still when try to access Entity we get a generic error.
        We noticed that on the screenshot in the above article it wasn’t clear regarding which option to select for “Pagination Mode” and it also wasn’t a mandatory field. Also “Return Inline Count” wasn’t shown in the screenshot.
        Not sure if either of the above have anything to do with the error we are seeing?
        Any help would be greatly appreciated!

    2. Jan-Willem Cobelens says:

      Thanks for this extensive walkthrough. Unfortunately we experience an error as well when saving the entity: “The evaluation of the current component(name=SystemForm, id=8514bedc-7313-40ab-9d13-0f31c7c59918) in the current operation (Create) failed during managed property evaluation of condition: Managed Property Name: isparentcustomizable; Component Name: SystemForm; “. Any ideas?

      1. Simon Hetzel says:

        I had this also in an instance that had been updated to build .1054 – the new build 1054 instances provisioned over this week end however do not appear to have this problem. See the following for more details:

        https://connect.microsoft.com/DynamicsCustomerEngagement/feedback/details/3140592/creating-custom-entity-gives-error-the-evaluation-of-the-current-component-name-systemform-id-in-the-current-operation-create-failed-during-managed-property-evaluation-of-condition

        Having said that I can’t get this to work using the instructions above using this new build.

  2. Simon Hetzel says:

    Hi Jesper – what build were you using to do this? Our instances were updated at the end of last to build .1054 then further new instances with build .1054 provisioned over the weekend. We can’t get these to work with Virtual Entities by following your instructions, nor the ones in the Quick Start Guide on Connect, nor with the OData endpoint listed here:

    https://connect.microsoft.com/DynamicsCustomerEngagement/feedback/details/3137528/bug-question-as-a-system-customizer-developer-i-am-unable-to-implement-virtual-entities

    The lack of any meaningful error messages with Virtual Entities at the moment makes it all really difficult to figure out what is going on.

  3. Abdul Majid says:

    Great Article! However, we too get “An Error has occurred”.

  4. Thanks for all the comments. Between writing this post and the time of the comments, a new preview build was deployed. An issue with the OData provider will be fixed with the next build as I understand it. Please be patient.

  5. Alex Shlega says:

    Hi,

    thank you for the article. Wondering if it’s going to be possible to define relationships between the virtual entities and “native” entities? For example.. if I wanted to surface some additional contact information in the contact records.. Would I be able to somehow display “quick view” form of the virtual entity on the contact forms?

  6. Khadim Ali says:

    The TechNet article says:
    “All entities in the external data source must have an associated GUID primary key”. Does it mean if the external data source wont have a Guid column, the D365 platform will not read external data into Virtual Entities?

    1. Simon Hetzel says:

      I believe so Khadmin yes but because of the problems with the latest build I haven’t been able to verify. This seems to mean that, (for the moment at least), the use of Virtual Entities can only be extremely limited as many OData sources that you might want to connect to are unlikely to use GUIDs. as the primary key.

      1. Ken says:

        Since the GUID’s won’t really be important long term (since nothing gets stored in the DB), they could just be dynamically generated with a wrapper. The GUID’s are just important for CRM to ‘treat’ it like a regular record. We have done ‘virtual’ records for a while via custom code on retrievemultiple plugins that talks to external data – this just makes it more accessible without needing to write plugin code. Adding a GUID attribute, or wrapping an OData service that you don’t control, to generate a GUID attribute should be by far less painful than the plugin code.

  7. The issue has now been fixed. One thing to notice; if a property is listed with Nullable = False then the corresponding Field Requirement must be marked as Business Required.

    1. Elango says:

      Still not working. I have followed all given steps. Pagination mode set as Client side..Any idea..?

  8. Sathish Ramamoorthy says:

    Thank you for the great article.
    Which Authentications methods are supported for Virtual Entities? In the ver. 1612 (Build 1103), I don’t see an option to specify/set up/configure authentication for the OData service.

    1. Ken says:

      I presume based on the interface for configuring data sources, your only authentication options will be to provide request parameters like client keys in the data source configuration. As long as only your system administrator can access the data source records this /may/ be enough depending on your security requirements. If your data does not have a GUID record id, you could add one, or create a wrapper for that data that does.

  9. John says:

    Thanks for the detailed write-up! This certainly helps with understanding how this all works. Is there a published roadmap on future plans for this capability? This thing has a lot of promise once the CRM data structure and oData constraints go away. Hopefully, that’s where this is headed?

  10. Ken says:

    Is this currently working for anyone? I just get OData errors even for this simple example…on version 1612 ( 9.0.0.2072 )

    1. Works for me without any problem. Just be careful about the Requirement level on the DateTime field – otherwise you’ll get an following error – Type mismatch of the external attribute of type ‘System.DateTimeOffset’ expected a ‘System.Nullable`1[[System.DateTimeOffset, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]]’ value. Attribute Name: new_airdate

      1. purnima says:

        I followed the blog step by step and I am getting an error “Not implemented”

        1. Matthias Back says:

          I ve got also the feedback “Not Implemented” CRM Online Version 1612 (9.0.0.2180)

  11. Jason Huxwell says:

    Hi Jesper
    I get the following error:

    Record is Unavailable
    The requested record was not found or you do not have sufficient permissions to view it.
    Using Dynamics 365 v9.0.0.2072

  12. Raheem Shaik says:

    I got this output. I make it work this example

  13. Jagadesh D says:

    No luck. Not able to get the records though followed all steps. I am using Dynamics 365 online trial to check this. Is it causing the error?

    1. Vikrant says:

      Hi All,

      I can able to configure the Virtual entity using above steps.
      But now i want to know how do we configure the virtual entity using the Credentials? See the above WebAPI url do not need the authentication so, now if i want to connect to my another CRM instance so how can i connect it using virtual entity.

      Thanks!

      Vikrant

  14. Shahnaz says:

    Hi there,

    I’m trying to see how we can register a custom data provider also, besides just a data source. The article at https://docs.microsoft.com/en-us/dynamics365/customer-engagement/developer/virtual-entities/custom-ve-data-providers#steps-to-use-a-custom-data-provider only says “Create a data provider solution” in CRM. There are no steps on how to go about this process. Any pointers?

    Thanks,
    Shahnaz

  15. indikaparane says:

    Hi Jesper,

    Could you please guide us how we can map Lookup type field from OData endpoint to Virtual entity field ?

    Thanks
    Indika

  16. Aishwarya says:

    Hello,

    Is there a way to dynamically pass Request parameters? I basically have to pass few attribute values to the WebService as parameter. I tried accessing entitydatasources through WebApi in Javascript, so that I can set value for connectiondefinitionsecrets. But unfortunately I am getting Internal Server error as response. Kindly share your thoughts.
    Thank you.

  17. Veena says:

    Such a clean and sorted article. Thank you!
    Cheers.

Skip to main content