DataStream

Introduction

Zentitle offers comprehensive and platform-wide data-streaming functionality.  Should you wish to try out DataStream, please contact your Account Manager to request access to it for your account (it is not available by default). 

DataStream provides near real-time, JSON-based, full-system access to your generated data relating to your licensing, analytics, and usage of the Zentitle platform. Key data and metadata can be streamed, for different event types, including client-side licensing and dashboard/API server events. This data can be streamed to your own custom end-point or to one of a select number of pre-defined end-points. In the way of pre-defined end-points, we currently offer Segment and this range will soon expand to include others such as SalesForce.   It is possible to stream data for client connections to either the Zentitle licensing server or the LAN Daemon.

This functionality brings a new level of scope and insight to your Zentitle experience. Key ways you can leverage this data to your advantage include:

• Fine-grained end-user reporting: use your chosen BI tools and reporting platforms to obtain a complete, historical picture of how your software is being used. All the events where the client application interacts with our licensing servers are logged and streamed out, so that they can be reported on, or further external processes can be driven by them, in near real-time.
• Total operations visibility: you are provided with data you need to be able to create comprehensive reports of the license management tasks carried out members of your team within the Zentitle UI, and similarly those tasks performed from your back-end infrastructure via API/webservice calls to Zentitle.
• Accelerated problem-solving: by having the data available to you to track licensing interaction events that have happened server-side, on the Zentitle's servers, together with the client-side view you already have via log files and testing, the full view these give in combination with each other aid you in diagnosis and resolution of end-user and license management issues.
• Further monetization: use the data provided to automatically or manually study the usage of particular clients, such as the high-watermark usage of multi-seat style license models of your business customers, to establish further sales opportunities and closer customer relationships.

Setting Up DataStream

If you wish to have DataStream functionality enabled on your account, please send a request to your account manager.

Once DataStream is enabled for your account, via the Zentitle UI (see below) you will be able to choose which DataStream event types you wish to have enabled.

The following table shows the value of the EventType parameter which occurs in each DataStream record together with a description of the type of event recorded.  This parameter is useful for filtering DataStream records by source.

When you have DataStream functionality enabled, you will be able to create and configure data your streams on the Zentitle dashboard. You can access this configuration page by going to the Webservices page:

and then clicking the Data Streaming tab to the right.   This will show the page below.

On the data streaming page, you can create new data streams, which will appear in the Data Streams table. Once created, any of these data streams can be edited or deleted by clicking the corresponding notepad or trashcan icon respectively.

Creating a New DataStream

You can create a new data stream by clicking the Generate New Data Stream button.   This will launch the following dialog:

You should choose a unique name for your data stream and select an event type from the corresponding dropdown.

In addition to the basic event types discussed above, you can also choose your data stream to apply to 'All Types' for convenience.   

You should also choose a destination. Currently, we support two destination types: 'Segment' and the general 'User-Defined'.  The latter would be your own custom end-point which you will have to setup.

If you select 'Segment' then you will also have to supply an API key.   If, on the other hand, you choose 'User-Defined', you will have to supply a URL for your custom end-point.

Please ensure data from Zentitle can reach your end-point by whitelisting the IP address of the Zentitle data streaming server, which is 2001:4801:1041:202:545:3474:e079:43bb (V6) or 166.78.76.228 (V4). 

Data to a custom URL will be sent in batches of items, with each event corresponding to one item.  Each batch will be of single event type. The default batch size is 1 although it can take any value up to 50.

Finally, the 'Enabled for use' checkbox can be used to enable/disable a particular data stream definition.

NOTE: Enabling an 'All Types' data stream will disable all other data streams with that destination.  Conversely, enabling a specific event type data stream will disable any 'All Types' data streams with that destination.

Data Format

All data sent by the data streamer is in JSON format.   Its structure will depend on the destination type. We describe below the structure for each of the available options.

Streaming to a Custom End-Point

Data streamed to a custom end-point will take the form of a JSON object having the following structure:

{
"batchnum": <batchnum>,
"batchdata": <batchdata>
}

Here <batchnum> is the number of the batch of a this particular type.  Also, <batchdata> is a JSON array with one element for each record in the batch. Here is a full example:

{
"batchnum": "21",
"batchdata": [
{
"ClientCreated": "2021-03-17T14:34:40.863Z",
"CompanyID": 5367,
"Created": "2021-03-17T14:34:41.101Z",
"Event": "InstallProducts",
"EventType": "UI",
"Payload": {
"numproducts": 1
},
"Ip": "::1",
"Processed": false,
"ProductID": 0,
"UserID": "nsa@nalpeiron.com"
},
{
"ClientCreated": "2021-03-17T14:35:00.654Z",
"CompanyID": 5367,
"Created": "2021-03-17T14:35:00.779Z",
"Event": "InstallProducts",
"EventType": "UI",
"Payload": {
"numproducts": 1
},
"Ip": "::1",
"Processed": false,
"ProductID": 0,
"UserID": "nsa@nalpeiron.com"
}
]
}

The fields shown in each record are defined as follows:

• ClientCreated: Java Date record.  For NSA type records, this is when the data was captures on the client.   For other record types, this is when processing of the event commenced on our server.
• Created:  Java Date record of when the event was written to our internal database.
• Event: The name of the event being reported on.
• Payload: A JSON object containing data specific to the event.
• Ip: The IP address of the machine on which the event occurred.
• Processed:  A Boolean flag indicating whether or not the data has yet been streamed.
• UserID:  A user identifier.   This will be a username wherever possible (e.g. the person logged in for UI events,, the webservices username for a SOAP webservice call.) 

Streaming to Segment

To send data to Segment, you will have configured a corresponding DataStream as above including setting an API key.   You will have obtained this from Segment when you configured, on your Segment account, Zentitle as a source.   The image below shows the Segment dashboard where we have setup this source as 'Zentitle (.NET)':

Click on this source and you will see a number of tabs.  The key one for displaying your streamed data is the 'Debugger' tab.  On selecting this, you will see an 'IDENTIFY' and a 'TRACK' record for each recently streamed event.   Here, we have one such event, a UI event of type 'InstallProducts'.

The 'IDENTIFY' record associates each user with their actions.   You can find out more about this in the Segment documentation.   The 'Raw' view, shown below, displays this record in JSON format:

The 'TRACK' record contains the actual streamed event data.   Clicking this for the event we have recorded, and selecting the 'Raw' view, will display the data streamed to Segment for this event:

In relation to data streaming, the key JSON elements here are the "event" element, whose value is the event type, and the "Payload" element, which is identical to the payload element described above for a custom end-point and will contain data specific to the event type.

DataStream Export via REST API

You can export data which has been captured via data streaming using our REST API (V2 or greater.)  The relevant API call is 'GET DataStreamer'.  Please see this document for further details.