Using the Data Export API for advanced AI agents

What's my plan?

Add-on

AI agents - Advanced

Our Data Export API serves as the key to unlocking the full potential of your AI agent, empowering you to export conversation data and integrate it across diverse platforms and applications.

Whether you're a developer, business analyst, or IT professional, our API provides a streamlined and secure solution to extract valuable insights and conversation metadata, enabling you to drive innovation, make informed decisions, and propel your organization to new heights.

This article contains the following topics:

• Get the organization ID
• Generate an API token
• About the API
• File schema
• Response example
• Popular metrics
• FAQs

Get the organization ID

This step can only be done by users with an admin role

1. In the left sidebar, click Organization management.
   Note: If you purchased AI agents - Advanced before October 7, 2024, click User management > Organization management instead.
2. Click your organization to open its profile.
3. Find your organization ID in the URL in your browser:

   https://dashboard.ultimate.ai/admin/organizations/77m57af6811115b53172431s

Generate an API token

Generating a Token can only be done by users with an admin role

1. In the left sidebar, click Organization management.
   Note: If you purchased AI agents - Advanced before October 7, 2024, click User management > Organization management instead.
2. Click your organization to open its profile.
3. Select the API key tab.
4. Click Generate.
5. Click Save.
6. Copy the key and keep it safe.

Once you exit the Organization management page, you won't have access to the token. If you have lost the token you generated before, you can simply revoke it by clicking on the Regenerate button. The validity of the old token will be revoked and you can use the newly generated token.

About the API

The Data Export API is updated once per day, at midnight UTC (Coordinated Universal Time). This update process can run for a while, so running the import early morning UTC time will help ensure that the data is already there.

For the EU: POST - https://api.ultimate.ai/data-export/v2/get-signed-urls

For the US: POST - https://api.us.ultimate.ai/data-export/v2/get-signed-urls

Header

name	required	type
botId	true	string
organizationId	true	string
authorization	true	string

Body

name	required	type
date	true	ISO date string

Response

HTTP code	response
200	{ "date": "string", "urls": [ "string" ] }
401	Unauthorized
500	Internal Server Error

Note:

• The URL to the file expires after one day from the request date. To get a valid link, an API call can be performed once again.
• You can request data going as far back as January 1, 2024. If you need older data, contact Zendesk customer support for help.

File schema

The output of the API is a JSON document with all the conversations for a given day, following the structure outlined here:

• The file is a list of JSON objects where each object is a conversation
• The file will have the name convention conversation_bot-id_date_000000000000.json

Property	Description	Type
"bot_id"	Bot's unique ID	"bot_id": { "type": "string" },
"bot_name"	Bot's name	"bot_name": { "type": "string" },
"conversation_id"	Generated conversation ID	"conversation_id": { "type": "string" },
"platform_conversation_id"	CRM-specific ID	"platform_conversation_id": { "type": "string" },
"conversation_start_time"	Date and time of when the conversation started based on UTC timezone	"conversation_start_time": { "type": "string" },
"conversation_end_time"	Date and time of when the conversation ended based on UTC timezone	"conversation_end_time": { "type": "string" },
"language"	Language of the conversation	"language": { "type": "string" },
"channel"	Channel of the conversation. Either messaging or email	"channel": { "type": "string" },
"labels"	list of all the labels associated with the conversation	"labels": { "type": "array", "items": { "type": "string" } },
"conversations_data"	The session parameters are associated with the conversations. It is a list of the parameter’s key where it had a value. The undefined keys won’t be in this list Note: even though it is stored as a string, the “conversations_data” contains another JSON object the BSAT score is provided in the conversations data object BSAT Score Values -1 = User was offered BSAT but did not leave a response NULL = User was not offered BSAT Example: "conversations_data": { "parameter1": true, "parameter2": "parameter_value", "parameter3": "1234" },	"conversations_data": { "type": "string" },
"test_mode"	a flag to identify if the conversation is a test conversation	"test_mode": { "type": "boolean" },
"conversation_status"	Resolution of the conversation Example: bot_handled	"conversation_status": { "type": "string" },
"last_resolution"	This is the final resolution of the conversation. It could be informed, resolved, escalated, or undefined.	"last_resolution": { "type": "string" },
"triggered_replies"		"triggered_replies": { "type": "array", "items": { "type": "string" } },
"triggered_intent_replies"	the intent-based replies	"triggered_intent_replies": { "type": "array", "items": { "type": "string" } },
"is_llm_conversation"	a flag to identify if the conversation is a LLM conversation (at least have one LLM answer)	"is_llm_conversation": { "type": "boolean" },
"llm_notUnderstood_count"	the count of not-understood messages in the LLM conversation	"llm_notUnderstood_count": { "type": "string" },
"llm_responseGenerated_count"	the count of response-generated messages in the LLM conversation	"llm_responseGenerated_count": { "type": "string" },
"llm_errorOccurred_count"	the count of error occurred messages in the LLM conversation	"llm_errorOccurred_count": { "type": "string" },
"llm_escalationRequired_count"	the count of escalationRequired messages in the LLM conversation	"llm_escalationRequired_count": { "type": "string" },
"llm_fallback_count"	the count of fallback messages in the LLM conversation	"llm_fallback_count": { "type": "string" },
"bot_messages_count"	the count of AI agent messages	"bot_messages_count": { "type": "string" },
"customer_messages_count"	the count of customer messages	"customer_messages_count": { "type": "string" },
"not_understood_messages_count"	the count of the not understood messages in general	"not_understood_messages_count": { "type": "string" },

Response example

The response of the API call is an url link to a file in a google storage bucket that you can access with a GET request:

{"date":"2024-05-03","urls":["https://storage.googleapis.com/production-eu-ultimateai-backend-data-export/files/your_bot_id/2024-05-0/conversation_your_bot_id_20240503_000000000000.json…"]}

The file contains each conversation as a json object following this format:

{
  "bot_id": "98174e8d635471c383b9ec7b",
  "bot_name": "INDUSTRY DEMO (Travel) - SunCo",
  "conversation_id": "67bc4129-6609-4v67-869d-db1d0186d1d8",
  "platform_conversation_id": "57459bd72555b8452378f693",
  "conversation_start_time": "2024-05-03T08:10:01.211+00:00",
  "conversation_end_time": "2024-05-03T08:10:25.744+00:00",
  "language": "eng",
  "channel": "chat",
  "labels": [
    "web",
    "API:getBookingDetails-Success"
  ],
  "conversations_data": {
    "bsatScore" : 5,
    "convoId": "552cd72556e8452378d344",
    "email": null,
    "location": null,
    "confidence_score": 95,
    "lastDetectedLanguage": "eng",
    "lastDetectedSentiment": "neutral",
    "usedLanguage": "eng",
    "channel": "web",
    "bookingNo": null,
    "booking": 
    [{
      "country": "Spain",
      "url": "https://cdn.pixabay.com/photo/2015/05/05/01/10/house-753270__340.jpg",
      "location": "41.3485806,1.9787689",
      "city": "Barcelona",
      "numOfGuests": 4,
      "days": 4,
      "arrivalDate": "12-12-2022"
    }],
    "city": "Barcelona",
    "manageBooking": "possible"
  },
  "test_mode": false,
  "conversation_status": "botHandled",
  "last_resolution": "resolved"
  "triggered_replies": [
  {
    "reply_timestamp": "2024-05-03T08:10:02.991+00:00",
    "reply_id": "53628e8e55b4f2459bcb2e72",
    "reply_language": "eng",
    "reply_name": "Greeting",
    "reply_type": "normal",
    "intent_id": "64628b8e55e4f2459bcb2e68"
  },
  {
    "reply_timestamp": "2024-05-03T08:10:03.355+00:00",
    "reply_id": "64628e8e55e4f2459bcb2f4b",
    "reply_language": "eng",
    "reply_name": "Welcome reply",
    "reply_type": "welcome"
  },
  {
    "reply_timestamp": "2024-05-03T08:10:12.951+00:00",
    "reply_id": "2475e571f88f9c35af3ff45e",
    "reply_language": "eng",
    "reply_name": "Office or store location and opening hours",
    "reply_type": "normal",
    "intent_id": "6385e571f88f9c35af3ff46e"
  }
  ],
  "triggered_intent_replies": [
  {
    "intent_timestamp": "2024-05-03T08:10:02.991+00:00",
    "intent_id": "53628e8e55b4f2459bcb2e72",
    "intent_name": "Greeting",
    "not_meaningful": true
  },
  {
    "intent_timestamp": "2024-05-03T08:10:12.951+00:00",
    "intent_id": "6375b571f88f9c35af3ff44e",
    "intent_name": "Find location of rental",
    "not_meaningful": false
  }
  ],
  "is_llm_conversation": false,
  "bot_messages_count": "7",
  "customer_messages_count": "4",
  "not_understood_messages_count": "0"
}

Popular metrics

Congratulations! You have successfully exported the AI agent's conversation data. Here are some of our suggestions to start the data exploration journey of the exported files.

AI agent metrics

SELECT
--Total conversations
count(distinct conversation_id) total_conversations,

--Bot handled rate
count(distinct case when conversation_status = 'botHandled' then conversation_id end) bot_handled_conversations,
count(distinct case when conversation_status = 'botHandled' then conversation_id end)/count(distinct conversation_id) bot_handled_rate,

--Deflection rate
count(distinct case when conversation_status not in ('email', 'agent', 'customEscalation') then conversation_id end)/count(distinct conversation_id) deflection_rate,

--Escalation rate
count(distinct case when conversation_status in ('email', 'agent', 'customEscalation') then conversation_id end)/count(distinct conversation_id) escalation_rate,

--Failed escalation rate
count(distinct case when conversation_status = 'failedEscalation' then conversation_id end)/count(distinct conversation_id) failed_escalation_rate,

--Informed rate
count(distinct case when last_resolution = 'informed' then conversation_id end)/count(distinct conversation_id) informed_rate,

--Custom resolution rate
count(distinct case when last_resolution in ('informed', 'resolved') then conversation_id end)/count(distinct conversation_id) automation_rate,

--Message understood rate
(sum(customer_messages_count)-sum(not_understood_messages_count
))/sum(customer_messages_count) messages_understood_rate

FROM TABLE

First / Last Intent from a conversation

select 
distinct conversation_id,
triggered_intent_replies[safe_offset(0)].intent_name first_intent,
array_reverse(triggered_intent_replies)[safe_offset(0)].intent_name last_intent
FROM TABLE

Metrics by first meaningful (or filtered) intent

with meaningful_intents as (
select conversation_id, first_meaningful_intent, conversation_status from 
(
select distinct conversation_id,
intent.intent_name first_meaningful_intent,
conversation_status,
--order by ascending intent_timestamp for last intent
ROW_NUMBER() OVER (PARTITION BY conversation_id order by intent.intent_timestamp asc) rn,
FROM TABLE
LEFT JOIN UNNEST(triggered_intent_replies) intent
--select only intents that are labeled as meaningful
where intent.not_meaningful is false 
--filter for specific intents
and intent.intent_name not in ('Greeting', 'Talk to a human/agent')
)
where rn=1
)

--calculate metrics for each intent
select first_meaningful_intent,
count(distinct conversation_id) total_conversations,
count(distinct case when conversation_status = 'botHandled' then conversation_id end)/count(distinct conversation_id) bot_handled_rate
from meaningful_intents
group by first_meaningful_intent
order by total_conversations desc

FAQs

• Are the exported files immutable or do they change over time, requiring you to re-sync them?
  The exported files are immutable, so they do not have to be reimported, thus making your BI pipeline less error-prone.
• I see differences in conversation counts for a specific date between the exported conversations and AI agent Summary analytics. What could be the reason?
  The exported data for a specific date has data only for conversations that ended on that date. This is a different approach than in the AI agent Summary analytics, where any conversation at any state is included. This is done for multiple reasons, among them are: export file immutability and duplicate data prevention.
• Then how can I achieve parity between the numbers reported in the AI agent summary and the exported files?
  To get a complete picture of all conversations that took place on a certain day x, you can just ingest the file generated for day x and day x + 1. By ingesting files from these two dates, you ensure that you will have counted all the conversations for date x, even if it spilled over to the next day.
• I see differences in conversation counts for a specific date between the exported conversations and Conversation Logs. What could be the reason?
  Remember that the exported data for a specific date has data only for conversations that ended on that date. This is a different approach than in the Conversation Logs, where any conversation at any state is included to enable real-time debugging.
• Does a conversation include information about all replies triggered in a conversation, even if those replies happened on a previous day?
  Yes. If a conversation is included in the file, it will have all of its data, even if it occurred on a previous day.
• My virtual agent has conversations as part of the suggestion engine, will those appear in the export?
  Yes. Although in the AI agent summary dashboard these are excluded, these will still be available in the exported data. To exclude these and achieve parity with AI agent summary dashboard, filter by "bot_messages_count > 0".