This white paper explores the purpose and benefits of XLAs and XPIs, how they differ from SLAs and KPIs, and why they are critical to ensuring that the digital workplace continuously drives improved productivity, user experiences and business outcomes.
Workers’ relationship with technology has rapidly changed, enabling them to be more independent and use new workplace support models. As a result:
Users widely expect a consumer-like experience that mirrors the ways they can interact with technology in their personal lives.
Businesses are better able to capitalize on new technology advancements that provide an improved user experience in the workplace.
Businesses are now recognizing the value of savings associated with an improved user experience, including employee productivity and engagement, which can now be measured more precisely through new tooling.
Business priorities are now aligning with creating an intelligent, continuously improved digital workplace that empowers employees with the experience they need to be productive and improve business outcomes. Understanding what the ideal experience is requires deep insights into how employees are interacting with IT and how successful those interactions are in terms of enabling productivity.
Traditional metrics can’t provide this level of insight. Businesses need the ability to fully track successful digital transformation and measure user sentiment, engagement and productivity—all critical to improving the user experience and business outcomes.
This need for a new level of insight has led to a new approach to measuring service value. Experience performance indicators (XPIs) are used to measure individual user experiences and business outcomes. The experience level agreement (XLA) is used to measure the end-to-end user experience and overall business outcomes.
Improving the user experience is not a new topic in the digital workplace; however, many interpret the term differently. The overall user experience is the combination of all interactions with a given solution, but assumptions are often made regarding what will provide a better user experience based on a narrower scope and little validation. For instance, experience is often used synonymously with satisfaction, which is often not actively measured.
Some types of experiences in the digital workplace include:
User satisfaction: user contentment
User effort: amount of input required from a user to complete a task
User outcomes: results for users, including increased productivity
User impact avoidance: removal of a negative event
A user experience may include one or more of these types, but it’s important to recognize each as being different. For instance, it’s been noted that user loyalty correlates more closely to user effort than to user satisfaction. A user may express satisfaction with the experience, based on the outcome, but feel it took more effort than expected, negatively affecting future solution engagement.
It’s also important to identify the various touchpoints that affect a user’s overall experience. For instance, when reviewing user support and considering service desk agent interactions, it’s important to consider other elements, such as the impacts of issues reported and unreported, the ease of use of support, and experiences with nonhuman channels. See Figure 1.
Overall, when determining the experience of the digital workplace, it’s important to review all user types and touchpoints of experience that should be considered.
Attributing new workplace experience metrics to today’s complex user IT journey
The user’s IT journey has become much more complex with the rapid growth and adoption of devices, applications, channels and remote working. Users access the tools and information required to perform their jobs without being aware of the teams involved in bringing this workplace experience together. For example, users aren’t aware of what teams are involved in their IT support experience; however, what they do know is that they want 24x7 access to the applications they need on their desktop, mobile, wearable device and the web, and they expect a consistent experience every time.
The need for the XLA arose because many different IT teams are responsible for the user’s overall IT support experience and their outcomes, which span workflows that can include multiple services and teams. The XLA provides a measure for the user experience and outcomes rather than focusing on the execution of individual services.
Definitions of XPI and XLA:
An XPI targets a part of the user’s end-to-end experience or business outcomes.
An XLA comprises multiple indicators (XPIs) that target an improved end-to-end user experience and business outcomes across a specific area of the business, such as getting effective IT support or HR processes.
Defining new workplace metrics to align with IT services
To ensure worker productivity in the digital age, companies need the XLA and XPIs to measure user experiences and outcomes. These may be the sum of multiple teams and decision makers versus a single service. Therefore, attributing the XLA or XPI to one team can be inequitable, and assigning responsibility to one team can be subjective rather than objective. This is one of the reasons why many proposed XLAs and XPIs are not part of traditional service level agreements (SLAs) and key performance indicators (KPIs)—not because SLAs purposefully don’t focus on user outcomes but because the XLA or XPI is not entirely within one team's or service’s control. As a result, it may not be possible for a team to commit to achieving the target in isolation. For example, on a service desk, you can make changes to improve average speed to answer, call abandonment rate or first call resolution, but a user effort score could be impacted by company processes, security policies, the intranet interface, the ITSM setup or internal communications, as shown in Figure 2.
Figure 2: Impacts to user effort score
Teams that contribute to the user’s effort in getting support:
HR team and onboarding: letting the user know how to use IT support
User’s team: company culture and guidance on getting support
ITSM team: portal design
Applications team: chat availability
IT operations and security teams: processes to follow
Intranet team: to find support portal
Network team: chat availability
Service desk team: agent support
Level 3 support team: guidance documents
This complex spiderweb of responsibility across teams leads many to refer to XLAs and XPIs as KPIs only. This often conflicts with enterprises’ objectives to seek commitments regarding the value of the service as defined with these new metrics. To resolve this, algorithms and data elements should be aligned to their services. It is also worth creating XPIs that do not completely align to one service and therefore cannot be committed to by one third party, but enable more effective measurement of the overall user outcomes.
Businesses should clearly define how they will use XLA and XPIs to make continuous improvements across the end-to-end experience, which may involve several teams.
Digital workplace user experience focus areas
When identifying the optimal experience that will measure user outcomes across the digital workplace, our analysis defined 5 key areas under which metrics can be categorized. To measure the end-to-end user experience (the XLA), it’s important to evaluate all of these key areas:
Device. User productivity impact at a device level
Automation. The quality and quantity of automation that can reduce or remove user impacts and speed up processes
Self-service. Effectiveness of self-service capabilities in providing users with support
User effort. The ease of user interactions and resolutions, which is typically an intangible aspect of the digital workplace
Agent channel. Outcome-based metrics that holistically measure traditional agent channels from the user’s perspective
Targeting metrics from these focus areas (XPIs) individually and reviewing them together alongside some traditional metrics help ensure that user outcomes and user experiences are more holistically understood, measured, targeted and improved.
Technologies required for a successful XLA and experience analysis
To successfully deliver against an XLA, it’s important to cross-analyze and investigate experience-focused data to identify root causes of issues and determine effective improvements. Therefore, alongside measuring an XLA and XPI, it’s key to use an insights engine capable of enabling this experience analysis and investigation.
Because of the variety of experience types and touchpoints, this experience-focused data is often captured in multiple tools and formats and therefore requires collocating and compiling for experience analysis. Data inputs required for measuring an XLA come from structured, standardized format; unstructured, without a predefined data model; and semi-structured data sources, and therefore need a robust systems architecture built around big data concepts. Figure 3 provides a high-level architecture view of typical systems that must be in place before XLAs and experience insights can be effectively implemented.
Figure 3: XLAs in the digital workplace
A comprehensive XLA reporting solution and insights engine will typically consist of components such as:
These components will collect, collate and store a comprehensive omnichannel view of enterprise users’ interactions for further analysis across voice, chat, email, mobile apps and web.
The components will collate direct feedback from users regarding their workplace experience through surveys and other feedback mechanisms.
These components will typically report experience from enterprise users’ various physical devices, including laptops, desktops and mobile devices. Various tools are available that establish endpoint sensors and data collectors optimized for specific device types. These tools collect data at varying levels of detail, aggregate the data as needed and push the data to downstream consuming services.
Various data integration tools are required to collect and compile the different types of data needed. These can consist of using APIs and extract, transform and load (ETL) methods to extract data from the appropriate omnichannel solution on a periodic basis. Data stewardship is also an important aspect for protecting personal data and applying the appropriate level of data governance.
Within the cloud-based repository, data from each channel is ingested and stored in a persistent manner. In some cases, the source data may be semi-structured or unstructured and may benefit from additional data enrichment using AI tools. For example, text analytics techniques such as clustering can be useful for analyzing unstructured data and extracting additional meaning from the data. For advanced use cases, telemetry data from the endpoint sensors can be used to perform anomaly detection and to create predictive insights. Select high-value data can be extracted from the cloud data repository and enriched with AI tools. It may also be necessary to combine or merge various sets of data to build the required record set.
The data should be extracted and stored in a cloud-based data repository for additional processing and analysis. Cloud-based repositories provide a great deal of flexibility and scalability, depending on an organization’s needs. Data from each of the relevant user data sources should be extracted and staged.
Data from existing ITSM tools, such as ticket data, can be analyzed to provide insights into the user experience.
These enriched data sets can then be used to drive the end-to-end experience visualizations for an enterprise’s services. This data can be consumed from the analytic dashboards and can drive other actions. The dashboards should support various user personas to provide users with the relevant view of data needed for their role. Detailed data from user data sources and channel interactions can also provide insight into how well existing services are performing and highlight needed business process changes in some cases.
Important factors to consider when deploying an XLA and XPIs
Experience shows that a range of XPIs can measure key experiences and outcomes in the digital workplace. These can be combined into an XLA score to provide a holistic view of the end-to-end user experience and business outcomes.
The following list provides some of the key enablers for enterprises to ensure the XLA and XPIs are successfully deployed and provide the anticipated value:
Focus on quality XPIs over quantity. Quality XPIs measure the end-to-end user experience and can be realistically reviewed in collaborative workshops. Improvement plans can be agreed upon, regardless of where the impact is and which team in the service will need to resolve issues.
Ensure that governance structures factor in user experience improvements, with service providers and the client enterprises aligning goals and resources to the agreed-upon transformational change.
Replace traditional SLAs with new XPIs that can more accurately measure user outcomes. For example, if the user effort score is good, measuring the average speed to answer and call abandonment rate provides little value and can increase costs.
Include proportional incentives or penalties for XPIs. Although some XPIs may expand outside of the service provider’s scope, some XPIs will reflect the value of the service, which should be represented with a financial commitment with SLA penalty regime caveats.
Avoid combining incomparable data types into a single score to define experience (a performance index) because this can lead to the XLAs losing meaning or being distorted by one data set. For example, combining user effort score and endpoint experience score into an average would reduce visibility of underlying individual experience impacts.
The future of XLAs and XPIs is currently undetermined, and the approach is likely to be specific to individual companies in the near to mid future. Where XLAs and XPIs conform more to SLA and KPI standards, the XLA term may be deemed redundant as SLAs and KPIs focused on experience and outcome become more common. Where operating models are transformed more dramatically, XLAs and XPIs may replace SLAs and KPIs entirely.
One of the primary factors determining how XLAs and XPIs will be used is how well enterprises can translate user experience and outcomes to business outcomes. For instance, can the enterprise translate increased user productivity into business savings and revenues or otherwise recognize the value in the soft benefits at a senior level?
Figure 4 represents four potential outcomes for XLAs and XPIs. Determining where enterprises are placed on this figure will be partially based on digital workplace maturity and how impacted they are by the shifts in the market. IBM expects many clients will be in the forward looking and progressive categories, although the number of enterprises in the pioneering category is increasing.
Traditional digital workplace metrics are used. XLAs and XPIs are not included. Enterprises will continue using existing SLAs and KPIs.
XLAs and XPIs become new SLAs and KPIs. No traditional SLAs are replaced. Enterprises will not change internal targets and expect service providers to provide XLAs and XPIs in addition to traditional SLAs and KPIs. During procurement processes, cost of services will likely take significant priority over XLAs and XPIs.
XLAs and XPIs become new SLAs and KPIs. Some traditional SLAs are replaced. Enterprises may assign XLA and XPI targets to their internal teams. Commercial models will remain largely unchanged but will include XLAs and XPIs, and organizations will be willing to remove or reduce older SLA and KPI targets that add unnecessary burdens. For example, if the user effort score is good, measuring average speed to answer provides little value and can increase costs.
Commercial models are XLA and XPI focused. Many traditional SLAs are removed. Enterprises will target internal teams based on XLAs and XPIs. Procurement decisions will focus on XLAs and XPIs rather than targeting fixed cost reductions. Targets and budgets will be based on improving user outcomes and, for example, could include variable charges and incentives based on levels of proactive fixes rather than fixed user costs or incident ticket volumes. This could lead to the removal of many traditional SLAs.
