Chapter 1.1: The Importance of IT Service Management

IT Service Management (ITSM) is a critical organizational function that utilizes innovation to drive value, enhance user productivity, and provide comprehensive tech services. Despite its significant role, interactions with IT support can often involve lengthy and cumbersome conversations via web or phone. Whether you're requesting a password reset, submitting application configuration changes, or seeking assistance, the experience can often be frustrating. The challenge arises because IT leaders find it difficult to adequately staff and support a holistic help-desk management system for the entire organization.

Even with good intentions, support teams frequently struggle with efficient ITSM. In a recent project for a global medical device client, I was tasked with addressing the challenges surrounding enterprise incident ticket triage. Leadership opted to invest in an expensive legacy solution using ServiceNow (a well-known ITSM workflow platform) and an external vendor to enhance response times, but with little success. The rigid SLA requirements and generalized usage led to inaccuracies, causing tickets to bounce around among business groups. Users were frustrated, support staff were overwhelmed, and leadership was left perplexed.

In 2019, over 60,000 tickets were submitted to my client's ServiceNow platform, targeting nearly 15 different business units. Each ticket incurred a cost of $13 for the IT organization, yet the average accuracy score for reaching the intended destination was only 40%. Misassigned tickets lingered in the system for an average of 21 days before being properly directed. The issues of cost, delay, and accuracy resulted in poor user experiences.

When a user submits a support ticket, it enters the system via email, phone, or an embedded portal. Each ticket includes a description of the issue or request, which a support professional quickly reviews before passing it along. Once the appropriate assignment group receives the ticket, some amount of work gets completed, and the ticket is eventually marked as closed.

This situation presents a prime opportunity for Multinomial Classification through Supervised Machine Learning to categorize support tickets according to a defined set of business groups. We can efficiently extract text and category data from each ticket and train a model to associate specific words and phrases with designated categories. My hypothesis is straightforward: machine learning can yield immediate cost savings, improved SLA results, and more accurate predictions compared to human efforts. Let's dive in!

Chapter 1.2: Data Collection and Analysis

Before selecting and training our machine learning models, we must first analyze the data to identify trends in incident tickets. ServiceNow offers a robust Table API framework that enables us to extract data directly from the platform.

# Import Statements

import requests

import json

import pandas as pd

# Initialize url

# Set simple authorization

user = "{username}"

pwd = "{password}"

# Set proper headers

headers = {"Content-Type": "application/json",

"Accept": "application/json",

"accept-encoding": "gzip, deflate, br"}

# Initialize GET response

response = requests.get(url, auth=(user, pwd), headers=headers)

data = json.loads(response.text)

dataframe = pd.DataFrame(data['result'])

ServiceNow's embedded API Explorer facilitates an in-depth exploration of RESTful API tuning, allowing users to create custom API requests easily. You can access popular tables (like incident and task) or formulate complex queries, making it a valuable tool for data professionals.

As we focus on linking text to relevant classifiers, we can leverage a categorical variable such as "u_portfolio" to label each row in our dataframe. Given a significant class imbalance (with "Global Support Services" representing nearly 65% of all records) and over 2,000 missing values, we aim to eliminate categories with fewer than 100 tickets to reduce noise and concentrate on relevant classifications. We will create a consolidated text column named "text" by merging "short_description" and "description". Visualizing the updated dataframe is essential!

import matplotlib.pyplot as plt

import seaborn as sns

# Eliminate categories with fewer than 100 tickets

classifier = "u_portfolio"

ticket_threshold = 100

df_classifiers = df[df.groupby(classifier).transform(len) > ticket_threshold]

# Print number of relevant categories & shape

print("Categories: " + str(df_classifiers[classifier].nunique()))

# Plot the classifiers

fig = plt.figure(figsize=(10, 6))

sns.barplot(df_classifiers[classifier].value_counts().index, df_classifiers[classifier].value_counts())

plt.xticks(rotation=20)

plt.show()

After setting the threshold to 100 tickets, we have eliminated over five categories that lacked sufficient business value. Upon reviewing the data, I confirmed that the removed categories had not been utilized in over a year and could be safely disregarded.

It's worth noting the class imbalance in enterprise consulting: Global Support Services (GSS) accounts for more than 60% of all support tickets. This means we could create a simple program to assign GSS to every incoming ticket and still be correct over half the time!

The remaining categories will serve as labels for training and testing the model. Let's save them as a list:

category_labels = list(df_classifiers[classifier].value_counts().index)

Now that we have our category labels, we need to analyze text patterns for each ticket type. By examining the ServiceNow platform, I quickly identified several themes by category: GSS typically handles password resets and hardware issues, Business Intelligence focuses on reporting and data inquiries, Customer deals with SalesForce and other customer-related applications, and SAP S/4 Security manages all ERP-related access and configuration. If you're familiar with this corporate environment, these patterns likely resonate with you.

However, the most frequently occurring words showed minimal differences across categories. I discovered that over 75% of ticket submissions came through email, and all internal employees included a confidentiality notice in their signatures, obscuring significant differences between categories. We could experiment with adjusting the parameters to uncover other patterns or hard-code the email signature into a STOPLIST variable, but this would not address the fundamental issue. Instead, we aim to identify words and phrases that correlate with each label in our category list—a process known as Term Selection.

Let's delve into some machine learning solutions for assessing and evaluating correlation!

Chapter 1.3: Model Development

Natural Language Processing (NLP) lies at the intersection of computer science and linguistics, defining how machines and human languages can interact. Essentially, NLP processes human language by analyzing and manipulating data (often in text form) to extract meaning. To achieve this, we convert data shared between humans into a numeric format that's machine-readable. This process, known as Vectorization, enables computational operations like applying mathematical rules and performing matrix calculations that yield valuable insights.

While there are exciting emerging methods for vectorizing text data for NLP, such as Transfer Learning and advanced Neural Networks, we will utilize a more straightforward technique called Term Frequency—Inverse Document Frequency (TF-IDF). TF-IDF values increase in proportion to how often a word or phrase (n-gram) appears in a document, adjusted for the total number of documents. Although it may sound complex, it effectively highlights the importance of an n-gram to a document without favoring frequently occurring words, making it especially useful for processing imbalanced text datasets like ours.

Now that we grasp how computers process text data, we can explore different models! Below is some starter code to experiment with several options:

We will use Logistic Regression as our model of choice. As a data scientist, it's essential to apply various techniques to a project and select the one best suited to the task. In my experience, human psychology significantly influences the success of a use case; guiding an organization to embrace emerging, disruptive technologies requires time and effort! Marketing, branding, and effectively communicating your solution are just as crucial as developing an elegant algorithm. Let's build our model!

For this project, I had numerous opportunities to discuss concepts methodically and address questions in real time. A significant aspect of my success criteria was ensuring leadership understood and disseminated these data science concepts in context. Given that many other algorithms/models could enhance performance based on the data, I encourage you to experiment!