The Biggest Problem With Oncall Management, And How You Can Fix It

With the increase in digitalization of business services, the worldwide business depends on IT services more than ever before. An outage to an IT Service can affect millions of people, with real impact: They can’t pay their bills, they can’t book their flights, they can’t contact family or friends.

These IT incidents not only cost businesses $700 billion per year in North America alone — but also on the reputation of your company, your product, and your team.

More than ever, organizations need a way to instantly and accurately spin up a precise multi-team, business-wide response for major incidents and accelerate the speed of resolution, to mitigate increasingly costly impacts from unexpected disruptions. 

The Problem

This always-on, always-available expectations of digital services have increased the availability of the IT teams to be ready to provide a response around the clock. Being on-call means that a person should be able to be contacted at any time in order to investigate and fix issues that may arise for the system he/she is responsible for. This leads to anxiety in IT teams on how to be ready round the clock and balance personal life. One of the biggest challenges for teams taking on a new on-call responsibility is the reputation that on-call is disruptive to responder’s lives in a very detrimental way. No one wants to miss family events, holidays, and sleep. Before we learn on how to solve this problem, let us understand few fundamentals 

What is Oncall

On-call is the practice of designating a specific person to be available at specific on date and times to respond in the event of an IT service disruption, even though it's outside normal business hours”. 

On-call is a critical responsibility inside many Devops, Secops, NOC, IT Ops, developer, and customer support teams who run services where customers expect 24/7 availability. 

The Solution

Let's see how Oncall anxiety can be solved. Creating a better on-call experience for your team requires cultural best practices and Oncall Management software. 

Guidelines for Planning Oncall 

To alleviate the fear of going Oncall, Teams can follow below guidelines to navigate the murky waters of on-call for teams that haven’t been on call before. 

1. Clearly define the on-call schedule dates & times

   Oncall during should be clearly defined. This helps prevent burnout, confusion, and frustration. We suggest documenting your incident response process and expectations for what it means to be on call.

2. Have primary and secondary responders and responsibilities

   Life doesn’t stop just because someone is on call. Just like an unexpected personal emergency can take a developer offline during the work day, the same can happen when they’re on call. Putting a backup in place limits the potential damage from this kind of interruption. 

3. Make sure alerts are being assigned to the right person

   Getting your alerting tooling dialed in effectively shouldn’t be overlooked. Making sure to have a clear altering flow and escalation process with the right notifications and overrides can avoid a lot of headaches.

4. Fine-tune and review schedules

   Teams are not static things, neither should be your on-call schedule. We recommend a culture of continuously reviewing, adjusting, and improving your on-call practices.

5. Make sure they have access and familiarity with all the relevant diagnostics tools

   Every team varies in the tools they use to track operational health, application performance, resource utilization, etc., Make sure your on-call engineers are familiar with the tools used and have proper access to them.

Oncall Management Software 

Following are the important features to look for Oncall Management Software 

Oncall  Planning and Notification Channel preference 

Your on-call Management software can help you plan and manage the staffing schedules per team, responsibilities of oncall members with a team, and which notification routes will be most effective—whether email, text message, phone calls, chat messages, or other methods. Then help your team configure their on-call accounts with the appropriate notifications to meet their needs and response requirements. 

Readiness Reports 

"On-Call Readiness Report” helps your team get organized around notification types. The Readiness Report will look at your team members in your Oncall Management software and determine if they have set up their notifications to meet certain standards, ranging from “More than email” to “Never miss a page”. Different teams may have different preferences for how their notifications are set up based on the services they support. Some organizations may set this as a top-down mandate, or it could be an individual team decision. However you set your standards, the On-Call Readiness Report is a useful tool to ensure standardization across the team. 

 Analytics 

Both Response teams and management should be able to analyze following metrics to address the burnout of the response teams and improve their productivity. 

• Total number of Incidents received per Team 
• Average number of Oncall Members per schedule 
• Number of days or hours a specific person is Oncall
• Number of Incidents assigned to a specific Oncall Person 

The Zapoj CEM platform has a number of useful tools for you to use to make sure your team is ready to go on call