Virtual War Rooms to spend up Incident Response

Devops is not just a technology, it's also changing our work processes and culture. The "war room" is a traditional method of problem solving across different functional teams. What was once a physical location has now gone virtual, using ChatOps and collaboration platforms.

When a major incident occurs, many teams prefer to gather all subject matter experts (SMEs) to resolve the problem as quickly as possible. Having everyone in the same space can decrease the chances of lapses in communication or delays in coordination.

Before the rise of remote work, many teams had physical war rooms within their offices. These were conference rooms that were purposefully arranged and stocked for major incidents. However, as many organizations shift to distributed teams, this version of the war room is becoming less common. Instead, virtual war rooms are how many teams now communicate during response.

Benefits of an Virtual war rooms

As Agile methodology became a commonly adopted method of working, organizations began using war rooms regularly. This makes sense, as Agile touts the importance of understanding and optimizing how humans interact with technology. Communication and adaptability are two of the most important parts of incident response, and ones that require humans to successfully execute. War rooms were created to enhance this. Here are some of the benefits of a war room, whether it’s physical or virtual:

• Everyone you need is right there: Major incidents usually involve multiple teams. War rooms bring together these disparate teams and provide the space and place for SMEs to work together rather than siloed.

• You can communicate and clarify quickly: Communication works both ways, and responders resolving complex problems will often have questions or suggestions. War rooms ensure there are no delays in communicating these.

• War rooms eliminate distractions: Normal work can be interrupted if necessary. Teammates and coworkers can drop by your desk or ask you to jump on a call. But these asks are almost never as urgent as the major incident. War rooms ensure that responders are able to work as free from distractions as possible. For everyone outside the war room, it’s a clear sign to leave the responders to their work.

• As the incident changes shape, you can adapt: As an incident progresses, it can change shape, scope, teams, and more. Rarely are major incidents straightforward. With a war room, these changes can be communicated quickly and the team can formulate a new plan of action within minutes rather than playing a game of telephone where crucial information is lost.

• Stakeholders remain informed without interrupting the process: Stakeholders are people who may not actively be involved in resolving the issue, but who need to know what’s going on to coordinate with other parts of the business. With a war room, responders can regularly update these stakeholders without needing to track individuals down. Additionally, stakeholders know where to go for information and are less likely to interrupt the response process asking for updates if the updates are already being provided regularly.

Best practices for virtual war rooms

• Create backup communication methods. In the event that an incident causes your primary communication method to fall through, you need to be prepared with a backup. This backup needs to be well known and documented so that all involved parties can participate without scrambling to find the right video conferencing link or sending multiple emails.

• Familiarize everyone with the tools you use. If you use a tool or platform to manage incident response, do all teams know how to use it? Train relevant teams ahead of time so nobody has to learn a new tool during an incident.

• Have well-established communication guidelines. What you use to communicate is important, but understanding how you communicate is just as crucial. Make sure you document how teams are expected to collaborate. Are separate direct messages okay, or should you share any thoughts you have with the team via the video conference? Are stakeholders updated via email or messaging tool, or should they drop into the virtual war room at a certain time each hour? Setting these standards ahead of time will prevent confusion.