<!DOCTYPE html>
<!--[if lt IE 7 ]><html class="no-js ie6"><![endif]-->
<!--[if IE 7 ]><html class="no-js ie7"><![endif]-->
<!--[if IE 8 ]><html class="no-js ie8"><![endif]-->
<!--[if IE 9 ]><html class="no-js ie9"><![endif]-->
<!--[if (gt IE 9)|!(IE)]><!--> <html class="no-js" lang="en"> <!--<![endif]-->
<head>
<meta charset="utf-8">
<title>Spearhead Systems Incident Response Documentation</title>
<!-- Author and License -->
<meta name="author" content="Spearhead Systems, Inc." />
<meta name="dcterms.license" content="http://www.apache.org/licenses/LICENSE-2.0" />
<!-- Page Description -->
<meta name="description" content="A collection of information about the Spearhead Systems incident response process. Not only how to prepare new employees for on-call responsibilities, but also how to handle major incidents, both in preparation and after-work.">
<meta name="keywords" content="spearhead, incident, response" />
<meta name="robots" content="index, follow, noarchive" />
<!-- Mobile -->
<meta name="viewport" content="width=device-width, initial-scale=1.0, minimum-scale=1.0" />
<meta name="theme-color" content="#1f293a" />
<!-- Canonical Link -->
<link rel="canonical" href="https://response.spearhead.systems/">
<!-- Favicon -->
<link rel="shortcut icon" type="image/x-icon" href="./assets/img/icon.png" />
<link rel="icon" type="image/x-icon" href="./assets/img/icon.png" />
<!-- Apple -->
<meta name="apple-mobile-web-app-title" content="Spearhead Systems Incident Response Documentation" />
<meta name="apple-mobile-web-app-capable" content="yes" />
<meta name="apple-mobile-web-app-status-bar-style" content="black-translucent" />
<link rel="apple-touch-icon" href="./assets/img/icon.png">
<!-- Open Graph -->
<meta property="og:url" content="https://response.spearhead.systems/" />
<meta property="og:title" content="Spearhead Systems Incident Response Documentation" />
<meta property="og:site_name" content="Spearhead Systems Incident Response Documentation" />
<meta property="og:description" content="A collection of information about the Spearhead Systems incident response process. Not only how to prepare new employees for on-call responsibilities, but also how to handle major incidents, both in preparation and after-work." />
<meta property="og:image" content="https://response.spearhead.systems/assets/img/cover.png" />
<meta property="og:locale" content="en_US" />
<meta property="og:type" content="website" />
<!-- Twitter -->
<meta name="twitter:card" content="summary_large_image" />
<meta name="twitter:title" content="Spearhead Systems Incident Response Documentation" />
<meta name="twitter:description" content="A collection of information about the Spearhead Systems incident response process. Not only how to prepare new employees for on-call responsibilities, but also how to handle major incidents, both in preparation and after-work." />
<meta name="twitter:image" content="https://response.spearhead.systems/assets/img/cover.png" />
<!-- Style -->
<style>
@font-face {
font-family: 'Icon';
src: url('./assets/fonts/icon.eot?52m981');
src: url('./assets/fonts/icon.eot?#iefix52m981')
format('embedded-opentype'),
url('./assets/fonts/icon.woff?52m981')
format('woff'),
url('./assets/fonts/icon.ttf?52m981')
format('truetype'),
url('./assets/fonts/icon.svg?52m981#icon')
format('svg');
font-weight: normal;
font-style: normal;
}
</style>
<link rel="stylesheet" href="./assets/stylesheets/application-a422ff04cc.css">
<link rel="stylesheet" href="./assets/stylesheets/palettes-05ab2406df.css">
<link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Colfax Regular:400,700|Roboto+Mono">
<style>
body, input {
font-family: 'Colfax Regular', Helvetica, Arial, sans-serif;
}
pre, code {
font-family: 'Roboto Mono', 'Courier New', 'Courier', monospace;
}
</style>
<link rel="stylesheet" href="./assets/css/extra.css">
<!-- Scripts -->
<script src="./assets/javascripts/modernizr-4ab42b99fd.js"></script>
</head>
<body class="palette-primary-green palette-accent-blue-grey">
<div class="backdrop">
<div class="backdrop-paper"></div>
</div>
<input class="toggle" type="checkbox" id="toggle-drawer">
<input class="toggle" type="checkbox" id="toggle-search">
<label class="toggle-button overlay" for="toggle-drawer"></label>
<header class="header">
<nav aria-label="Header">
<div class="bar default">
<div class="button button-menu" role="button" aria-label="Menu">
<label class="toggle-button icon icon-menu" for="toggle-drawer">
<span></span>
</label>
</div>
<div class="stretch">
<div class="mainlogo">
<a href="/" title="Go to homepage.">
<img src="./assets/img/logo.png" title="PagerDuty" />
</a>
</div>
<div class="title">
<span class="">
Incident Response
</span>
<span class="path">
</span>
</div>
</div>
<div class="button button-twitter" role="button" aria-label="Twitter">
<a href="https://twitter.com/spearhead_sys" title="@spearhead_sys on Twitter" target="_blank" class="toggle-button icon icon-twitter"></a>
</div>
<div class="button button-github" role="button" aria-label="GitHub">
<a href="https://github.com/spearheadsys" title="@spearheadsys on GitHub" target="_blank" class="toggle-button icon icon-github"></a>
</div>
<div class="button button-search" role="button" aria-label="Search">
<label class="toggle-button icon icon-search" title="Search" for="toggle-search"></label>
</div>
</div>
<div class="bar search">
<div class="button button-close" role="button" aria-label="Close">
<label class="toggle-button icon icon-back" for="toggle-search"></label>
</div>
<div class="stretch">
<div class="field">
<input class="query" type="text" placeholder="Search" autocapitalize="off" autocorrect="off" autocomplete="off" spellcheck="false">
</div>
</div>
<div class="button button-reset" role="button" aria-label="Search">
<button class="toggle-button icon icon-close" id="reset-search"></button>
</div>
</div>
</nav>
</header>
<main class="main">
<div class="drawer">
<nav aria-label="Navigation">
<a href="https://github.com/spearheadsys/issue-response-docs" class="project">
<div class="banner">
<div class="logo">
<img src="./assets/img/icon.png">
</div>
<div class="name">
<strong>
Spearhead Systems Incident Response Documentation
<span class="version">
</span>
</strong>
<br>
spearheadsys/issue-response-docs
</div>
</div>
</a>
<div class="scrollable">
<div class="wrapper">
<ul class="repo">
<li class="repo-download">
<a href="https://github.com/spearheadsys/issue-response-docs/archive/master.zip" target="_blank" title="Download" data-action="download">
<i class="icon icon-download"></i> Download
</a>
</li>
<li class="repo-stars">
<a href="https://github.com/spearheadsys/issue-response-docs/stargazers" target="_blank" title="Stargazers" data-action="star">
<i class="icon icon-star"></i> Stars
<span class="count">–</span>
</a>
</li>
</ul>
<hr/>
<div class="toc">
<ul>
<li>
<a class="current" title="Home" href=".">
Home
</a>
<ul>
<li class="anchor">
<a title="Being On-Call" href="#being-on-call">
Being On-Call
</a>
</li>
<li class="anchor">
<a title="Before an Incident" href="#before-an-incident">
Before an Incident
</a>
</li>
<li class="anchor">
<a title="During an Incident" href="#during-an-incident">
During an Incident
</a>
</li>
<li class="anchor">
<a title="After an Incident" href="#after-an-incident">
After an Incident
</a>
</li>
<li class="anchor">
<a title="Training" href="#training">
Training
</a>
</li>
<li class="anchor">
<a title="Additional Reading" href="#additional-reading">
Additional Reading
</a>
</li>
</ul>
</li>
<li>
<span class="section">On-Call</span>
<ul>
<li>
<a class="" title="Being On-Call" href="oncall/being_oncall/">
Being On-Call
</a>
</li>
<li>
<a class="" title="Alerting Principles" href="oncall/alerting_principles/">
Alerting Principles
</a>
</li>
</ul>
</li>
<li>
<span class="section">Before an Incident</span>
<ul>
<li>
<a class="" title="Severity Levels" href="before/severity_levels/">
Severity Levels
</a>
</li>
<li>
<a class="" title="Different Roles" href="before/different_roles/">
Different Roles
</a>
</li>
<li>
<a class="" title="Call Etiquette" href="before/call_etiquette/">
Call Etiquette
</a>
</li>
</ul>
</li>
<li>
<span class="section">During an Incident</span>
<ul>
<li>
<a class="" title="During An Incident" href="during/during_an_incident/">
During An Incident
</a>
</li>
<li>
<a class="" title="Security Incident" href="during/security_incident_response/">
Security Incident
</a>
</li>
</ul>
</li>
<li>
<span class="section">After an Incident</span>
<ul>
<li>
<a class="" title="Post-Mortem Process" href="after/post_mortem_process/">
Post-Mortem Process
</a>
</li>
<li>
<a class="" title="Post-Mortem Template" href="after/post_mortem_template/">
Post-Mortem Template
</a>
</li>
</ul>
</li>
<li>
<span class="section">Training</span>
<ul>
<li>
<a class="" title="Overview" href="training/overview/">
Overview
</a>
</li>
<li>
<a class="" title="Incident Commander" href="training/incident_commander/">
Incident Commander
</a>
</li>
<li>
<a class="" title="Deputy" href="training/deputy/">
Deputy
</a>
</li>
<li>
<a class="" title="Scribe" href="training/scribe/">
Scribe
</a>
</li>
<li>
<a class="" title="Subject Matter Expert" href="training/subject_matter_expert/">
Subject Matter Expert
</a>
</li>
<li>
<a class="" title="Glossary" href="training/glossary/">
Glossary
</a>
</li>
</ul>
</li>
<li>
<a class="" title="About" href="about/">
About
</a>
</li>
</ul>
</div>
</div>
</div>
</nav>
</div>
<article class="article">
<div class="wrapper">
<h1>Spearhead Systems Incident Response Documentation</h1>
<p>This documentation covers parts of the Spearhead Systems Issue Response process. It is a copy of <a href="https://github.com/PagerDuty/incident-response-docs/">PagerDuty's</a> documentation and furthermore a cut-down version of our own internal documentation, used at Spearhead Systems for any issue (incident or service request), and to prepare new employees for on-call responsibilities. It provides information not only on preparing for an incident, but also what to do during and after. It is intended to be used by on-call practitioners and those involved in an operational incident response process (or those wishing to enact a formal incident response process). See the <a href="about/">about page</a> for more information on what this documentation is and why it exists. This documentation is complementary to what is available in our <a href="https://sphsys.sharepoint.com">existing wiki</a> and may not yet be open sourced.</p>
<div class="admonition note">
<p class="admonition-title">Issue, Incident and Service Request</p>
<p>At Spearhead we use the term <em>issue</em> to define any request from our customers. Issues fall into two categories: "Service Requests (SR)" and "Incidents (IN)". Note that we use the term Incident to describe both a service request as well as incidents. For brevity we will use SR and IN throughout this documentation.</p>
</div>
<p>A "service request" is usually initiated by a human and is generally not critical for the normal functioning of the business while an "incident" is an issue that is or can cause interruption to normal business functions. </p>
<p><img alt="Issue Response at Spearhead Systems" src="./assets/img/headers/sph_ir.jpg" /></p>
<h2 id="being-on-call">Being On-Call<a class="headerlink" href="#being-on-call" title="Permanent link">#</a></h2>
<p>If you've never been on-call before, you might be wondering what it's all about. These pages describe what the expectations of being on-call are, along with some resources to help you.</p>
<ul>
<li><a href="oncall/being_oncall/">Being On-Call</a> - <em>A guide to being on-call, both what your responsibilities are, and what they are not.</em></li>
<li><a href="oncall/alerting_principles/">Alerting Principles</a> - <em>The principles we use to determine what things page an engineer, and what time of day they page.</em></li>
</ul>
<h2 id="before-an-incident">Before an Incident<a class="headerlink" href="#before-an-incident" title="Permanent link">#</a></h2>
<p>Reading material for things you probably want to know before an incident occurs. You likely don't want to be reading these during an actual incident.</p>
<ul>
<li><a href="before/severity_levels/">Severity Levels</a> - <em>Information on our severity level classification. What constitutes a Low issue? What's a "Major Incident"?, etc.</em></li>
<li><a href="before/different_roles/">Different Roles for Incidents</a> - <em>Information on the roles during an incident; Incident Commander, Scribe, etc.</em></li>
<li><a href="before/call_etiquette/">Incident Call Etiquette</a> - <em>Our etiquette guidelines for incident calls, before you find yourself in one.</em></li>
</ul>
<h2 id="during-an-incident">During an Incident<a class="headerlink" href="#during-an-incident" title="Permanent link">#</a></h2>
<p>Information and processes during an incident.</p>
<ul>
<li><a href="during/during_an_incident/">During an Incident</a> - <em>Information on what to do during an incident, and how to constructively contribute.</em></li>
<li><a href="during/security_incident_response/">Security Incident Response</a> - <em>Security incidents are handled differently to normal operational incidents.</em></li>
</ul>
<h2 id="after-an-incident">After an Incident<a class="headerlink" href="#after-an-incident" title="Permanent link">#</a></h2>
<p>Our followup processes, how we make sure we don't repeat mistakes and are always improving.</p>
<ul>
<li><a href="after/post_mortem_process/">Post-Mortem Process</a> - <em>Information on our post-mortem process; what's involved and how to write or run a post-mortem.</em></li>
<li><a href="after/post_mortem_template/">Post-Mortem Template</a> - <em>The template we use for writing our post-mortems for major incidents.</em></li>
</ul>
<h2 id="training">Training<a class="headerlink" href="#training" title="Permanent link">#</a></h2>
<p>So, you want to learn about incident response? You've come to the right place.</p>
<ul>
<li><a href="training/overview/">Training Overview</a> - <em>An overview of our training guides and additional training material from third-parties.</em></li>
<li><a href="training/incident_commander/">Incident Commander Training</a> - <em>A guide to becoming our next Incident Commander.</em></li>
<li><a href="training/deputy/">Deputy Training</a> - <em>How to be a deputy and back up the Incident Commander.</em></li>
<li><a href="training/scribe/">Scribe Training</a> - <em>A guide to scribing.</em></li>
<li><a href="training/subject_matter_expert/">Subject Matter Expert Training</a> - <em>A guide on responsibilities and behavior for all participants in a major incident.</em></li>
<li><a href="training/glossary/">Glossary of Incident Response Terms</a> - <em>A collection of terms that you may hear being used, along with their definition.</em></li>
</ul>
<h2 id="additional-reading">Additional Reading<a class="headerlink" href="#additional-reading" title="Permanent link">#</a></h2>
<p>Useful material and resources from external parties that are relevant to incident response.</p>
<ul>
<li><a href="http://shop.oreilly.com/product/0636920036159.do">Incident Management for Operations</a> (O'Reilly)</li>
<li><a href="http://shop.oreilly.com/product/9780596001308.do">Incident Response</a> (O'Reilly)</li>
<li><a href="http://extfiles.etsy.com/DebriefingFacilitationGuide.pdf">Debriefing Facilitation Guide</a> (Etsy)</li>
<li><a href="https://www.fema.gov/national-incident-management-system">US National Incident Management System (NIMS)</a> (FEMA)</li>
<li><a href="https://www.heavybit.com/library/video/every-minute-counts-coordinating-herokus-incident-response/">Every Minute Counts: Leading Heroku's Incident Response</a> (Blake Gentry)</li>
</ul>
<aside class="copyright" role="note">
Copyright © Spearhead Systems, Inc. –
Documentation built with
<a href="http://www.mkdocs.org" target="_blank">MkDocs</a>
using the
<a href="http://squidfunk.github.io/mkdocs-material/" target="_blank">
Material
</a>
theme.
</aside>
<footer class="footer">
<nav class="pagination" aria-label="Footer">
<div class="previous">
</div>
<div class="next">
<a href="oncall/being_oncall/" title="Being On-Call">
<span class="direction">
Next
</span>
<div class="page">
<div class="stretch">
<div class="title">
Being On-Call
</div>
</div>
<div class="button button-next" role="button" aria-label="Next">
<i class="icon icon-forward"></i>
</div>
</div>
</a>
</div>
</nav>
</footer>
</div>
</article>
<div class="results" role="status" aria-live="polite">
<div class="scrollable">
<div class="wrapper">
<div class="meta"></div>
<div class="list"></div>
</div>
</div>
</div>
</main>
<script>
var base_url = '.';
var repo_id = 'spearheadsys/issue-response-docs';
</script>
<script src="./assets/javascripts/application-997097ee0c.js"></script>
</body>
</html>