Routing alerts
LogicMonitor allows you to route alert notifications to different teams within your organization, so that the right people are notified when an issue occurs.
Escalation Chains
Escalation chains tell LogicMonitor what people, or what groups of people, alert notifications should be sent to. You can create and manage your escalation chains from the Settings tab of your LogicMonitor account. Each escalation chain is made up of one or more stages of recipients, where each of these stages includes at least one user or a group of users and a contact method. If the recipients in one stage are unable to deal with or respond to the alert, the alert will be automatically escalated to the recipients in the next stage in the escalation chain. This process repeats until someone responds to the alert.
Additional escalation chain configuration options include:
- Rate limiting – This feature allows you to set a maximum number of alerts that should be delivered to this escalation chain in a specified period of time
- Time-based routing – This functionality allows you to change what recipients the alert notifications are routed to based on the time of day that the alert triggers
Alert Rules
Alert rules allow you to choose what escalation chains alerts should be routed to. Each rule has a priority, and the priority of an alert rule determines in what order the rule is checked. Triggered alerts are checked against alert rules one at a time, starting with the rule with the lowest number priority, until a rule matches the alert. When a rule matches the criteria for the triggered alert, processing stops and alert notifications are sent out to the escalation chains specified in the matching rule.
A rule will match a triggered alert when the alert level (severity) specified in the rule matches the alert severity and when the alert was triggered on a datapoint for an instance, DataSource, device/service, and group that is specified in the alert rule. Note that a * indicates that the field is wildcarded, and is equivalent to 'all'. Additional alert rule configuration options include:
- Suppress Alert Clear – Choose whether notifications should be sent to recipients when the alert clears
- Escalation Interval – The amount of time that should pass after the alert notification is sent to the first stage in an escalation chain, before it is 'escalated' and is sent to the next stage. The alert stops escalating when it is acknowledged by someone. Note that if only one stage is specified, the alert notifications will just keep resending to the same stage until the alert is acknowledged.
By default, your LogicMonitor account comes with alert rules that will match all alerts. These rules point to Escalation Chains with no recipients. This means that alerts matching these rules are not routed and will only appear in your account. If you want an individual or group notified of these alerts, define recipients for these escalation chain.
If you do keep these default rules, change their Priority to larger values to make sure they appear at the bottom. It is important that they appear after all your other custom rules because they are so general.
Routing strategy: proactive vs. reactive alerts
Receiving too many LogicMonitor alert notifications can ultimately lead to people ignoring important alerts. For this reason, we recommend that you configure your alert rules so that you are only routing alerts that someone needs to be notified for. As a rule of thumb, reactive alerts (typically critical alerts) that need to be handled immediately should be routed to different teams within your organization, and proactive alerts (typically warning alerts) that do not require immediate action should be reviewed in reports. Viewing proactive alerts in LogicMonitor reports can greatly reduce the number of alerts your team is notified for—you still want to know about these issues, but you don't necessarily want to be woken up in the middle of the night to deal with them.
