Trend-based alerting using the EWMA algorithm

Trend-based alerting using the EWMA algorithm

While configuring AppLogs alerts, you have to choose if your alert should be count-based or trend-based. For trend-based alerts, to calculate the average values, Site24x7 uses the Exponentially Weighted Moving Average (EWMA) algorithm to identify abnormal spikes and changes in the log trend.

What is the EWMA algorithm?

The EWMA refers to an average of data that is used to track the movement of the value by checking the results and output. It considers different factors, assigns a weight to the value, tracks results to evaluate performance, and recommends improvements.

The weight for an EWMA is reduced exponentially as it is further removed over time. Also, since the EWMA contains the previously calculated average, the result of the EWMA will be cumulative. Because of this, all the data points will be contributing to the result, but the contribution factor will be reduced in the next period in which the EWMA is calculated.

The moving average EWMA for a given time t is calculated as follows:
EWMA(t) = a * x(t) + (1-a) * EWMA(t-1)

EWMA(t) = moving average at time t.
a = degree of mixing parameter value between 0 and 1. This parameter shows the rate at which the older data will come into calculation.
x(t) = value of an alert query output at time t.

If a=1, only the most recent data has been used to measure the EWMA. 
If a is nearing 0, then more weightage is given to older data.
If a is nearing 1, then more weightage is given to newer data.

Use case: IIS access logs

Let's consider the use case of Internet Information Services (IIS) access logs with a 500 status code error. In this case, when trend-based alerts are configured with a check frequency of one hour, the trend will be observed for the configured number of days, and you'll receive alerts based on the moving average value if there is a sudden increase in the exception count.

For example, let us consider the exception count for every hour as shown below:
a = 0.3 (Site24x7 fixes the value of a as 0.3 since it gives weightage to older data) 
Time (t)
Value at t
Percentage growth

If you have configured a trend-based alert for your log search queries, to receive a notification when a 100 percent threshold is reached, you will receive an alert when the current exception count value growth is >100 percent when compared to the previous EWMA value.  

Percentage Growth = (t -old EWMA)/old EWMA * 100
Here, at 2pm,
Percentage growth= (453-38.57)/38.57*100 = 1074.48 %

In this case, you will receive the alert at 2pm since the exception count growth has increased to 1074 percent when compared to the previous EWMA value of 38.57. 

    • Related Articles

    • Licensing for AppLogs Alerts

      Every log type is treated as a monitor and any search query can be configured for alerting. Here's the number of search queries for which you can configure to receive alerts for free:  Log size Number of free search queries 10GB 5 ≤ 100GB 10 ≤ 500GB ...
    • Configuring voice calls and SMS alerts for AppLogs

      For the AppLogs Alerts that you configure in the AppLogs tab, you can set up email, SMS, voice calls, and instant messenger alerts at the monitor level. For example, if you want to configure email or voice call alerts for your Syslog monitor, edit ...
    • How to manually install the AppLogs agent

      The AppLogs agent is used to automatically discover and upload your server and application logs to Site24x7. Site24x7's AppLogs agent works with an existing Site24x7 server monitoring agent and comes with AppLogs installed. You can also manually ...
    • How do I disable the AppLogs agent (process/service)?

      When you disable the AppLogs agent, log collection on that server will stop and the AppLogs Agent service/process will not run in your server. To disable the AppLogs agent: Navigate to the Server tab. Click on the server monitor in which the agent ...
    • Alert suppression at the monitor level – Achieve better protection against false alerts through monitor-level dependency configurations

      A network outage usually triggers hundreds of alert messages and calls to multiple personnel. However, some of those notifications will likely be redundant. Without the proper alert configurations, an outage event could quickly burn through all your ...