Trend-based alerting using the EWMA algorithm

Trend-based alerting using the EWMA algorithm

While configuring AppLogs alerts, you have to choose if your alert should be count-based or trend-based. For trend-based alerts, to calculate the average values, Site24x7 uses the Exponentially Weighted Moving Average (EWMA) algorithm to identify abnormal spikes and changes in the log trend.

What is the EWMA algorithm?

The EWMA refers to an average of data that is used to track the movement of the value by checking the results and output. It considers different factors, assigns a weight to the value, tracks results to evaluate performance, and recommends improvements.

The weight for an EWMA is reduced exponentially as it is further removed over time. Also, since the EWMA contains the previously calculated average, the result of the EWMA will be cumulative. Because of this, all the data points will be contributing to the result, but the contribution factor will be reduced in the next period in which the EWMA is calculated.

The moving average EWMA for a given time t is calculated as follows:
EWMA(t) = a * x(t) + (1-a) * EWMA(t-1)

Here,
EWMA(t) = moving average at time t.
a = degree of mixing parameter value between 0 and 1. This parameter shows the rate at which the older data will come into calculation.
x(t) = value of an alert query output at time t.

If a=1, only the most recent data has been used to measure the EWMA. 
If a is nearing 0, then more weightage is given to older data.
If a is nearing 1, then more weightage is given to newer data.

Use case: IIS access logs

Let's consider the use case of Internet Information Services (IIS) access logs with a 500 status code error. In this case, when trend-based alerts are configured with a check frequency of one hour, the trend will be observed for the configured number of days, and you'll receive alerts based on the moving average value if there is a sudden increase in the exception count.

For example, let us consider the exception count for every hour as shown below:
a = 0.3 (Site24x7 fixes the value of a as 0.3 since it gives weightage to older data) 
Time (t)
Value at t
EWMA
Percentage growth
10am
27
27
0
11am
30
27.9
11.11
Noon
40
31.53
43.37
1pm
55
38.57
74.44
2pm
453
162.9
1074.49
3pm
105
145.53
-35.54

If you have configured a trend-based alert for your log search queries, to receive a notification when a 100 percent threshold is reached, you will receive an alert when the current exception count value growth is >100 percent when compared to the previous EWMA value.  

Percentage Growth = (t -old EWMA)/old EWMA * 100
Here, at 2pm,
Percentage growth= (453-38.57)/38.57*100 = 1074.48 %

In this case, you will receive the alert at 2pm since the exception count growth has increased to 1074 percent when compared to the previous EWMA value of 38.57. 


    • Related Articles

    • Configuring voice calls and SMS alerts for AppLogs

      Set up e-mail, SMS, voice call, and instant messenger alerts for AppLogs Alerts (that you configure in the AppLogs tab). Log in to your Site24x7 account. Click Admin > User and Alert Management > Users and Alerts. In the Users screen, you can edit an ...
    • How do I disable the AppLogs agent (process/service)?

      When you disable the AppLogs agent, log collection on that server will stop and the AppLogs Agent service/process will not run in your server. To disable the AppLogs agent: Navigate to the Server tab. Click on the server monitor in which the agent ...
    • How to manually install the AppLogs agent

      The AppLogs agent is used to automatically discover and upload your server and application logs to Site24x7. Site24x7's AppLogs agent works with an existing Site24x7 server monitoring agent and comes with AppLogs installed. You can also manually ...
    • Average resource utilization by AppLogs agent

      Metrics Windows Linux CPU < 1% 0.3% Memory 40 MB 25 MB Peak network 2 MB in 30 seconds 2 MB in 30 seconds Disk 100 MB 100 MB This resource utilization is subject to change when more log types are associated with a single server or the same AppLogs ...
    • Stop phone call alerts on outages and opt only for SMS mode of alerting

      Site24x7 offers alert notifications via Email, SMS, Phone, and IM channels. The alert settings can be customized by logging to Site24x7 and navigating to Admin > User Management > Users > (Select the user's name to customize) > 'Alert Settings' If ...