Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

What is the difference between Real-time Anomaly Detection and Anomaly Detection?

Hence, the following derives: What isa clear the definition of Real-time Anomaly Detection?

I am investigating the field of Anomaly Detection and in many papers the approach is defined Real-time, while in many other it is simply called Anomaly Detection.

I happened to discovery, correct me whether I am wrong, that most of the so called real-time approaches are instead something like near-real-time. Specifically, they are some sort of unsupervised context-based anomaly detection on time series, where the context is almost always the bucket size. In other words, the algorithms processes micro-batches of data, hence from here follows the near-real-time.

Now, I was wondering whether there is a difference between the two kind of anomaly detection. If so, how they differ from each other and what is the threshold in the bucket size (if there is one)?

This set of questions come from the fact that I am conducting a study on performance/quality of prediction of different frameworks for Anomaly Detection and I was wondering whether this difference is substantial, since it implies two different evaluation metrics. I would like to read some certified sources regarding this matter.

like image 544
dadadima Avatar asked Feb 04 '20 09:02

dadadima


2 Answers

Interestingly, I've recently thought of some similar topics for a hobby project and found some interesting blogs by Crunchmetrics, a company specializing in ML based anomaly detection. The gist:

Real time - there is a training or baseline dataset which a system can reference. The reference "lookup" is fast to appear as real-time, if optimized of course.

Near Real Time - has no existing training or statistical models and the system must compute baselines, data frames or ranges as it goes thus impacting the speed of decision making.

One blog I found useful... (I have no relationship with this company): anomaly blog post

like image 83
user3507825 Avatar answered Sep 26 '22 16:09

user3507825


In computer graphics real-time processing means fast enough to appear as motion. In practise this means processing an image at a minimum of 24 fps.

Applying this meaning to real-time anomaly detection we have the example of a live video feed. Where we must process the anomaly detection algorithm fast enough to keep up with the video feed. In this case the anomaly detection must be completed in under ~40ms per frame.

This would restraint has a drastic change on the trade off of quality of the anomaly detection vs speed of processing.

A quick google throws up some literature on this trade off. https://www.researchgate.net/publication/224258100_Real-time_camera_anomaly_detection_for_real-world_video_surveillance

like image 37
Culzean Avatar answered Sep 25 '22 16:09

Culzean