In an Intrusion Detection System, There are two techniques called Anomaly Detection and Behaviour Detection. I am implementing an IDS from scratch and was checking for some signatures and from some site they were given as different types of methods for detection. What is the basic difference in them? As it seems to me that both are same and therefore same signatures should be able to detect these kind of attacks.
Example for Anomaly detection as given on the site: Detecting a function call that is not part of the normal profile
Example for Behaviour detection as given on the site: Searching for any remote invocation of cmd.exe.
Now as it seems to me that both are kind of same things i.e. deviation from normal behaviour so why they were characterised as different methods??
There is indeed a difference between anomaly-based and behavioral detection. Before exploring the two, I would like to point out that the intrusion detection community uses two additional styles: misused-based (aka. signature-based) and specification-based detection, yet these are unrelated to your question.
Definition: A two-step approach that involves first training a system with data to establish some notion of normality and then use the established profile on real data to flag deviations.
Example: look at some features of benign URLs, e.g., their length, character distribution, etc., to find define what a "normal" URL looks like. With this notion of normality, you would then flag URLs that are too far off the normal URL length or have too many abnormal characters in them.
Pros:
Cons:
Definition: Looking for evidence of compromise rather than the attack itself.
Example: monitor the shell history for unset HISTFILE, a command that typically only attackers enter after compromising a machine.
Pros:
Cons: