Anomaly-Transformer - Innovative Time Series Anomaly Detection Using Association Discrepancy

Introduction to Anomaly Transformer: Time Series Anomaly Detection with Association Discrepancy

The Anomaly Transformer is a groundbreaking project designed to tackle the unsupervised detection of anomalies within time series data. Presented at the ICLR 2022 conference, this project introduces an innovative methodology that makes detecting anomalies more effective and precise. Understanding and identifying unusual patterns in data, known as anomalies, pose a significant challenge. These anomalies can often indicate critical issues such as system faults or fraud. The Anomaly Transformer addresses this challenge through a novel approach, focusing on three key aspects:

Core Contributions

Association Discrepancy as a Detection Criterion: The Anomaly Transformer defines a unique criterion for detecting anomalies known as the Association Discrepancy. This criterion forms the foundation for distinguishing between normal and abnormal data behaviors.
Anomaly-Attention Mechanism: A new attention mechanism is introduced to compute the association discrepancy effectively. This mechanism allows the model to pay special attention to potential anomalies in the data, making it a crucial part of the detection process.
Minimax Strategy: To enhance the model's ability to differentiate between normal and anomalous data, a minimax strategy is employed. This strategy works by maximizing the distinction between normal and abnormal data, thereby improving the model's accuracy.

Structure

How to Get Started

To begin experimenting with the Anomaly Transformer, one should follow these steps:

Installation: Ensure the system has Python 3.6 and PyTorch version 1.4.0 or newer installed. This setup is crucial for running the Anomaly Transformer effectively.
Data Acquisition: Four benchmark datasets are available for download from Google Cloud. These datasets are pre-processed for ease of use. The SWaT dataset can be accessed through its official tutorial.
Training and Evaluation: Predefined scripts are provided to facilitate training and evaluation across different datasets. Users can reproduce experiments and results by running scripts located in the ./scripts directory, such as:
```
bash ./scripts/SMD.sh
bash ./scripts/MSL.sh
bash ./scripts/SMAP.sh
bash ./scripts/PSM.sh
```

For evaluation, an adjustment operation as proposed by Xu et al., 2018 is utilized, ensuring robust model assessment.

Key Findings

The Anomaly Transformer’s performance was compared with 15 other baseline models, including THOC and InterFusion. Remarkably, the Anomaly Transformer consistently outperformed these models, establishing itself as the state-of-the-art (SOTA) in time series anomaly detection.

Results

Citation

Researchers and developers interested in utilizing or citing the Anomaly Transformer can refer to the following citation format:

@inproceedings{
xu2022anomaly,
title={Anomaly Transformer: Time Series Anomaly Detection with Association Discrepancy},
author={Jiehui Xu and Haixu Wu and Jianmin Wang and Mingsheng Long},
booktitle={International Conference on Learning Representations},
year={2022},
url={https://openreview.net/forum?id=LzQQ89U1qm_}
}

Contact

For further questions or assistance, you can reach out via email to [email protected].

The Anomaly Transformer provides a sophisticated yet accessible solution for anomaly detection in time series data, proving to be an invaluable tool for researchers and practitioners alike.