Researchers at the MIT AI Lab and IBM Watson have developed a machine learning method that automatically detects anomalies in power grids in real time.

The scientists explained that an anomaly is an event, the probability of which is small, it can be, for example, a sudden power surge. Scientists process power grid data according to the law of probability distribution, that is, they can identify values ​​with low probability. Such values ​​are considered an anomaly.

Estimating probabilities is complicated by the fact that there is a lot of data, and the sensors that record information depend on each other. To study the probability distribution of the data, the researchers used a special type of deep learning model, flow normalization, which is particularly effective at estimating the probability density of a sample. They augmented this model with a Bayesian network, a graph that can handle the complex structure of cause and effect relationships between different sensors. This technique is especially powerful because the network can explore the data unsupervised.

The experimental model outperformed all baselines and showed a higher percentage of true anomaly detection in each data set. The information they used for testing contained anomalies already noted by humans, so the researchers were able to compare the anomalies detected by their model with actual failures in each system.

“Systems based on domain knowledge discovery rules, even backed up by statistical data analysis, require a lot of work and experience. We show that we can automate this process as well as extract patterns from data using advanced machine learning techniques,” says senior author Jie Chen, research fellow and manager of the MIT-IBM Watson Artificial Intelligence Lab.

The developed methodology is flexible. Armed with a large untagged dataset, scientists can tune the model to predict anomalies in other situations, such as traffic patterns, for example.