Questions tagged [anomaly-detection]
In data mining, anomaly detection (also outlier detection) is the identification of items, events or observations which do not conform to an expected pattern or other items in a dataset.
anomaly-detection
448
questions
-1
votes
0
answers
26
views
How to apply augmentations to batch on each step of epoch in Keras?
I'm trying to solve an anomaly detection task using an autoencoder model, but I'm getting poor results after training. My teacher suggested I use data augmentation to batch on each step during epoch. ...
- 37
0
votes
0
answers
17
views
Opensearch anomaly detection based on elevated log ingestion rate/severity
I currently use Opensearch to ingest container logs directly from Docker/Nomad/Kubernetes containers/jobs/pods using the Fluentd logging driver. Each log entry is automatically tagged with the name of ...
- 1,074
-2
votes
0
answers
17
views
Time-serie outlier detection using LLM
I want to preform outliers detection on time-serie data using an LLM (the use of LLM is critical). so basically i have a simple dataframe containing the time serie data (timestamp column, and value ...
-2
votes
0
answers
55
views
Change point detection in a timeseries
I have a few questions about using the PELT algorithm for change point detection in the ruptures library.
Optimal Penalty Value for PELT Algorithm
I'm using the Pruned Exact Linear Time (PELT) ...
- 1
-1
votes
1
answer
42
views
Random Cut Forest on Multiple Timeseries anomaly detection
I am trying to find anomaly on multiple sensor data using random cut forest, how can i use random cut forest for multi times series data.
i have created a model for one time series data and that works ...
- 9
0
votes
0
answers
17
views
OpenSearch Anomaly Detector Missing Real-Time Anomalies Despite Successful Historical Detection
We created an anomaly detector using OpenSearch with a 10-minute window interval, focusing on three features: total_count, logstats, and flowstats, using sum aggregation and a shingle size of 8.
To ...
2
votes
1
answer
100
views
Confused with Isolation Forest
Let say, I have the anomaly detection (unsupervised learning) dataset with 10 observations (two features). The datasets is like below:
After executing the model, following are the results (anomalies ...
- 309
0
votes
0
answers
7
views
Input mismatch in dense layer
I am using this autoencoder model to detect anomaly.
class AnomalyDetector(Model):
def __init__(self):
super(AnomalyDetector, self).__init__()
self.encoder = tf.keras.Sequential([
...
- 1
0
votes
0
answers
15
views
Best normalization method for auto-encoder
I am building an intrusion detection system to detect malicious traffic in my network using an AutoEncoder. I've been training my model to learn from benign traffic and minimize the Mean Squared Error ...
0
votes
0
answers
35
views
"Incompatible dimensions for a grouped binary operation" error in computing anomaly using climatological data in Python
As a beginner in Python, I am trying to compute monthly anomalies of temperature from AIRS-based timeseries observations with reference to monthly climatological (1950-1980) data from ecmwf. AIRS data ...
0
votes
0
answers
18
views
How can I get an AWS anomaly at the resource-level rather than service-level?
Currently, I am using the CostExplorer API to retrieve anomalies within AWS. However, when I call
get_anomalies(), this is the lowest level of detail I can get from the anomaly:
'RootCauses': [
...
- 1
0
votes
0
answers
23
views
Why not use the anomaly datasets to train an autoencoder?
When using autoencoders for detecting network anomalies, why not train them using anomaly datasets? Reconstruction errors smaller than a threshold could signify anomalies, while those larger than the ...
- 1
0
votes
1
answer
38
views
Reproducible Training of an autoencoder in Tensorflow
I tried to implement an autoencoder-based anomaly detector finding anomalies in the dataset KDDTrain+. This is actually a pretty straight forward implementation. Unfortunately I failed in implementing ...
- 1
0
votes
0
answers
27
views
Surprising but confusing ML results
can anyone help me in understanding the following issue, please?
I am working on anomaly detection using supervised machine learning technique. I am surprised and badly fed-up with results, as for one ...
1
vote
1
answer
46
views
How to realign Python fill_between with data points
I have been working on a project that involves time-series total electron content data. My goal is to apply statistical analysis and find anomalies in TEC due to earthquake.
I am following this ...
- 61