--- title: "DeleteAnomalies" language: "en" type: "Symbol" summary: "DeleteAnomalies[{example1, example2, ...}] gives a list in which examplei that are considered anomalous have been dropped. DeleteAnomalies[fun, data] drops anomalies in data using the given AnomalyDetectorFunction[...] or LearnedDistribution[...]." keywords: - delete anomalies - anomaly removal - data cleaning - dataset preprocessing - outlier detection - remove outliers - anomaly detection - data preprocessing - clean dataset - outlier removal - data anomaly cleaning - noise removal - data filtering - machine learning preprocessing - data science tools - data normalization - pre-trained anomaly detector - custom anomaly removal - anomaly detection algorithm - unsupervised anomaly detection - supervised anomaly detection - data quality improvement - data sanitization - robust statistics - handling missing data - data transformation - feature engineering - anomaly elimination - data anomaly handling - anomaly cleanup - remove noise from data - irregular data handling - anomaly resolution - data quality assurance - outlier elimination - pre-trained model integration - automatic anomaly removal - bulk data cleaning - real-time anomaly detection - batch anomaly cleaning - statistical anomaly removal - pattern recognition anomalies - machine learning anomaly handling canonical_url: "https://reference.wolfram.com/language/ref/DeleteAnomalies.html" source: "Wolfram Language Documentation" related_guides: - title: "Data Transforms and Smoothing" link: "https://reference.wolfram.com/language/guide/DataTransformsAndSmoothing.en.md" - title: "Unsupervised Machine Learning" link: "https://reference.wolfram.com/language/guide/UnsupervisedMachineLearning.en.md" - title: "Tabular Data Cleaning" link: "https://reference.wolfram.com/language/guide/TabularCleaning.en.md" related_functions: - title: "FindAnomalies" link: "https://reference.wolfram.com/language/ref/FindAnomalies.en.md" - title: "FindClusters" link: "https://reference.wolfram.com/language/ref/FindClusters.en.md" - title: "AnomalyDetection" link: "https://reference.wolfram.com/language/ref/AnomalyDetection.en.md" - title: "AnomalyDetectorFunction" link: "https://reference.wolfram.com/language/ref/AnomalyDetectorFunction.en.md" - title: "LearnDistribution" link: "https://reference.wolfram.com/language/ref/LearnDistribution.en.md" - title: "RarerProbability" link: "https://reference.wolfram.com/language/ref/RarerProbability.en.md" - title: "TransformAnomalies" link: "https://reference.wolfram.com/language/ref/TransformAnomalies.en.md" - title: "Classify" link: "https://reference.wolfram.com/language/ref/Classify.en.md" - title: "DimensionReduction" link: "https://reference.wolfram.com/language/ref/DimensionReduction.en.md" - title: "DeleteMissing" link: "https://reference.wolfram.com/language/ref/DeleteMissing.en.md" - title: "DeleteCases" link: "https://reference.wolfram.com/language/ref/DeleteCases.en.md" - title: "PlotRange" link: "https://reference.wolfram.com/language/ref/PlotRange.en.md" - title: "RemoveBackground" link: "https://reference.wolfram.com/language/ref/RemoveBackground.en.md" --- # DeleteAnomalies DeleteAnomalies[{example1, example2, …}] gives a list in which examplei that are considered anomalous have been dropped. DeleteAnomalies[fun, data] drops anomalies in data using the given AnomalyDetectorFunction[…] or LearnedDistribution[…]. ## Details and Options * ``DeleteAnomalies`` can be used on many types of data, including numerical, nominal and images. * Each ``examplei`` can be a single data element, a list of data elements or an association of data elements. Examples can also be given as a ``Dataset`` or a ``Tabular`` object. * ``DeleteAnomalies`` attempts to model the distribution of non-anomalous data in order to detect anomalies (i.e. "out-of-distribution" examples). Examples are considered anomalous when their ``RarerProbability`` is below the value specified for ``AcceptanceThreshold``. * In ``DeleteAnomalies[AnomalyDetectorFunction[…], data]``, if ``data`` comes from the same distribution as the training examples for the detector, ``AcceptanceThreshold`` corresponds to the anomaly detection false-positive rate. * The following options can be given: | | | | | ------------------------- | --------- | ----------------------------------------------------------------- | | AcceptanceThreshold | 0.001 | RarerProbability threshold to consider an example anomalous | | FeatureExtractor | Identity | how to extract features from which to learn | | FeatureNames | Automatic | feature names to assign for input data | | FeatureTypes | Automatic | feature types to assume for input data | | Method | Automatic | which modeling algorithm to use | | PerformanceGoal | Automatic | aspects of performance to optimize | | RandomSeeding | 1234 | what seeding of pseudorandom generators should be done internally | | TimeGoal | Automatic | how long to spend training the detector | | TrainingProgressReporting | Automatic | how to report progress during training | | ValidationSet | Automatic | the set of data on which to evaluate the model during training | * Possible settings for ``PerformanceGoal`` include: | | | | ----------------- | --------------------------------------------------- | | "Quality" | maximize the modeling quality of the detector | | "Speed" | maximize speed for detecting anomalies | | Automatic | automatic tradeoff among speeds, quality and memory | | {goal1, goal2, …} | automatically combine goal1, goal2, etc. | * Possible settings for ``Method`` are as given in ``LearnDistribution[…]``. * The following settings for ``TrainingProgressReporting`` can be used: | | | | ------------------- | -------------------------------------------------- | | "Panel" | show a dynamically updating graphical panel | | "Print" | periodically report information using Print | | "ProgressIndicator" | show a simple ProgressIndicator | | "SimplePanel" | dynamically updating panel without learning curves | | None | do not report any information | * ``DeleteAnomalies[…, FeatureExtractor -> "Minimal"]`` indicates that the internal preprocessing should be as simple as possible. --- ## Examples (12) ### Basic Examples (3) Drop anomalous examples in a numeric dataset: ```wl In[1]:= DeleteAnomalies[{1.2, 2.5, 3.2, 100, 4.6, 5, 5.1}] Out[1]= {1.2, 2.5, 3.2, 4.6, 5, 5.1} ``` --- Delete anomalous examples in a nominal dataset: ```wl In[1]:= DeleteAnomalies[{{"A", 1}, {"A", 1.1}, {"B", 2}, {"B", 2.5}, {"C", 3}, {"C", 4}, {"C", 4.5}, {"C", 20}}] Out[1]= {{"A", 1}, {"A", 1.1}, {"B", 2}, {"B", 2.5}, {"C", 3}, {"C", 4}, {"C", 4.5}} ``` --- Delete anomalies from a list of colors: ```wl In[1]:= DeleteAnomalies[{RGBColor[1, 0, 0], RGBColor[0, 0, 1], RGBColor[0, 1, 0], RGBColor[0.5659592309624889, 0.9770206495502198, 0.9985863063171145], RGBColor[0.3754327846016599, 0.7732937679156299, 0.9934076415002092], RGBColor[0.6161706939425584, -0.14481564113292222, 1.00353426972292], RGBColor[0.41787305847097933, 0.7295653444285688, 0.9971213930223727], RGBColor[0.38756575689146133, 0.8557000434145627, 0.9985713784934372], RGBColor[0.7237746394462758, 0.6864615128156799, 1.0049558953085165], RGBColor[0.4930933634567368, 1.0123748348501285, 0.9922647112874262], RGBColor[0.5083906475863398, 0.23671687553023163, 0.9933475855880759], RGBColor[0.5356308588856586, 0.9101219694870664, 0.9954192657779505], RGBColor[0.35371315176390594, 0.49763789511715706, 0.9991939268536737], RGBColor[0.5774616748672672, 0.43906208545640635, 1.0027712315253168], RGBColor[0.8301864069760181, 0.5690019228798344, 1.0080380129216597], RGBColor[0.7252770979304081, 0.6865831707732285, 0.9933529799272527], RGBColor[0.40911415636021287, 0.7226193316264145, 0.9966957402415179], RGBColor[0.8482079770014748, 0.8501425937052987, 0.9984394751245201], RGBColor[0.46509466164008856, 0.6496001687112135, 0.9943044940805594], RGBColor[0.5223381695947401, 0.9492448201228033, 0.9909519535914891], RGBColor[0.4056317599258231, 0.5979537199100144, 0.9960609147027923], RGBColor[0.5681282152142627, 0.6145506274472882, 0.9942053386481897], RGBColor[0.7340135990764531, 0.7363032159353553, 1.0051356464686494], RGBColor[0.568680351544366, 0.8178935615276761, 0.9972973909156875], RGBColor[0.8782992044353817, 0.5676721429387377, 0.9976822312013324], RGBColor[0.8912340489623893, 0.7418023277818317, 1.0056537818949918], RGBColor[0.5024620333700911, 0.5047283647090592, 0.9983530694802806], RGBColor[0.8078717924205181, 0.653899537453991, 0.9975680282006709], RGBColor[0.5263860266188425, 0.8891481475122383, 0.9967074371571546], RGBColor[0.4509497768400042, 0.9746743907386307, 0.9921412365780125], RGBColor[0.36709487152223774, 0.7030260587378026, 0.990996524067415], RGBColor[0.3486258167177018, 0.8782565717625381, 0.9924313945690186], RGBColor[0.2362783021030173, 1.0366668357638806, 0.9920321150148772], RGBColor[0.5037753263210084, 1.1372490894202187, 0.9970364159364161], RGBColor[0.48515674347257165, 0.623438767322434, 0.9987062896640942], RGBColor[0.309752273104561, 0.5516216087074916, 0.9979006130216704], RGBColor[0.791886226973098, 0.8555278868479104, 0.9964013988720439], RGBColor[0.36489380887741857, 0.6538971286146514, 0.9947083539138546], RGBColor[0.5788067899336176, 0.518500061180151, 1.0059894700291705], RGBColor[0.7591973658560403, 0.32754424645484553, 1.0031595845968302], RGBColor[0.611080929102045, 0.6093006791401225, 0.9968229964712378], RGBColor[0.7071196306427605, 0.6017237761211243, 1.002238791867162], RGBColor[0.607800621747923, 0.688729617245539, 0.9948471916926879], RGBColor[0.5228212118948214, 0.9621391171030302, 0.9989102310042973], RGBColor[0.4034263843628224, 0.6961712245533224, 0.9945334634229808], RGBColor[0.4624897911143086, 1.0117646816528425, 0.9976308623651218], RGBColor[0.22546551277377974, 0.49968908858854955, 0.9892293741848242], RGBColor[0.39610094617200603, 0.6406324106874157, 0.9958241193542212], RGBColor[0.2980675811368619, 0.7367494980194614, 0.9938982243446775], RGBColor[0.366116364693152, 0.5879590062582961, 0.9971279176250676], RGBColor[0.7627350568599146, 0.8758122841547133, 1.0006254489536797], RGBColor[0.5606336793064048, 0.7364374406357912, 0.9976736921570344], RGBColor[0.611217969265626, 0.4461420579128267, 0.9902433094513072], RGBColor[0.7156569387180267, 0.7123170061839005, 0.9962110198471927], RGBColor[0.6791440275858908, 0.9777377725744023, 0.9993352675306073], RGBColor[0.7535427926452951, 0.8140291014876614, 0.9999779030671445], RGBColor[0.4224730449691919, 0.7048172988308568, 1.000218197645998], RGBColor[1.0831053391201486, 0.5162623535933131, 1.0053053485793753], RGBColor[0.5908326301132645, 0.9331899955428621, 0.9993293758777952], RGBColor[0.4084175124285697, 0.5030906121830307, 0.9934012321919623], RGBColor[0.5906929070515831, 0.774539136049385, 1.0013136342783098], RGBColor[0.7279592862294082, 0.808024376838693, 0.990598201263683], RGBColor[0.4526619806424369, 0.8572369865359456, 0.992027772161482], RGBColor[0.6116378507223711, 0.9692792232881027, 1.0007282546056637], RGBColor[0.6728030121309461, 0.6159945503679183, 0.999994265026891], RGBColor[0.6187262817015476, 0.38329666431149445, 0.9983959873247816], RGBColor[0.5537139731404712, 1.00220867269463, 0.9971841114269797], RGBColor[0.5824929564854407, 0.8568652855691417, 0.9978052285631961], RGBColor[0.6467916416926835, 0.6989278893033707, 1.0040647924000072], RGBColor[0.4715405812654946, 0.48471583692401254, 0.9919917090290089], RGBColor[0.7501337350307926, 0.5478514012539901, 1.0001675930894274], RGBColor[0.7935860581270134, 0.0865495328323066, 1.0003613958535735], RGBColor[0.6957752871650936, 0.6750985254752383, 1.0028749839459759], RGBColor[0.5286599401622628, 0.752375592353536, 0.9924914932179304], RGBColor[0.7803166955792776, 0.6640679089392894, 1.0007720191828262], RGBColor[0.8149567117022473, 0.9107449749917345, 1.0018847201660235], RGBColor[0.28687387932890723, 0.5251822879837312, 1.0048453553240377], RGBColor[0.6339138155603266, 0.9499413635111722, 0.9957923550554846], RGBColor[0.6195935090234708, 0.9104921054525388, 0.9993262188604629], RGBColor[0.629437512360706, 0.7836610961087073, 0.9969560811106394], RGBColor[0.9539599691655274, 0.5727469496664113, 1.005124666562766], RGBColor[0.461135511071761, 0.7047561907538, 0.9971199066160322], RGBColor[0.6114235548448876, 1.052922842230812, 1.0000460379208387], RGBColor[0.8353121161086462, 0.4688421901010138, 1.0009262851959584], RGBColor[0.46781661731476154, 1.1396495538311269, 0.9890405978905257], RGBColor[0.8292770313746848, 0.4360701445302748, 1.0009047584693789], RGBColor[0.5170804874340198, 1.064408222280961, 1.0012411854402568], RGBColor[0.7650765408336083, 0.36936107116122646, 0.9945172422937401], RGBColor[0.45119945401989625, 0.856693556198354, 0.9957778134338929], RGBColor[0.39761045159524083, 0.6933289418577382, 0.9976912701802133], RGBColor[0.8159235329521966, 0.4299485471655765, 0.9984939390404028], RGBColor[0.357164361739695, 0.7217436860994018, 0.9966322520929208], RGBColor[0.5610070009904424, 0.8699407233165275, 0.9982375345503843], RGBColor[0.3854123195663661, 0.9861016334439296, 0.9907281406060822], RGBColor[0.6412771660095753, 0.5406870672701729, 1.0025080649577107], RGBColor[0.9426674051744555, 0.8144838170656485, 1.004881312262141], RGBColor[0.443342731164991, 0.8970900892291637, 0.9970488025463211], RGBColor[0.41572227417341934, 0.7900666387545331, 0.9906429085451064], RGBColor[0.6004982532903719, 1.0116685653582027, 0.9998841040044251], RGBColor[0.47986941698642815, 0.7665862166936707, 1.0001696262621942], RGBColor[0.7321221274856494, 0.7850360738337789, 0.9947731915575051], RGBColor[0.9012009151280851, 0.5175692646620489, 0.9983446734037713], RGBColor[0.7035730534656404, 0.46501229404822697, 0.9990571252937068]}] Out[1]= {RGBColor[0.5659592309624889, 0.9770206495502198, 0.9985863063171145], RGBColor[0.3754327846016599, 0.7732937679156299, 0.9934076415002092], RGBColor[0.6161706939425584, -0.14481564113292222, 1.00353426972292], RGBColor[0.41787305847097933, 0.72956 ... 65862166936707, 1.0001696262621942], RGBColor[0.7321221274856494, 0.7850360738337789, 0.9947731915575051], RGBColor[0.9012009151280851, 0.5175692646620489, 0.9983446734037713], RGBColor[0.7035730534656404, 0.46501229404822697, 0.9990571252937068]} ``` ### Scope (3) Train an ``AnomalyDetectorFunction`` on a two-dimensional array of pseudorandom real numbers: ```wl In[1]:= ad = AnomalyDetection[RandomReal[1, {20, 2}]] Out[1]= AnomalyDetectorFunction[Association["Preprocessor" -> MachineLearning`MLProcessor["ToMLDataset", Association["Input" -> Association["f1" -> Association["Type" -> "NumericalVector", "Length" -> 2]], "Output" -> Association["f1" -> Ass ... e" -> DateObject[{2025, 7, 7, 14, 54, 10.870976`7.788843523378523}, "Instant", "Gregorian", -5.], "ProcessorCount" -> 12, "ProcessorType" -> "ARM64", "OperatingSystem" -> "MacOSX", "SystemWordLength" -> 64, "Evaluations" -> {}]]] ``` Use the trained ``AnomalyDetectorFunction`` with ``DeleteAnomalies`` to drop anomalous examples: ```wl In[2]:= DeleteAnomalies[ad, {{5, 0.6}, {0.3, 0.5}, {0.1, 0.2}, {5, 0.6}}] Out[2]= {{0.3, 0.5}, {0.1, 0.2}} ``` --- Delete anomalies from a ``Tabular`` object: ```wl In[1]:= DeleteAnomalies@Tabular[Association["RawSchema" -> Association["ColumnProperties" -> Association["Value" -> Association["ElementType" -> "NumberExpression"]], "KeyColumns" -> None, "Backend" -> "WolframKernel"], "Options" -> {}, "BackendData" -> Association["ColumnData" -> DataStructure["ColumnTable", {{TabularColumn[Association["Data" -> {{-0.7, -0.3288, -0.515, -0.67, -0.17, 0.4, 100}, {}, None}, "ElementType" -> "NumberExpression"]]}}]]]] Out[1]= Tabular[Association["RawSchema" -> Association["ColumnProperties" -> Association["Value" -> Association["ElementType" -> "NumberExpression"]], "KeyColumns" -> None, "Backend" -> "WolframKernel"], "Options" -> {}, "BackendData" -> Association["ColumnData" -> DataStructure["ColumnTable", {{TabularColumn[Association["Data" -> {{-0.7, -0.3288, -0.515, -0.67, -0.17, 0.4}, {}, None}, "ElementType" -> "NumberExpression"]]}}]]]] ``` --- Obtain a random sample of training and test datasets of images: ```wl In[1]:= SeedRandom[123]; cifarsample = RandomSample[ResourceData["CIFAR-10"], 5000][[All, 1]]; {test, train} = TakeDrop[cifarsample, 100]; RandomSample[train, 10] Out[1]= {[image], [image], [image], [image], [image], [image], [image], [image], [image], [image]} ``` Add anomalous examples to corrupt the datasets: ```wl In[2]:= traincorup = Flatten[Join[Table[RandomImage[1, {12, 12}], 4], train ], 1]; testcorup = Flatten[Join[Table[RandomImage[1, {12, 12}], 4], test], 1] Out[2]= {[image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [ima ... ge], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image]} ``` Train a distribution on images: ```wl In[3]:= ldi = LearnDistribution[traincorup, PerformanceGoal -> "Quality"] Out[3]= [image] ``` Use the trained distribution to drop anomalous examples in the test set: ```wl In[4]:= DeleteAnomalies[ldi, testcorup] Out[4]= {[image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [ima ... ge], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image]} ``` ### Options (2) #### AcceptanceThreshold (1) Specify the ``AcceptanceThreshold`` for dropping anomalies from a list of colors: ```wl In[1]:= DeleteAnomalies[{RGBColor[1, 0, 0], RGBColor[0, 0, 1], RGBColor[0, 1, 0], RGBColor[0.5659592309624889, 0.9770206495502198, 0.9985863063171145], RGBColor[0.3754327846016599, 0.7732937679156299, 0.9934076415002092], RGBColor[0.6161706939425584, -0.14481564113292222, 1.00353426972292], RGBColor[0.41787305847097933, 0.7295653444285688, 0.9971213930223727], RGBColor[0.38756575689146133, 0.8557000434145627, 0.9985713784934372], RGBColor[0.7237746394462758, 0.6864615128156799, 1.0049558953085165], RGBColor[0.4930933634567368, 1.0123748348501285, 0.9922647112874262], RGBColor[0.5083906475863398, 0.23671687553023163, 0.9933475855880759], RGBColor[0.5356308588856586, 0.9101219694870664, 0.9954192657779505], RGBColor[0.35371315176390594, 0.49763789511715706, 0.9991939268536737], RGBColor[0.5774616748672672, 0.43906208545640635, 1.0027712315253168], RGBColor[0.8301864069760181, 0.5690019228798344, 1.0080380129216597], RGBColor[0.7252770979304081, 0.6865831707732285, 0.9933529799272527], RGBColor[0.40911415636021287, 0.7226193316264145, 0.9966957402415179], RGBColor[0.8482079770014748, 0.8501425937052987, 0.9984394751245201], RGBColor[0.46509466164008856, 0.6496001687112135, 0.9943044940805594], RGBColor[0.5223381695947401, 0.9492448201228033, 0.9909519535914891], RGBColor[0.4056317599258231, 0.5979537199100144, 0.9960609147027923], RGBColor[0.5681282152142627, 0.6145506274472882, 0.9942053386481897], RGBColor[0.7340135990764531, 0.7363032159353553, 1.0051356464686494], RGBColor[0.568680351544366, 0.8178935615276761, 0.9972973909156875], RGBColor[0.8782992044353817, 0.5676721429387377, 0.9976822312013324], RGBColor[0.8912340489623893, 0.7418023277818317, 1.0056537818949918], RGBColor[0.5024620333700911, 0.5047283647090592, 0.9983530694802806], RGBColor[0.8078717924205181, 0.653899537453991, 0.9975680282006709], RGBColor[0.5263860266188425, 0.8891481475122383, 0.9967074371571546], RGBColor[0.4509497768400042, 0.9746743907386307, 0.9921412365780125], RGBColor[0.36709487152223774, 0.7030260587378026, 0.990996524067415], RGBColor[0.3486258167177018, 0.8782565717625381, 0.9924313945690186], RGBColor[0.2362783021030173, 1.0366668357638806, 0.9920321150148772], RGBColor[0.5037753263210084, 1.1372490894202187, 0.9970364159364161], RGBColor[0.48515674347257165, 0.623438767322434, 0.9987062896640942], RGBColor[0.309752273104561, 0.5516216087074916, 0.9979006130216704], RGBColor[0.791886226973098, 0.8555278868479104, 0.9964013988720439], RGBColor[0.36489380887741857, 0.6538971286146514, 0.9947083539138546], RGBColor[0.5788067899336176, 0.518500061180151, 1.0059894700291705], RGBColor[0.7591973658560403, 0.32754424645484553, 1.0031595845968302], RGBColor[0.611080929102045, 0.6093006791401225, 0.9968229964712378], RGBColor[0.7071196306427605, 0.6017237761211243, 1.002238791867162], RGBColor[0.607800621747923, 0.688729617245539, 0.9948471916926879], RGBColor[0.5228212118948214, 0.9621391171030302, 0.9989102310042973], RGBColor[0.4034263843628224, 0.6961712245533224, 0.9945334634229808], RGBColor[0.4624897911143086, 1.0117646816528425, 0.9976308623651218], RGBColor[0.22546551277377974, 0.49968908858854955, 0.9892293741848242], RGBColor[0.39610094617200603, 0.6406324106874157, 0.9958241193542212], RGBColor[0.2980675811368619, 0.7367494980194614, 0.9938982243446775], RGBColor[0.366116364693152, 0.5879590062582961, 0.9971279176250676], RGBColor[0.7627350568599146, 0.8758122841547133, 1.0006254489536797], RGBColor[0.5606336793064048, 0.7364374406357912, 0.9976736921570344], RGBColor[0.611217969265626, 0.4461420579128267, 0.9902433094513072], RGBColor[0.7156569387180267, 0.7123170061839005, 0.9962110198471927], RGBColor[0.6791440275858908, 0.9777377725744023, 0.9993352675306073], RGBColor[0.7535427926452951, 0.8140291014876614, 0.9999779030671445], RGBColor[0.4224730449691919, 0.7048172988308568, 1.000218197645998], RGBColor[1.0831053391201486, 0.5162623535933131, 1.0053053485793753], RGBColor[0.5908326301132645, 0.9331899955428621, 0.9993293758777952], RGBColor[0.4084175124285697, 0.5030906121830307, 0.9934012321919623], RGBColor[0.5906929070515831, 0.774539136049385, 1.0013136342783098], RGBColor[0.7279592862294082, 0.808024376838693, 0.990598201263683], RGBColor[0.4526619806424369, 0.8572369865359456, 0.992027772161482], RGBColor[0.6116378507223711, 0.9692792232881027, 1.0007282546056637], RGBColor[0.6728030121309461, 0.6159945503679183, 0.999994265026891], RGBColor[0.6187262817015476, 0.38329666431149445, 0.9983959873247816], RGBColor[0.5537139731404712, 1.00220867269463, 0.9971841114269797], RGBColor[0.5824929564854407, 0.8568652855691417, 0.9978052285631961], RGBColor[0.6467916416926835, 0.6989278893033707, 1.0040647924000072], RGBColor[0.4715405812654946, 0.48471583692401254, 0.9919917090290089], RGBColor[0.7501337350307926, 0.5478514012539901, 1.0001675930894274], RGBColor[0.7935860581270134, 0.0865495328323066, 1.0003613958535735], RGBColor[0.6957752871650936, 0.6750985254752383, 1.0028749839459759], RGBColor[0.5286599401622628, 0.752375592353536, 0.9924914932179304], RGBColor[0.7803166955792776, 0.6640679089392894, 1.0007720191828262], RGBColor[0.8149567117022473, 0.9107449749917345, 1.0018847201660235], RGBColor[0.28687387932890723, 0.5251822879837312, 1.0048453553240377], RGBColor[0.6339138155603266, 0.9499413635111722, 0.9957923550554846], RGBColor[0.6195935090234708, 0.9104921054525388, 0.9993262188604629], RGBColor[0.629437512360706, 0.7836610961087073, 0.9969560811106394], RGBColor[0.9539599691655274, 0.5727469496664113, 1.005124666562766], RGBColor[0.461135511071761, 0.7047561907538, 0.9971199066160322], RGBColor[0.6114235548448876, 1.052922842230812, 1.0000460379208387], RGBColor[0.8353121161086462, 0.4688421901010138, 1.0009262851959584], RGBColor[0.46781661731476154, 1.1396495538311269, 0.9890405978905257], RGBColor[0.8292770313746848, 0.4360701445302748, 1.0009047584693789], RGBColor[0.5170804874340198, 1.064408222280961, 1.0012411854402568], RGBColor[0.7650765408336083, 0.36936107116122646, 0.9945172422937401], RGBColor[0.45119945401989625, 0.856693556198354, 0.9957778134338929], RGBColor[0.39761045159524083, 0.6933289418577382, 0.9976912701802133], RGBColor[0.8159235329521966, 0.4299485471655765, 0.9984939390404028], RGBColor[0.357164361739695, 0.7217436860994018, 0.9966322520929208], RGBColor[0.5610070009904424, 0.8699407233165275, 0.9982375345503843], RGBColor[0.3854123195663661, 0.9861016334439296, 0.9907281406060822], RGBColor[0.6412771660095753, 0.5406870672701729, 1.0025080649577107], RGBColor[0.9426674051744555, 0.8144838170656485, 1.004881312262141], RGBColor[0.443342731164991, 0.8970900892291637, 0.9970488025463211], RGBColor[0.41572227417341934, 0.7900666387545331, 0.9906429085451064], RGBColor[0.6004982532903719, 1.0116685653582027, 0.9998841040044251], RGBColor[0.47986941698642815, 0.7665862166936707, 1.0001696262621942], RGBColor[0.7321221274856494, 0.7850360738337789, 0.9947731915575051], RGBColor[0.9012009151280851, 0.5175692646620489, 0.9983446734037713], RGBColor[0.7035730534656404, 0.46501229404822697, 0.9990571252937068]}, AcceptanceThreshold -> 0.5] Out[1]= {RGBColor[0.5659592309624889, 0.9770206495502198, 0.9985863063171145], RGBColor[0.3754327846016599, 0.7732937679156299, 0.9934076415002092], RGBColor[0.41787305847097933, 0.7295653444285688, 0.9971213930223727], RGBColor[0.38756575689146133, 0.8557 ... 70900892291637, 0.9970488025463211], RGBColor[0.6004982532903719, 1.0116685653582027, 0.9998841040044251], RGBColor[0.47986941698642815, 0.7665862166936707, 1.0001696262621942], RGBColor[0.7321221274856494, 0.7850360738337789, 0.9947731915575051]} ``` #### Method (1) Create a dataset sampled from two different distributions: ```wl In[1]:= data = Join[RandomVariate[NormalDistribution[-1, 0.5], 70], RandomVariate[ChiDistribution[6], 2]] Out[1]= {-1.30421, -0.877751, -0.0275, -0.829094, -0.285438, -1.86924, -0.536187, -1.70286, -1.09622, -1.53435, -0.185815, -0.932228, -0.392102, -0.636749, -0.523112, -0.76753, -1.96098, -0.541443, -1.00106, -0.373463, -0.596879, -0.863837, -2.17911, -0.27 ... -1.64516, -1.02551, -1.72144, 0.0137513, -0.940919, -1.09621, -2.02574, -1.27502, -0.313154, -1.31769, -0.212728, -1.43116, -0.548494, -0.603053, -1.5866, -1.24111, -0.543165, -0.0926575, -1.11422, -1.49748, -0.669169, -1.25927, 2.85186, 1.84458} ``` Delete anomalies in the dataset using the ``"Multinormal"`` method: ```wl In[2]:= DeleteAnomalies[data, Method -> "Multinormal"] Out[2]= {-1.30421, -0.877751, -0.0275, -0.829094, -0.285438, -1.86924, -0.536187, -1.70286, -1.09622, -1.53435, -0.185815, -0.932228, -0.392102, -0.636749, -0.523112, -0.76753, -1.96098, -0.541443, -1.00106, -0.373463, -0.596879, -0.863837, -2.17911, -0.27 ... 1.07768, -1.22941, -1.64516, -1.02551, -1.72144, 0.0137513, -0.940919, -1.09621, -2.02574, -1.27502, -0.313154, -1.31769, -0.212728, -1.43116, -0.548494, -0.603053, -1.5866, -1.24111, -0.543165, -0.0926575, -1.11422, -1.49748, -0.669169, -1.25927} ``` Delete anomalies in the dataset using the ``"KernelDensityEstimation"`` method: ```wl In[3]:= DeleteAnomalies[data, Method -> "KernelDensityEstimation"] Out[3]= {-1.30421, -0.877751, -0.0275, -0.829094, -0.285438, -1.86924, -0.536187, -1.70286, -1.09622, -1.53435, -0.185815, -0.932228, -0.392102, -0.636749, -0.523112, -0.76753, -1.96098, -0.541443, -1.00106, -0.373463, -0.596879, -0.863837, -2.17911, -0.27 ... 0.594006, -1.07768, -1.22941, -1.64516, -1.02551, -1.72144, -0.940919, -1.09621, -2.02574, -1.27502, -0.313154, -1.31769, -0.212728, -1.43116, -0.548494, -0.603053, -1.5866, -1.24111, -0.543165, -0.0926575, -1.11422, -1.49748, -0.669169, -1.25927} ``` ### Applications (4) Find the statistical mean of numeric values with an anomaly: ```wl In[1]:= Mean[{1.2, 2.2, 3.4, 4.1, 200, 5.7, 6, 7, 8, 8.3}] Out[1]= 24.59 ``` Delete anomalies before computing the mean: ```wl In[2]:= Mean[DeleteAnomalies[{1.2, 2.2, 3.4, 4.1, 200, 5.7, 6, 7, 8, 8.3}]] Out[2]= 5.1 ``` --- Plot a list of numeric values that contains anomalies: ```wl In[1]:= data = {2, 1, 3, 100, 2, 1, 2, 1000, 6, 2, 5, 8, 15}; ListLinePlot[data, PlotRange -> All] Out[1]= [image] ``` Plot a list of numeric values after removing anomalies: ```wl In[2]:= ListLinePlot[DeleteAnomalies[data]] Out[2]= [image] ``` --- Find the linear fit for numerical data: ```wl In[1]:= data = {{0, 1}, {1, 0}, {3, 2}, {5, 4}, {6, 7}, {200, 8}}; In[2]:= LinearModelFit[data, x, x] Out[2]= FittedModel[Association["Type" -> "Linear", "Model" -> Association["FittedParameters" -> {2.6927873697869735, 0.02717802688966586}, "IndependentVariables" -> {x}, "BasisFunctions" -> {1, x}, "LinearOffset" -> Association["Function" -> ... }, {6, 7}, {200, 8}}, "UserDefinedDesignMatrixQ" -> False, "DesignMatrix" -> {{1., 0.}, {1., 1.}, {1., 3.}, {1., 5.}, {1., 6.}, {1., 200.}}, "Localizer" -> Function[Null, Internal`LocalizedBlock[{x}, #1], {HoldAll}], "Options" -> {}]] ``` Delete outliers before modeling the fit: ```wl In[3]:= LinearModelFit[DeleteAnomalies[data], x, x] Out[3]= FittedModel[Association["Type" -> "Linear", "Model" -> Association["FittedParameters" -> {-0.2, 1.}, "IndependentVariables" -> {x}, "BasisFunctions" -> {1, x}, "LinearOffset" -> Association["Function" -> 0, "Values" -> 0]], "Weights" -> ... {0, 1}, {1, 0}, {3, 2}, {5, 4}, {6, 7}}, "UserDefinedDesignMatrixQ" -> False, "DesignMatrix" -> {{1., 0.}, {1., 1.}, {1., 3.}, {1., 5.}, {1., 6.}}, "Localizer" -> Function[Null, Internal`LocalizedBlock[{x}, #1], {HoldAll}], "Options" -> {}]] ``` --- Obtain a dataset of images: ```wl In[1]:= train = RandomSample[ResourceData["MNIST", "TrainingData"][[All, 1]], 30000]; In[2]:= RandomSample[train, 10] Out[2]= {[image], [image], [image], [image], [image], [image], [image], [image], [image], [image]} ``` Train an anomaly detector on the training set: ```wl In[3]:= ad = AnomalyDetection[train, Method -> "Multinormal"] Out[3]= AnomalyDetectorFunction[«1»] ``` Drop anomalous examples in the test set: ```wl In[4]:= DeleteAnomalies[ad, {[image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image]}] Out[4]= {[image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image], [image]} ``` ## See Also * [`FindAnomalies`](https://reference.wolfram.com/language/ref/FindAnomalies.en.md) * [`FindClusters`](https://reference.wolfram.com/language/ref/FindClusters.en.md) * [`AnomalyDetection`](https://reference.wolfram.com/language/ref/AnomalyDetection.en.md) * [`AnomalyDetectorFunction`](https://reference.wolfram.com/language/ref/AnomalyDetectorFunction.en.md) * [`LearnDistribution`](https://reference.wolfram.com/language/ref/LearnDistribution.en.md) * [`RarerProbability`](https://reference.wolfram.com/language/ref/RarerProbability.en.md) * [`TransformAnomalies`](https://reference.wolfram.com/language/ref/TransformAnomalies.en.md) * [`Classify`](https://reference.wolfram.com/language/ref/Classify.en.md) * [`DimensionReduction`](https://reference.wolfram.com/language/ref/DimensionReduction.en.md) * [`DeleteMissing`](https://reference.wolfram.com/language/ref/DeleteMissing.en.md) * [`DeleteCases`](https://reference.wolfram.com/language/ref/DeleteCases.en.md) * [`PlotRange`](https://reference.wolfram.com/language/ref/PlotRange.en.md) * [`RemoveBackground`](https://reference.wolfram.com/language/ref/RemoveBackground.en.md) ## Related Guides * [Data Transforms and Smoothing](https://reference.wolfram.com/language/guide/DataTransformsAndSmoothing.en.md) * [Unsupervised Machine Learning](https://reference.wolfram.com/language/guide/UnsupervisedMachineLearning.en.md) * [Tabular Data Cleaning](https://reference.wolfram.com/language/guide/TabularCleaning.en.md) ## Related Links * [An Elementary Introduction to the Wolfram Language: Machine Learning](https://www.wolfram.com/language/elementary-introduction/22-machine-learning.html) ## History * [Introduced in 2019 (12.0)](https://reference.wolfram.com/language/guide/SummaryOfNewFeaturesIn120.en.md) \| [Updated in 2020 (12.1)](https://reference.wolfram.com/language/guide/SummaryOfNewFeaturesIn121.en.md) ▪ [2025 (14.2)](https://reference.wolfram.com/language/guide/SummaryOfNewFeaturesIn142.en.md)