anomaly-1.1.0/0000755000175000017500000000000012214651465012441 5ustar mogaalmogaalanomaly-1.1.0/AUTHORS0000644000175000017500000000061012214651444013503 0ustar mogaalmogaalThe development of anomaly was made possible by the significant contributions of the following people: Paul Beckingham (Principal Author) Federico Hernandez (Package Maintainer & Contributing Author) The following submitted code, packages or analysis, and deserve special thanks: ... Thanks to the following, who submitted detailed bug reports and excellent suggestions: ... anomaly-1.1.0/LICENSE0000644000175000017500000000217712214651444013452 0ustar mogaalmogaalanomaly - Anomalous data detection Copyright 2013, Göteborg Bit Factory. Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. http://www.opensource.org/licenses/mit-license.php anomaly-1.1.0/COPYING0000644000175000017500000000217412214651444013475 0ustar mogaalmogaalanomaly - data anomaly detector Copyright 2013, Göteborg Bit Factory. Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. http://www.opensource.org/licenses/mit-license.php anomaly-1.1.0/doc/0000755000175000017500000000000012214651444013203 5ustar mogaalmogaalanomaly-1.1.0/doc/CMakeLists.txt0000644000175000017500000000057312214651444015750 0ustar mogaalmogaalcmake_minimum_required (VERSION 2.8) message ("-- Configuring man pages") set (man_FILES anomaly.1) foreach (man_FILE ${man_FILES}) configure_file ( man/${man_FILE}.in man/${man_FILE}) endforeach (man_FILE) install (DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}/man/ DESTINATION share/man/man1 FILES_MATCHING PATTERN "*.1") anomaly-1.1.0/doc/man/0000755000175000017500000000000012214651444013756 5ustar mogaalmogaalanomaly-1.1.0/doc/man/.gitignore0000644000175000017500000000001212214651444015737 0ustar mogaalmogaalanomaly.1 anomaly-1.1.0/doc/man/anomaly.1.in0000644000175000017500000001207112214651444016106 0ustar mogaalmogaal.TH anomaly 1 2013-05-05 "${PACKAGE_STRING}" "User Manuals" .SH NAME anomaly \- anomalous data detection .SH SYNOPSIS .B anomaly [-h|--help] [-v|--version] [-d|--details] [-t|--threshold] [--min N] [--max N] [-s|--stddev] [-n|--sample N] [-c|--coefficient N] [-q|--quiet] [-e|--execute PROGRAM] [-p|--pid PID] .SH DESCRIPTION Anomaly can detect anomalous data in a numeric stream. In order to do this, anomaly needs to see a stream of numeric data, and apply one of its detection methods. If an anomaly is detected, a response is made, chosen from one or more built in methods. .SH NUMERIC STREAM Anomaly works best in a pipe, and will read only numeric data from its input. As a simple example, suppose you wish to monitor load average and look for unusual spikes. The load average can be obtained from the 'uptime' command: .RS $ uptime .br 11:40 up 15 days, 4:04, 6 users, load averages: 0.38 0.32 0.32 .RE We can extract the 5-minute load (the second of the three numbers) using this: .RS $ uptime | cut -f 13 -d ' ' .br 0.29 .RE That number can be extracted once a minute, using this: .RS $ while [ 1 ]; do uptime | cut -f 13 -d ' '; sleep 60; done .br 0.29 .br 0.26 .br 0.19 .RE That is the kind of data stream that anomaly monitors. White space (spaces, tabs, newlines) between the numbers are ignored, so we can simulate the above stream like this: .RS $ echo 0.29 0.26 0.19 .RE This is a convenient way to demonstrate anomaly, shown below. .SH DETECTION - THRESHOLD The simplest detection method is threshold, which compares the data to an absolute value. This method can use a minimum and a maximum value for comparison. These alternatives are all valid, and make use of --min, --max or both: .RS anomaly --threshold --min 1.22 --max 9.75 .br anomaly --threshold --min 1.22 .br anomaly --threshold --max 9.75 .RE In the following example, the values '1' and '10' would be detected as anomalies: .RS $ echo 2 1 3 6 10 5 | anomaly --threshold --min 1.5 --max 8 .br Anomalous data detected. The value 1 is below the minimum of 1.5. .br Anomalous data detected. The value 10 is above the maximum of 8. .RE .SH DETECTION - STANDARD DEVIATION Standard deviation measures differences from the mean value of a sample of data, and is useful for detecting extraordinary values. The sample size can be chosen such that there is enough data to determine a good mean value, but defaults to 10. The limited sample size means that a rolling window of data is used, and therefore the mean and standard deviation is updated for the current window. This makes the monitoring somewhat adaptive. Here is an example: .RS anomaly --stddev --sample 20 .RE This uses a sample size of the 20 most recent values, and will detect any values that are +/- 1 standard deviation from the mean. An example: .RS $ echo 1 2 3 4 5 6 | anomaly --stddev --sample 5 .br Anomalous data detected. The value 6 is more than 1 sigma(s) above the mean value 3, with a sample size of 5. .RE With a sample size of 5, comparisons being only after the 6th value is seen. In the example, the mean value of [1 2 3 4 5] is 3, and the standard deviation is 1.58. This means that the 6th value is considered an anomaly if it is within the range (3 +/- 1.58), which is between 1.42 and 4.58. To make this less sensitive, a coefficient is introduced, which defaults to 1.0 (as above) but can be overridden: .RS $ echo 1 2 3 4 5 6 | anomaly --stddev --sample 5 --coefficient 1.9 .br $ .RE In this example, the 6th value is not considered an anomaly because it is within the range (3 +/- (1.9 * 1.58)), which is between -0.002 and 6.002. .SH RESPONSE - MESSAGE The message response is the default, and consists of a single line of printed text. It is a description of why the data value is considered an anomaly. Here is an example: .RS $ echo 1 2 3 | anomaly --threshold --max 2.5 .br Anomalous data detected. The value 3 is above the maximum of 2.5. .RE The message can be suppressed, but another response must be specified, so that there is some kind of response: .RS $ echo 1 2 3 | anomaly --threshold --max 2.5 --quiet ... .RE .SH RESPONSE - EXECUTE Anomaly can execute a program in response to detection. Here an example uses the 'date' command, but any program can be used: .RS $ echo 1 2 3 | anomaly --threshold --max 2.5 --quiet --execute '/bin/date +%s' .br 1361727327 .RE .SH RESPONSE - SIGNAL Anomaly can send a USR1 signal to a program in response to detection: .RS $ echo 1 2 3 | anomaly --threshold --max 2.5 --quiet --pid 12345 .RE This sends the USR1 signal to the process with PID 12345. The receiving program would need to respond accordingly. .SH "CREDITS & COPYRIGHTS" Copyright (C) 2013 Göteborg Bit Factory. Anomaly is distributed under the MIT license. See http://www.opensource.org/licenses/mit-license.php for more information. .SH SEE ALSO For more information, see: .TP The official site at .TP You can contact the project by writing an email to .SH REPORTING BUGS .TP Bugs in anomaly may be reported to anomaly-1.1.0/.gitignore0000644000175000017500000000012212214651444014421 0ustar mogaalmogaalCMakeFiles CMakeCache.txt *.cmake Makefile install_manifest.txt cmake.h commit.h anomaly-1.1.0/test/0000755000175000017500000000000012214651444013415 5ustar mogaalmogaalanomaly-1.1.0/test/errors.t0000755000175000017500000000453412214651444015127 0ustar mogaalmogaal#! /usr/bin/env perl ################################################################################ ## anomaly - Anomalous data detection ## ## Copyright 2013, Göteborg Bit Factory. ## ## Permission is hereby granted, free of charge, to any person obtaining a copy ## of this software and associated documentation files (the "Software"), to deal ## in the Software without restriction, including without limitation the rights ## to use, copy, modify, merge, publish, distribute, sublicense, and/or sell ## copies of the Software, and to permit persons to whom the Software is ## furnished to do so, subject to the following conditions: ## ## The above copyright notice and this permission notice shall be included ## in all copies or substantial portions of the Software. ## ## THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS ## OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, ## FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL ## THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER ## LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, ## OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE ## SOFTWARE. ## ## http://www.opensource.org/licenses/mit-license.php ## ################################################################################ use strict; use warnings; use Test::More tests => 9; my $output = qx{../src/anomaly --foo}; like ($output, qr/Unrecognized/, 'Unrecognized arg -foo'); $output = qx{../src/anomaly --threshold --max 3.1 --min 3.2}; like ($output, qr/must be min/, '--min > --higher'); $output = qx{../src/anomaly --threshold --min 3.2 --max 3.1}; like ($output, qr/must be higher/, '--min > --higher'); $output = qx{../src/anomaly --stddev --sample 1}; like ($output, qr/Sample size/, '--sample < 2'); $output = qx{../src/anomaly --execute ''}; like ($output, qr/non-trivial path/, '--execute \'\''); $output = qx{../src/anomaly --pid 0}; like ($output, qr/non-zero/, '--pid 0'); $output = qx{../src/anomaly --threshold}; like ($output, qr/or both/, '--threshold, no bounds'); $output = qx{../src/anomaly --quiet}; like ($output, qr/reaction must be/, 'no reaction'); $output = qx{../src/anomaly --coefficient 0.0000001}; like ($output, qr/non-trivial coefficient/, '--coefficient 0.0000001'); exit 0; anomaly-1.1.0/test/.gitignore0000644000175000017500000000001112214651444015375 0ustar mogaalmogaalall.log anomaly-1.1.0/test/threshold.t0000755000175000017500000000505612214651444015607 0ustar mogaalmogaal#! /usr/bin/env perl ################################################################################ ## anomaly - Anomalous data detection ## ## Copyright 2013, Göteborg Bit Factory. ## ## Permission is hereby granted, free of charge, to any person obtaining a copy ## of this software and associated documentation files (the "Software"), to deal ## in the Software without restriction, including without limitation the rights ## to use, copy, modify, merge, publish, distribute, sublicense, and/or sell ## copies of the Software, and to permit persons to whom the Software is ## furnished to do so, subject to the following conditions: ## ## The above copyright notice and this permission notice shall be included ## in all copies or substantial portions of the Software. ## ## THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS ## OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, ## FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL ## THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER ## LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, ## OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE ## SOFTWARE. ## ## http://www.opensource.org/licenses/mit-license.php ## ################################################################################ use strict; use warnings; use Test::More tests => 8; # anomaly --threshold --max 3.14 my $output = qx{echo '1' | ../src/anomaly --threshold --max 3.14}; is ($output, '', 'threshold (max 3.14) 1 -->'); $output = qx{echo '1 2' | ../src/anomaly --threshold --max 3.14}; is ($output, '', 'threshold (max 3.14) 1 2 -->'); $output = qx{echo '1 2 3' | ../src/anomaly --threshold --max 3.14}; is ($output, '', 'threshold (max 3.14) 1 2 3 -->'); $output = qx{echo '1 2 3 4' | ../src/anomaly --threshold --max 3.14}; like ($output, qr/Anomalous data detected/, 'threshold (max 3.14) 1 2 3 4 --> Anomaly'); # anomaly --threshold --min 3.14 $output = qx{echo '4' | ../src/anomaly --threshold --min 3.14}; is ($output, '', 'threshold (min 3.14) 4 -->'); $output = qx{echo '4 3' | ../src/anomaly --threshold --min 3.14}; like ($output, qr/Anomalous data detected/, 'threshold (min 3.14) 4 3 --> Anomaly'); # Test '--execute' $output = qx{echo '1' | ../src/anomaly --threshold --max 2 --execute 'echo hello' --quiet}; is ($output, '', 'threshold --> no execute'); $output = qx{echo '3' | ../src/anomaly --threshold --max 2 --execute 'echo hello' --quiet}; is ($output, "hello\n", 'threshold --> execute'); exit 0; anomaly-1.1.0/test/run_all0000755000175000017500000000211712214651444015000 0ustar mogaalmogaal#! /bin/sh if [ x"$1" = x"--verbose" ]; then for i in ./*.t do echo '#' $i $i > test.log 2>&1 while read LINE do echo $LINE done < test.log rm test.log done else date > all.log # Perl is used here to get the time in seconds # because 'date +%s' isn't supported on Solaris. STARTEPOCH=`perl -e 'print time'` VRAMSTEG=`which vramsteg` BAR=0 if [ -x "$VRAMSTEG" ]; then BAR=1 COUNT=0 TOTAL=`ls ./*.t | wc -l` START=`$VRAMSTEG --now` fi for i in ./*.t do echo '#' $i >>all.log if [ $BAR -eq 1 ]; then $VRAMSTEG --label 'All tests' --min 0 --max $TOTAL --current $COUNT --percentage --start $START --estimate COUNT=`expr $COUNT + 1` fi $i >> all.log 2>&1 done if [ $BAR -eq 1 ]; then $VRAMSTEG --remove fi date >> all.log ENDEPOCH=`perl -e 'print time'` RUNTIME=`expr $ENDEPOCH - $STARTEPOCH` printf "Pass: %5d\n" `grep -c '^ok' all.log` printf "Fail: %5d\n" `grep -c '^not' all.log` printf "Skipped: %5d\n" `grep -c '^skip' all.log` printf "Runtime: %5d seconds\n" $RUNTIME fi anomaly-1.1.0/test/version.t0000755000175000017500000000312712214651444015275 0ustar mogaalmogaal#! /usr/bin/env perl ################################################################################ ## anomaly - Anomalous data detection ## ## Copyright 2013, Göteborg Bit Factory. ## ## Permission is hereby granted, free of charge, to any person obtaining a copy ## of this software and associated documentation files (the "Software"), to deal ## in the Software without restriction, including without limitation the rights ## to use, copy, modify, merge, publish, distribute, sublicense, and/or sell ## copies of the Software, and to permit persons to whom the Software is ## furnished to do so, subject to the following conditions: ## ## The above copyright notice and this permission notice shall be included ## in all copies or substantial portions of the Software. ## ## THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS ## OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, ## FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL ## THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER ## LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, ## OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE ## SOFTWARE. ## ## http://www.opensource.org/licenses/mit-license.php ## ################################################################################ use strict; use warnings; use Test::More tests => 1; my $year = (localtime (time))[5] + 1900; my $output = qx{../src/anomaly --version 2>&1}; like ($output, qr/Copyright \(C\) $year/, 'Copyright is current'); exit 0; anomaly-1.1.0/test/stddev.t0000755000175000017500000000465012214651444015103 0ustar mogaalmogaal#! /usr/bin/env perl ################################################################################ ## anomaly - Anomalous data detection ## ## Copyright 2013, Göteborg Bit Factory. ## ## Permission is hereby granted, free of charge, to any person obtaining a copy ## of this software and associated documentation files (the "Software"), to deal ## in the Software without restriction, including without limitation the rights ## to use, copy, modify, merge, publish, distribute, sublicense, and/or sell ## copies of the Software, and to permit persons to whom the Software is ## furnished to do so, subject to the following conditions: ## ## The above copyright notice and this permission notice shall be included ## in all copies or substantial portions of the Software. ## ## THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS ## OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, ## FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL ## THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER ## LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, ## OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE ## SOFTWARE. ## ## http://www.opensource.org/licenses/mit-license.php ## ################################################################################ use strict; use warnings; use Test::More tests => 6; # anomaly --stddev --sample 5, 1 2 3 4 5 6 -> Anomaly # The sequence 1 2 3 4 5 has mean 3, sigma 1.58. my $output = qx{echo '1' | ../src/anomaly --stddev --sample 5 --coefficient 1.0}; is ($output, '', 'stddev (sample 5 coeff 1.0) 1 -->'); $output = qx{echo '1 2' | ../src/anomaly --stddev --sample 5 --coefficient 1.0}; is ($output, '', 'stddev (sample 5 coeff 1.0) 1 2 -->'); $output = qx{echo '1 2 3' | ../src/anomaly --stddev --sample 5 --coefficient 1.0}; is ($output, '', 'stddev (sample 5 coeff 1.0) 1 2 3 -->'); $output = qx{echo '1 2 3 4' | ../src/anomaly --stddev --sample 5 --coefficient 1.0}; is ($output, '', 'stddev (sample 5 coeff 1.0) 1 2 3 4 -->'); $output = qx{echo '1 2 3 4 5' | ../src/anomaly --stddev --sample 5 --coefficient 1.0}; is ($output, '', 'stddev (sample 5 coeff 1.0) 1 2 3 4 5 -->'); $output = qx{echo '1 2 3 4 5 6' | ../src/anomaly --stddev --sample 5 --coefficient 1.0}; like ($output, qr/Anomalous data detected/, 'stddev (sample 5 coeff 1.0) 1 2 3 4 5 6 --> Anomaly'); exit 0; anomaly-1.1.0/test/problems0000755000175000017500000000056712214651444015176 0ustar mogaalmogaal#!/usr/bin/env perl use strict; use warnings; if (open my $fh, '<', 'all.log') { my $test_file; my %errors; while (my $line = <$fh>) { $test_file = $1 if $line =~ /^# (\S+\.t)$/; $errors{$test_file}++ if $line =~ /^not /; } close $fh; printf "%-24s %4d\n", $_, $errors{$_} for sort {$errors{$b} <=> $errors{$a}} keys %errors; } exit 0; anomaly-1.1.0/test/CMakeLists.txt0000644000175000017500000000037212214651444016157 0ustar mogaalmogaalcmake_minimum_required (VERSION 2.8) include_directories (${CMAKE_SOURCE_DIR} ${CMAKE_SOURCE_DIR}/src ${CMAKE_SOURCE_DIR}/test) add_custom_target (test ./run_all WORKING_DIRECTORY ${CMAKE_SOURCE_DIR}/test) anomaly-1.1.0/NEWS0000644000175000017500000000061512214651444013137 0ustar mogaalmogaal New Features in anomaly 1.1.0 - Please refer to the ChangeLog file for full details. --- Anomaly has been built and tested on the following configurations: * OS X 10.8 Mountain Lion * Ubuntu 10.04 Natty Narwhal * Cygwin 1.7 --- While anomaly has undergone testing, bugs are sure to remain. If you encounter a bug, please send a message to: support@tasktools.org Thank you. anomaly-1.1.0/src/0000755000175000017500000000000012214651444013225 5ustar mogaalmogaalanomaly-1.1.0/src/.gitignore0000644000175000017500000000001012214651444015204 0ustar mogaalmogaalanomaly anomaly-1.1.0/src/anomaly.cpp0000644000175000017500000001333712214651444015400 0ustar mogaalmogaal//////////////////////////////////////////////////////////////////////////////// // anomaly - Anomalous data detection // // Copyright 2013, Göteborg Bit Factory. // // Permission is hereby granted, free of charge, to any person obtaining a copy // of this software and associated documentation files (the "Software"), to deal // in the Software without restriction, including without limitation the rights // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell // copies of the Software, and to permit persons to whom the Software is // furnished to do so, subject to the following conditions: // // The above copyright notice and this permission notice shall be included // in all copies or substantial portions of the Software. // // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS // OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL // THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE // SOFTWARE. // // http://www.opensource.org/licenses/mit-license.php // //////////////////////////////////////////////////////////////////////////////// #include #include #include #include #include #include //////////////////////////////////////////////////////////////////////////////// static int usage () { std::cout << "\n" << "Usage: anomaly [options]\n" << "\n" << "Options:\n" << " -h|--help Show this message\n" << " -v|--version Show anomaly version & copyright\n" << " -d|--details Show internal details\n" << "\n" << "Detection: Threshold\n" << " -t|--threshold Threshold algorithm\n" << " --min N Lower threshold\n" << " --max N Upper threshold\n" << "\n" << "Detection: Standard Deviation\n" << " -s|--stddev Standard Deviation algorithm (default)\n" << " -n|--sample N Sample size (default: 10)\n" << " -c|--coefficient N Coefficient for --stddev (default: 1.0)\n" << "\n" << "Responses:\n" << " -q|--quiet Suppresses anomaly description output\n" << " -e|--execute PROGRAM Runs program\n" << " -p|--pid PID Sends USR1 signal to pid\n" << "\n"; return 0; } //////////////////////////////////////////////////////////////////////////////// static int version () { std::cout << "\n" << PACKAGE_STRING #if defined (HAVE_COMMIT) << "." << COMMIT #endif << " built for " #if defined (DARWIN) << "darwin" #elif defined (SOLARIS) << "solaris" #elif defined (CYGWIN) << "cygwin" #elif defined (OPENBSD) << "openbsd" #elif defined (HAIKU) << "haiku" #elif defined (NETBSD) << "netbsd" #elif defined (FREEBSD) << "freebsd" #elif defined (LINUX) << "linux" #else << "unknown" #endif << "\n" << "Copyright (C) 2013 Göteborg Bit Factory\n" << "\n" << "Anomaly may be copied only under the terms of the MIT " "license, which may be found in the source kit.\n" << "\n" << "Documentation for anomaly can be found using 'man anomaly' " "or at http://tasktools.org/projects/anomaly.html\n" << "\n"; return 0; } //////////////////////////////////////////////////////////////////////////////// int main (int argc, char** argv) { try { Detector detector; // Process command line arguments. for (int i = 1; i < argc; ++i) { if (!strcmp (argv[i], "-h") || !strcmp (argv[i], "--help")) return usage (); else if (!strcmp (argv[i], "-v") || !strcmp (argv[i], "--version")) return version (); else if (!strcmp (argv[i], "-d") || !strcmp (argv[i], "--details")) detector.debug (); else if (!strcmp (argv[i], "-t") || !strcmp (argv[i], "--threshold")) detector.algorithm ("threshold"); else if (!strcmp (argv[i], "--max")) detector.max (strtod (argv[++i], NULL)); else if (!strcmp (argv[i], "--min")) detector.min (strtod (argv[++i], NULL)); else if (!strcmp (argv[i], "-s") || !strcmp (argv[i], "--stddev")) detector.algorithm ("stddev"); else if (!strcmp (argv[i], "-n") || !strcmp (argv[i], "--sample")) detector.sample (strtol (argv[++i], NULL, 10)); else if (!strcmp (argv[i], "-c") || !strcmp (argv[i], "--coefficient")) detector.coefficient (strtod (argv[++i], NULL)); else if (!strcmp (argv[i], "-q") || !strcmp (argv[i], "--quiet")) detector.quiet (); else if (!strcmp (argv[i], "-p") || !strcmp (argv[i], "--pid")) detector.pid (strtol (argv[++i], NULL, 10)); else if (!strcmp (argv[i], "-e") || !strcmp (argv[i], "--execute")) detector.execute (argv[++i]); else throw std::string ("Unrecognized argument '") + argv[i] + "'."; } // Dispatch to selected algorithm. detector.run (); } catch (const std::string& error) { std::cout << error << "\n"; return -1; } catch (...) { std::cout << "Unknown error\n"; return -2; } return 0; } //////////////////////////////////////////////////////////////////////////////// anomaly-1.1.0/src/Detector.cpp0000644000175000017500000002225312214651444015506 0ustar mogaalmogaal//////////////////////////////////////////////////////////////////////////////// // Detector - Anomalous data detection // // Copyright 2013, Göteborg Bit Factory. // // Permission is hereby granted, free of charge, to any person obtaining a copy // of this software and associated documentation files (the "Software"), to deal // in the Software without restriction, including without limitation the rights // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell // copies of the Software, and to permit persons to whom the Software is // furnished to do so, subject to the following conditions: // // The above copyright notice and this permission notice shall be included // in all copies or substantial portions of the Software. // // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS // OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL // THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE // SOFTWARE. // // http://www.opensource.org/licenses/mit-license.php // //////////////////////////////////////////////////////////////////////////////// #include #include #include #include #include #include #include //////////////////////////////////////////////////////////////////////////////// Detector::Detector () : _algorithm ("stddev") , _use_max (false) , _use_min (false) , _max (0.0) , _min (0.0) , _sample (10) , _coefficient (1.0) , _quiet (false) , _script ("") , _pid (0) , _debug (false) , _counter (0) { } //////////////////////////////////////////////////////////////////////////////// void Detector::algorithm (const std::string& value) { if (value != "stddev" && value != "threshold") throw std::string ("Unrecognized algorithm '") + value + "' specified."; _algorithm = value; } //////////////////////////////////////////////////////////////////////////////// void Detector::max (double value) { if (_use_min && value < _min) throw std::string ("The max value must be higher than the min value."); _use_max = true; _max = value; } //////////////////////////////////////////////////////////////////////////////// void Detector::min (double value) { if (_use_max && value > _max) throw std::string ("The min value must be min than the max value."); _use_min = true; _min = value; } //////////////////////////////////////////////////////////////////////////////// void Detector::sample (int value) { if (value < 2) throw std::string ("Sample size must be greater than 1."); _sample = value; } //////////////////////////////////////////////////////////////////////////////// void Detector::coefficient (double value) { _coefficient = value; } //////////////////////////////////////////////////////////////////////////////// void Detector::quiet () { _quiet = true; } //////////////////////////////////////////////////////////////////////////////// void Detector::execute (const std::string& value) { if (value == "") throw std::string ("A non-trivial path must be specified."); _script = value; } //////////////////////////////////////////////////////////////////////////////// void Detector::pid (int value) { if (value == 0) throw std::string ("A non-zero PID must be specified."); _pid = value; } //////////////////////////////////////////////////////////////////////////////// void Detector::debug () { _debug = true; } //////////////////////////////////////////////////////////////////////////////// void Detector::run () { if (_algorithm == "threshold") run_threshold (); else if (_algorithm == "stddev") run_stddev (); } //////////////////////////////////////////////////////////////////////////////// void Detector::run_threshold () { // Make sure settings are acceptable. if (!_use_max && !_use_min) throw std::string ("A min, and max, or both threshold values should be " "specified."); if (_quiet && _pid == 0 && _script == "") throw std::string ("A reaction must be specified."); double input; while (std::cin >> input) { ++_counter; if (_debug) { std::cout << "[" << _counter << "] "; if (_use_min) std::cout << "min " << _min << " <= "; std::cout << "value " << input; if (_use_max) std::cout << " <= max " << _max; if (_use_min && input < _min) std::cout << " = Anomaly\n"; else if (_use_max && input > _max) std::cout << " = Anomaly\n"; else std::cout << " = Nominal\n"; } if (_use_max && input > _max) { std::stringstream s; s << "Anomalous data detected. The value " << input << " is above the maximum of " << _max << "."; react (s.str ()); } else if (_use_min && input < _min) { std::stringstream s; s << "Anomalous data detected. The value " << input << " is below the minimum of " << _min << "."; react (s.str ()); } } } //////////////////////////////////////////////////////////////////////////////// void Detector::run_stddev () { // Make sure settings are acceptable. if (_sample < 2) throw std::string ("A sample size of 2 or more must be specified."); if (_coefficient < 1e-6) throw std::string ("A non-trivial coefficient should be specified."); if (_quiet && _pid == 0 && _script == "") throw std::string ("A reaction must be specified."); std::deque data; double input; while (std::cin >> input) { ++_counter; if (data.size () >= _sample) { // Calculate mean, standard deviation. double sum = 0.0; double sum_squares = 0.0; std::deque ::iterator i; for (i = data.begin (); i != data.end (); ++i) { sum += *i; sum_squares += (*i) * (*i); } double sigma = sqrt (((_sample * sum_squares) - (sum * sum)) / (_sample * (_sample - 1))); double mean = sum / _sample; if (input < (mean - (_coefficient * sigma))) { if (_debug) std::cout << "[" << _counter << "] " << "mean " << mean << ", sigma " << sigma << ", coeff " << _coefficient << ", value " << input << " < " << (mean - (_coefficient * sigma)) << " = Anomaly\n"; std::stringstream s; s << "Anomalous data detected. The value " << input << " is more than " << _coefficient << " sigma(s) below the mean value " << mean << ", with a sample size of " << _sample << "."; react (s.str ()); } else if (input > (mean + (_coefficient * sigma))) { if (_debug) std::cout << "[" << _counter << "] " << "mean " << mean << ", sigma " << sigma << ", coeff " << _coefficient << ", value " << input << " > " << (mean + (_coefficient * sigma)) << " = Anomaly\n"; std::stringstream s; s << "Anomalous data detected. The value " << input << " is more than " << _coefficient << " sigma(s) above the mean value " << mean << ", with a sample size of " << _sample << "."; react (s.str ()); } else { if (_debug) std::cout << "[" << _counter << "] " << "mean " << mean << ", sigma " << sigma << ", coeff " << _coefficient << ", " << (mean - (_coefficient * sigma)) << " <= value " << input << " <= " << (mean + (_coefficient * sigma)) << " = Nominal\n"; } } else { if (_debug) std::cout << "[" << _counter << "] " << "value " << input << ", insufficient data - need " << _sample << " items\n"; } data.push_back (input); if (data.size () > _sample) data.pop_front (); } } //////////////////////////////////////////////////////////////////////////////// void Detector::react (const std::string& message) { react_complain (message); react_sigusr1 (); react_execute (); } //////////////////////////////////////////////////////////////////////////////// void Detector::react_complain (const std::string& message) { if (!_quiet) std::cout << message << "\n"; } //////////////////////////////////////////////////////////////////////////////// void Detector::react_execute () { if (_pid) kill (_pid, SIGUSR1); } //////////////////////////////////////////////////////////////////////////////// void Detector::react_sigusr1 () { if (_script != "") system (_script.c_str ()); } //////////////////////////////////////////////////////////////////////////////// anomaly-1.1.0/src/CMakeLists.txt0000644000175000017500000000044712214651444015772 0ustar mogaalmogaalcmake_minimum_required(VERSION 2.8) include_directories (${CMAKE_SOURCE_DIR}/src ${CMAKE_SOURCE_DIR} ${ANOMALY_INCLUDE_DIRS}) add_executable(anomaly anomaly.cpp Detector.cpp Detector.h) install (TARGETS anomaly DESTINATION bin) anomaly-1.1.0/src/Detector.h0000644000175000017500000000437112214651444015154 0ustar mogaalmogaal//////////////////////////////////////////////////////////////////////////////// // Detector - Anomalous data detection // // Copyright 2013, Göteborg Bit Factory. // // Permission is hereby granted, free of charge, to any person obtaining a copy // of this software and associated documentation files (the "Software"), to deal // in the Software without restriction, including without limitation the rights // to use, copy, modify, merge, publish, distribute, sublicense, and/or sell // copies of the Software, and to permit persons to whom the Software is // furnished to do so, subject to the following conditions: // // The above copyright notice and this permission notice shall be included // in all copies or substantial portions of the Software. // // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS // OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL // THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, // OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE // SOFTWARE. // // http://www.opensource.org/licenses/mit-license.php // //////////////////////////////////////////////////////////////////////////////// #ifndef INCLUDED_DETECTOR_H #define INCLUDED_DETECTOR_H #include class Detector { public: Detector (); void algorithm (const std::string&); void max (double); void min (double); void sample (int); void coefficient (double); void quiet (); void execute (const std::string&); void pid (int); void debug (); void run (); private: void run_threshold (); void run_stddev (); void react (const std::string&); void react_complain (const std::string&); void react_execute (); void react_sigusr1 (); private: std::string _algorithm; bool _use_max; bool _use_min; double _max; double _min; int _sample; double _coefficient; bool _quiet; std::string _script; int _pid; bool _debug; long long _counter; }; #endif //////////////////////////////////////////////////////////////////////////////// anomaly-1.1.0/CMakeLists.txt0000644000175000017500000000465612214651444015211 0ustar mogaalmogaalcmake_minimum_required (VERSION 2.8) set (HAVE_CMAKE true) project (anomaly) set (PROJECT_VERSION "1.0.0") SET (ANOMALY_MAN3DIR share/man/man1 CACHE STRING "Installation directory for man pages, section 1") SET (ANOMALY_LIBDIR lib CACHE STRING "Installation directory for the library") SET (ANOMALY_DOCDIR share/doc/anomaly CACHE STRING "Installation directory for doc files") if (${CMAKE_SYSTEM_NAME} MATCHES "Linux") set (LINUX true) elseif (${CMAKE_SYSTEM_NAME} MATCHES "Darwin") set (DARWIN true) elseif (${CMAKE_SYSTEM_NAME} MATCHES "FreeBSD") set (FREEBSD true) elseif (${CMAKE_SYSTEM_NAME} MATCHES "OpenBSD") set (OPENBSD true) elseif (${CMAKE_SYSTEM_NAME} MATCHES "NetBSD") set (NETBSD true) elseif (${CMAKE_SYSTEM_NAME} MATCHES "SunOS") set (SOLARIS true) else (${CMAKE_SYSTEM_NAME} MATCHES "Linux") set (UNKNOWN true) endif (${CMAKE_SYSTEM_NAME} MATCHES "Linux") message ("-- Looking for SHA1 references") if (EXISTS ${CMAKE_SOURCE_DIR}/.git/index) set (HAVE_COMMIT true) execute_process (COMMAND git log -1 --pretty=format:%h WORKING_DIRECTORY ${CMAKE_SOURCE_DIR} OUTPUT_VARIABLE COMMIT) configure_file ( ${CMAKE_SOURCE_DIR}/commit.h.in ${CMAKE_SOURCE_DIR}/commit.h) message ("-- Found SHA1 reference: ${COMMIT}") endif (EXISTS ${CMAKE_SOURCE_DIR}/.git/index) set (PACKAGE "${PROJECT_NAME}") set (VERSION "${PROJECT_VERSION}") set (PACKAGE_BUGREPORT "support@yootabory.org") set (PACKAGE_NAME "${PACKAGE}") set (PACKAGE_TARNAME "${PACKAGE}") set (PACKAGE_VERSION "${VERSION}") set (PACKAGE_STRING "${PACKAGE} ${VERSION}") message ("-- Configuring cmake.h") configure_file ( ${CMAKE_SOURCE_DIR}/cmake.h.in ${CMAKE_SOURCE_DIR}/cmake.h) add_subdirectory (src) add_subdirectory (doc) if (EXISTS test) add_subdirectory (test EXCLUDE_FROM_ALL) endif (EXISTS test) set (doc_FILES NEWS ChangeLog INSTALL AUTHORS COPYING) foreach (doc_FILE ${doc_FILES}) install (FILES ${doc_FILE} DESTINATION ${ANOMALY_DOCDIR}) endforeach (doc_FILE) # --- set (CPACK_SOURCE_GENERATOR "TGZ") set (CPACK_SOURCE_PACKAGE_FILE_NAME ${PACKAGE_NAME}-${PACKAGE_VERSION}) set (CPACK_SOURCE_IGNORE_FILES "CMakeCache" "CMakeFiles" "CPackConfig" "CPackSourceConfig" "_CPack_Packages" "cmake_install" "install_manifest" "Makefile$" "test" "/\\.gitignore" "/\\.git/" "swp$") include (CPack) anomaly-1.1.0/ChangeLog0000644000175000017500000000143412214651444014212 0ustar mogaalmogaal------ current release --------------------------- 1.1.0 () <> + Implements '--details' to show internal calculations. + Documentation updates, corrections. ------ old releases ------------------------------ 1.0.0 (2013-02-24) + Unit test suite. + Implements '--pid' to signal a program on detection. + Implements '--execute' to run arbitrary programs on detection. + Implements '--quiet' to prevent detection messages. + Implements '--stddev' detection with '--coefficient' (multiplied by sigma) and '--sample' (sample size) support. + Implements '--threshold' detection with '--min' and '--max' bounds support. + Complete anomaly.1 man page, with examples. + Uses cmake build system. Project started February 23, 2013. ------ start ----------------------------------- anomaly-1.1.0/commit.h.in0000644000175000017500000000015112214651444014501 0ustar mogaalmogaal/* commit.h.in. Creates commit.h during a cmake run */ /* git information */ #define COMMIT "${COMMIT}" anomaly-1.1.0/INSTALL0000644000175000017500000000434012214651444013470 0ustar mogaalmogaalInstallation Instructions ------------------------- Please follow the instructions below to build anomaly with cmake. Pre-requisites -------------- You will need the 'cmake' build system installed in order to build anomaly from source. More information on cmake can be obtained at http://cmake.org Basic Installation ------------------ Briefly, these shell commands will unpack, build and install anomaly: $ tar xzf anomaly-X.Y.Z.tar.gz [1] $ cd anomaly-X.Y.Z [2] $ cmake . [3] $ make [4] $ make test [5] $ sudo make install [6] $ cd .. ; rm -r anomaly-X.Y.Z [7] These commands are explained below: 1. Unpacks the source tarball. This creates the directory anomaly-X.Y.Z, containing all the code. 2. Change directory to the root of the distribution. 3. Invokes cmake to scan for dependencies and machine-specific details, then generate the makefiles. 4. Builds anomaly. This may take a minute. 5. Optional step that runs the unit tests. On completion, will report the number of passing and failing tests. There should be zero failing tests. If you see failing tests, please report this. 6. Installs the program, documentation and other data files. 7. Removes the temporary directory. Uninstallation -------------- To uninstall anomaly, you need the Makefiles, so if you deleted them in step 7 above, they must first be regenerated by following steps [1], [2] and [3]. Then simply run: $ sudo make uninstall TODO TODO TODO -- cmake doesn't include uninstall Anomaly Build Notes ---------------- Anomaly 1.0.0 has dependencies that are detected by cmake in almost all cases, but there are situations and operating systems that mean you will need to offer a little help. If anomaly will not build on your system, first take a look at the Operating System notes below. If this doesn't help, then go to the Troubleshooting section, which includes instructions on how to contact us for help. Operating System Notes ---------------------- ... Troubleshooting --------------- ... --- anomaly-1.1.0/cmake.h.in0000644000175000017500000000115712214651444014300 0ustar mogaalmogaal/* cmake.h.in. Creates cmake.h during a build */ /* Package information */ #define PACKAGE "${PACKAGE}" #define VERSION "${VERSION}" #define PACKAGE_BUGREPORT "${PACKAGE_BUGREPORT}" #define PACKAGE_NAME "${PACKAGE_NAME}" #define PACKAGE_TARNAME "${PACKAGE_TARNAME}" #define PACKAGE_VERSION "${PACKAGE_VERSION}" #define PACKAGE_STRING "${PACKAGE_STRING}" /* git information */ #cmakedefine HAVE_COMMIT /* Compiling platform */ #cmakedefine LINUX #cmakedefine DARWIN #cmakedefine CYGWIN #cmakedefine FREEBSD #cmakedefine OPENBSD #cmakedefine NETBSD #cmakedefine HAIKU #cmakedefine SOLARIS #cmakedefine UNKNOWN