fast statistics 2.0.3

Fast Statistics description
Fast Statistics is a Statistical and Graphical Analysis program for MS Excel Fast Statistics is a Statistical and Graphical Analysis program for MS Excel, it can work as an Excel add-in and stand-alone software without excel installed, perform from basic data manipulations to the most advanced statistical analyses and produce sophisticated reports and charts. This is a ideal tool for Six Sigma, descriptive statistics, ANOVA, regression, dummy regression, intelligent data input, factor analysis, nonlinear regression, box plots, nonparametric tests, and correspondence analysis.

Fast Statistics includes more than 50 functions covering data and statistical-analysis requirements (preparing, describing, analyzing, and modeling data). Excel utilities have also been included to facilitate charting and data manipulation. Complementary modules are available.

Fast Statistic also gives professionals engaged in quality improvement the powerful , reliable, and easy to use tools they need to effectively analyze hard data while managing large and complex projects. It is the ideal package for Six Sigma and other quality improvement projects. From Statistical Process Control to Design of Experiments, it offers you the methods you need to implement every phase of your quality project.

Here are some key features of "Fast Statistics":

Easy to Use
· works as an Excel Add-ins
· Read and edit Excel 95/97/2000/XP files (do not need MS Excel installation)
· Intelligent data input system
· Comprehensive HTML Help
Data and File Management
· Export HTML Statistical and Graphical Analysis report
· Import and export: Excel, text, and other data formats
· Import Microsoft Excel workbook sheets individually
· support import currency formats
· Data manipulation
· Search and replace in Data window
· support Double-precision worksheets
Simulation and Distributions
· Random number generator
· Density, distribution, and inverse cumulative distribution functions
Statistics
· Basic statistics: Display descriptive statistic, Correlation, Covariance
· Hypothesis test: Sign test, Wilcoxon rank sum test, Z test, T test, Chi squared test, F test
· Analysis of Variance(ANOVA): One way ANOVA, Two way ANOVA, Interaction plot, Main effect plot
· Regression: Multiple linear regression, Dummy-variable regression, Non-linear regression
· Nonparametrics : Friedman, Kruskal-Wallis
· Goodness of Fit
· Time Series: Trend analysis, Moving average, Single exponential smoothing, Double exponential smoothing, Decomposition, Winters Method, Autocorrelation, Autocovariance, Partial autocorrelation
· DOE: Full factorial design, Fractional factorial design, Analyze factorial design, Factorial plot
Graphics
· Pictorial gallery and streamlined dialog boxes simplify graph creation
· Interactively edit attributes (axes, scale, etc.) and recreate custom graphs with new data
· Intuitive tool to place multiple graphs on one page
· Set your own preferences for graph attribute defaults
· support Rotating, Scrolling and Moving 3D data graphs
· Generate Animation chart
· Basic Graph: Line, HorizLine, Area, Point(Scatter), Bar, HorizBar, Pie, Shapes, Stairs Line, Arrows, Bubble
· Statistical Graph: Histogram, Box plot, Contour, Time series plot, 3D plot, Probability plot
· Quality Control Charts: X chart, R chart, S chart, P chart, NP chart, C chart, U chart, Pareto chart, EWMA
· Capability analysis

Statistics::ChiSquare - How well-distributed is your data?

SYNOPSIS

use Statistics::Chisquare;

print chisquare(@array_of_numbers);
Statistics::ChiSquare is available at a CPAN site near you.

Suppose you flip a coin 100 times, and it turns up heads 70 times. Is the coin fair?

Suppose you roll a die 100 times, and it shows 30 sixes. Is the die loaded?
In statistics, the chi-square test calculates how well a series of numbers fits a distribution. In this module, we only test for whether results fit an even distribution. It doesnt simply say "yes" or "no". Instead, it gives you a confidence interval, which sets upper and lower bounds on the likelihood that the variation in your data is due to chance. See the examples below.

If youve ever studied elementary genetics, youve probably heard about Georg Mendel. He was a wacky Austrian botanist who discovered (in 1865) that traits could be inherited in a predictable fashion. He did lots of experiments with cross breeding peas: green peas, yellow peas, smooth peas, wrinkled peas. A veritable Brave New World of legumes.

But Mendel faked his data. A statistician by the name of R. A. Fisher used the chi-square test to prove it.

Theres just one function in this module: chisquare(). Instead of returning the bounds on the confidence interval in a tidy little two-element array, it returns an English string. This was a deliberate design choice---many people misinterpret chi-square results, and the string helps clarify the meaning.

The string returned by chisquare() will always match one of these patterns:

"Theres a >d+% chance, and a or
"Theres a or
"I cant handle d+ choices without a better table."

That last one deserves a bit more explanation. The "modern" chi-square test uses a table of values (based on Pearsons approximation) to avoid expensive calculations. Thanks to the table, the chisquare() calculation is very fast, but there are some collections of data it cant handle, including any collection with more than 31 slots. So you cant calculate the randomness of a 50-sided die.
You will also notice that the percentage points that have been tabulated for different numbers of data points - that is, for different degrees of freedom - differ. The table in Jon Orwants original version has data tabulated for 100%, 99%, 95%, 90%, 70%, 50%, 30%, 10%, 5%, and 1% likelihoods. Data added later by David Cantrell is tabulated for 100%, 99%, 95%, 90%, 75%, 50%, 25%, 10%, 5%, and 1% likelihoods.

EXAMPLES

Imagine a coin flipped 1000 times. The expected outcome is 500 heads and 500 tails:

@coin = (500, 500);
print chisquare(@coin);

prints "Theres a >90% chance, and a <100% chance, that this data is random.
Imagine a die rolled 60 times that shows sixes just a wee bit too often.

@die1 = (8, 7, 9, 8, 8, 20);
print chisquare(@die1);

prints "Theres a >1% chance, and a <5% chance, that this data is random.
Imagine a die rolled 600 times that shows sixes way too often.

@die2 = (80, 70, 90, 80, 80, 200);
print chisquare(@die2);

prints "Theres a <1% chance that this data is random."
How random is rand()?

srand(time ^ $$);
@rands = ();
for ($i = 0; $i < 60000; $i++) {
$slot = int(rand(6));
$rands[$slot]++;
}
print "@randsn";
print chisquare(@rands);

prints (on my machine)

10156 10041 9991 9868 10034 9910
Theres a >10% chance, and a <50% chance, that this data is random.

So much for pseudorandom number generation.

Statistics::OLS is a Perl module to perform ordinary least squares and associated statistics.

SYNOPSIS

use Statistics::OLS;

my $ls = Statistics::OLS->new();

$ls->setData (@xydataset) or die( $ls->error() );
$ls->setData (@xdataset, @ydataset);

$ls->regress();

my ($intercept, $slope) = $ls->coefficients();
my $R_squared = $ls->rsq();
my ($tstat_intercept, $tstat_slope) = $ls->tstats();
my $sigma = $ls->sigma();
my $durbin_watson = $ls->dw();

my $sample_size = $ls->size();
my ($avX, $avY) = $ls->av();
my ($varX, $varY, $covXY) = $ls->var();
my ($xmin, $xmax, $ymin, $ymax) = $ls->minMax();

# returned arrays are x-y or y-only data
# depending on initial call to setData()
my @predictedYs = $ls->predicted();
my @residuals = $ls->residuals();

I wrote Statistics::OLS to perform Ordinary Least Squares (linear curve fitting) on two dimensional data: y = a + bx. The other simple statistical module I found on CPAN (Statistics::Descriptive) is designed for univariate analysis. It accomodates OLS, but somewhat inflexibly and without rich bivariate statistics. Nevertheless, it might make sense to fold OLS into that module or a supermodule someday.

Statistics::OLS computes the estimated slope and intercept of the regression line, their T-statistics, R squared, standard error of the regression and the Durbin-Watson statistic. It can also return the residuals.

It is pretty simple to do two dimensional least squares, but much harder to do multiple regression, so OLS is unlikely ever to work with multiple independent variables.

This is a beta code and has not been extensively tested. It has worked on a few published datasets. Feedback is welcome, particularly if you notice an error or try it with known results that are not reproduced correctly.

MaxEntropy is a Perl5 module for Maximum Entropy Modeling and Feature Induction.

SYNOPSIS

use Statistics::MaxEntropy;

# debugging messages; default 0
$Statistics::MaxEntropy::debug = 0;

# maximum number of iterations for IIS; default 100
$Statistics::MaxEntropy::NEWTON_max_it = 100;

# minimal distance between new and old x for Newtons method;
# default 0.001
$Statistics::MaxEntropy::NEWTON_min = 0.001;

# maximum number of iterations for Newtons method; default 100
$Statistics::MaxEntropy::KL_max_it = 100;

# minimal distance between new and old x; default 0.001
$Statistics::MaxEntropy::KL_min = 0.001;

# the size of Monte Carlo samples; default 1000
$Statistics::MaxEntropy::SAMPLE_size = 1000;

# creation of a new event space from an events file
$events = Statistics::MaxEntropy::new($file);

# Generalised Iterative Scaling, "corpus" means no sampling
$events->scale("corpus", "gis");

# Improved Iterative Scaling, "mc" means Monte Carlo sampling
$events->scale("mc", "iis");

# Feature Induction algorithm, also see Statistics::Candidates POD
$candidates = Statistics::Candidates->new($candidates_file);
$events->fi("iis", $candidates, $nr_to_add, "mc");

# writing new events, candidates, and parameters files
$events->write($some_other_file);
$events->write_parameters($file);
$events->write_parameters_with_names($file);

# dump/undump the event space to/from a file
$events->dump($file);
$events->undump($file);

This module is an implementation of the Generalised and Improved Iterative Scaling (GIS, IIS) algorithms and the Feature Induction (FI) algorithm as defined in (Darroch and Ratcliff 1972) and (Della Pietra et al. 1997). The purpose of the scaling algorithms is to find the maximum entropy distribution given a set of events and (optionally) an initial distribution.

Also a set of candidate features may be specified; then the FI algorithm may be applied to find and add the candidate feature(s) that give the largest `gain in terms of Kullback Leibler divergence when it is added to the current set of features.

Events are specified in terms of a set of feature functions (properties) f_1...f_k that map each event to {0,1}: an event is a string of bits. In addition of each event its frequency is given. We assume the event space to have a probability distribution that can be described by
The module requires the Bit::SparseVector module by Steffen Beyer and the Data::Dumper module by Gurusamy Sarathy. Both can be obtained from CPAN just like this module.

Statistics::LineFit module least squares line fit, weighted or unweighted.

SYNOPSIS

use Statistics::LineFit;
$lineFit = Statistics::LineFit->new();
$lineFit->setData (@xValues, @yValues) or die "Invalid data";
($intercept, $slope) = $lineFit->coefficients();
defined $intercept or die "Cant fit line if x values are all equal";
$rSquared = $lineFit->rSquared();
$meanSquaredError = $lineFit->meanSqError();
$durbinWatson = $lineFit->durbinWatson();
$sigma = $lineFit->sigma();
($tStatIntercept, $tStatSlope) = $lineFit->tStatistics();
@predictedYs = $lineFit->predictedYs();
@residuals = $lineFit->residuals();
(varianceIntercept, $varianceSlope) = $lineFit->varianceOfEstimates();

The Statistics::LineFit module does weighted or unweighted least-squares line fitting to two-dimensional data (y = a + b * x). (This is also called linear regression.) In addition to the slope and y-intercept, the module can return the square of the correlation coefficient (R squared), the Durbin-Watson statistic, the mean squared error, sigma, the t statistics, the variance of the estimates of the slope and y-intercept, the predicted y values and the residuals of the y values. (See the METHODS section for a description of these statistics.)
The module accepts input data in separate x and y arrays or a single 2-D array (an array of arrayrefs). The optional weights are input in a separate array. The module can optionally verify that the input data and weights are valid numbers. If weights are input, the line fit minimizes the weighted sum of the squared errors and the following statistics are weighted: the correlation coefficient, the Durbin-Watson statistic, the mean squared error, sigma and the t statistics.
The module is state-oriented and caches its results. Once you call the setData() method, you can call the other methods in any order or call a method several times without invoking redundant calculations. After calling setData(), you can modify the input data or weights without affecting the modules results.
The decision to use or not use weighting could be made using your a priori knowledge of the data or using supplemental data. If the data is sparse or contains non-random noise, weighting can degrade the solution. Weighting is a good option if some points are suspect or less relevant (e.g., older terms in a time series, points that are known to have more noise).

Statistics::Hartigan is a Perl extension for the stopping rule proposed by Hartigan J. Hartigan, J. (1975). Clustering Algorithms. John Wiley and Sons, New York, NY, US.

SYNOPSIS

use Statistics::Hartigan;
&hartigan(InputFile, "agglo", 6, 10);

Input file is expected in the "dense" format -
Sample Input file:

6 5
1 1 0 0 1
1 0 0 0 0
1 1 0 0 1
1 1 0 0 1
1 0 0 0 1
1 1 0 0 1

Hartigan J. uses the Within Cluster/Group Sum of Squares (WGSS) to estimate the number of clusters a given data naturally falls into. The is goal is to minimize WG.

X-Statistics analyses certain aspects of the syslog files, where Mac OS X logs many status information of the system, and shows the information as diagram, graph or table.

At the moment X-Statistics is able to show the following information (future versions will show additional information):Transmitted amount of data (sent, received) for PPP connections (internet)Amount of time being online.

Application crashesMounted volumes (CDs, DVD, Disk Images etc.)General system status (system running, sleeping, switched off)In case youre looking for a new Internet Service Provider you may want to know how much data youre usually upload and download and how long youre usually online per month.

Then you know if you can sign the cheaper contract which includes a certain traffic or a certain number of online hours, or if you need a more expensive one which allows more traffic or a longer online time. X-Statistics can tell you for the last months, how long you were online and how much data youve up- and downloaded.

Here are some key features of "XStatistics":

· Transmitted amount of data (sent, received) for PPP connections (internet)
· Amount of time being online.
· Application crashes
· Mounted volumes (CDs, DVD, Disk Images etc.)
· General system status (system running, sleeping, switched off).


Limitations:

· Registration screen at startup.


Whats New in This Release:

· X-Statistics is now compatible for Mac OS X 10.4.x "Tiger"
· Fpr the PPP statistics the program uses another logfile. So now the PPP information is available for a longer time.

Statistics::GaussHelmert is a general weighted least squares estimation module.

SYNOPSIS

use Statistics::GaussHelmert;

# create an empty model
my $estimation = new Statistics::GaussHelmert;

# setup the model given observations $y, covariance matrices
# $Sigma_yy, an initial guess $b0 for the unknown parameters.
$estimation->observations($y);
$estimation->covariance_observations($Sigma_yy);
$estimation->initial_guess($b0);

# specify the implicit model function and its Jacobians by using
# closures.
$estimation->observation_equations(sub { ... });
$estimation->Jacobian_unknowns(sub { ... });
$estimation->Jacobian_observations(sub { ... });

# Maybe we want to impose some constraints on the unknown
# parameters, this is not mandatory
$estimation->constraints(sub { ... });
$estimation->Jacobian_constraints(sub { ... });

# start estimation
$estimation->start(verbose => 1);

# print result
print $estimation->estimated_unknown(),
$estimation->covariance_unknown();

This module is a flexible tool for estimating model parameters given a set of observations. The module provides function for a linear estimation model, the underlying model is called Gauss-Helmert model.

Statistics::GaussHelmert is different to modules such as Statistics::OLS in the sense that it may fit arbitrary functions to data of any dimensions. You have to specify an implicit minimization function (in contrast to explicit functions as in traditional regression methods) and its derivatives with respects to the unknown and the observations. You may also specify constraint function on the unknowns (with its derivative). Furthermore you already need an approximate solution. For some problems it is easy finding approximate solutions by directly solving for the unknown parameters with some well chosen observations.

Statistics::LTU is an implementation of Linear Threshold Units.

SYNOPSIS

use Statistics::LTU;

my $acr_ltu = new Statistics::LTU::ACR(3, 1); # 3 attributes, scaled

$ltu->train([1,3,2], $LTU_PLUS);
$ltu->train([-1,3,0], $LTU_MINUS);
...
print "LTU looks like this:n";
$ltu->print;

print "[1,5,2] is in class ";
if ($ltu->test([1,5,2]) > $LTU_THRESHOLD) { print "PLUS" }
else { print "MINUS" };

$ltu->save("ACR.saved") or die "Save failed!";
$ltu2 = restore Statistics::LTU("ACR.saved");

EXPORTS

For readability, LTU.pm exports three scalar constants: $LTU_PLUS (+1), $LTU_MINUS (-1) and $LTU_THRESHOLD (0).

Statistics::LTU defines methods for creating, destroying, training and testing Linear Threshold Units. A linear threshold unit is a 1-layer neural network, also called a perceptron. LTUs are used to learn classifications from examples.
An LTU learns to distinguish between two classes based on the data given to it. After training on a number of examples, the LTU can then be used to classify new (unseen) examples. Technically, LTUs learn to distinguish two classes by fitting a hyperplane between examples; if the examples have n features, the hyperplane will have n dimensions. In general, the LTUs weights will converge to a define the separating hyperplane.

The LTU.pm file defines an uninstantiable base class, LTU, and four other instantiable classes built on top of LTU. The four individual classes differs in the training rules used:

ACR - Absolute Correction Rule
TACR - Thermal Absolute Correction Rule (thermal annealing)
LMS - Least Mean Squares rule
RLS - Recursive Least Squares rule

Each of these training rules behaves somewhat differently. Exact details of how these work are beyond the scope of this document; see the additional documentation file (ltu.doc) for discussion.