Conditional Expected Value

            Conditional Expected Value is the expected value of a real random variable given than a particular event has occurred. Thus, if X is a random variable taking values in a set S, and A is an event whose probability is not 0 (P(A) ≠ 0), the conditional expected value of X given A, denoted by E(X|A), is the expected value of X if event A has occurred. Similarly, if Y is another random variable taking values in a set Y, the conditional expected value of X given Y=y, denoted by E(X|Y=y) is the expected value of X if Y is equal to a real value y. (Grimmett 146)

            In inferential statistics, we try to estimate a parameter of the population from the samples drawn from it. Confidence Interval is the measure of the accuracy of our estimate. A 95% confidence interval over an interval [a,b] indicates that the probability the true population slope coefficient – or simply put, the correct value – is between a and b, denoted by P([a,b]), is equal to 0.95. Thus, a wider confidence interval means we are less certain of our estimate of the parameter. Also, the higher the level of confidence, that is P([a,b]), the wider the confidence interval will be. (“Two-Variable Regression”)

            Ordinary Least Squares is an approach for assigning values to unknown quantities in a statistical model, based on observed data. Its values are the values minimizing the variance. (Abdi 322) A least-squares estimator is said to be BLUE if it is the best linear unbiased estimator. When a coefficient is unbiased, it means that the expected value is equal to the true value. Moreover, when a least-squares estimator has the minimum variance, it means that it is the most efficient within the class of all linear unbiased estimators. (Abdi 189)

            In statistical hypothesis testing, the p-value is the probability that the drawn sample could have been drawn from the population being tested, assuming the null hypothesis is true. Hence, a p-value of 0.05 means that there is a 5% chance of drawing the sample being tested, provided the null hypothesis is true. Moreover, a p-value close to zero indicates that the null hypothesis is false, while values close to 1 indicate that there is no detectable difference for the sample size used. (“P-Value”) The Level of Significance of the test result is the probability that the null hypothesis will be rejected in error when it is true (Type I error). The level of significance is inversely proportional to the p-level; the smaller the p-value, the more significant the result is said to be. (Goodman 96)

            R², or the Coefficient of Determination, is the percentage of variance in a data set of a statistical model. It is the statistic that gives information about the integrity of the fit of the model. (Smith 294) F Statistic is the ratio of two sample variances, and is used to test the null hypothesis. R2 and F statistic are directly related; an R2 value closer to 1.00 is considered because it explains a lot of variation in the dependent variable. (Mangiero) The quantity 1-R2 is defined as the Coefficient of Non-determination, and is the percent of variation which is unexplained by the regression equation. It is used in the t-test to see if there is significant linear correlation. (Jones)

References:

Abdi, H. “Encyclopedia for research methods for the social sciences.” Least-squares. 2003.

Goodman, S. “Toward evidence-based medical statistics. 1: The P value fallacy”. Ann Intern Med. 1999.

Grimmett, G. R., and Stirzaker, D. R. Probability and Random Processes. 1993.

Gujarati, Damodar N. (2003). Basic Econometrics. 4th Edition. New York, NY: McGraw-Hill.

Jones, James. “Stats: Coefficient of Determination”.  Math 170 Lecture Notes. 2007. <http://www.richland.edu/james/lecture/m170/ch11-rsq.html>.

Mangiero, G. A., and Mangiero, S. M. “Anatomy of Regression”. 2007. Financial Engineering News. 08 May 2007. <http://www.fenews.com/fen16/regression.html>. 

P-Value. 2007. i Six Sigma. 08 May 2007. <http://www.isixsigma.com/dictionary/P-Value-301.htm>.

Smith, H., and Draper N. R. Applied Regression Analysis. Third Edition. New York NY: John Wiley and Sons, Inc., 1998.

Two-Variable Regression: Interval Estimation and Hypothesis Testing. 08 May 2007. <unsite.wits.ac.za/sebs/downloads/2005/chapter_5_.doc>.