When the covariance matrix Sigma(pxp) does not satisfy the formal factor analysis model for m factors, there will be no factor matrix Lambda(pxm) such that Upsilon= (Sigma-Lambda Lambda') is diagonal. The factor analysis model may then be replaced by a tautology where Upsilon is regarded as the covariance matrix of a set of "residual variates." These residual variates are linear combinations of "discarded" common factors and unique factors and are correlated. Maximum likelihood, alpha and iterated principal factor analysis are compared in terms of the manner in which Upsilon is defined, a "maximum determinant" derivation for alpha factor analysis being given. Weighted least squares solutions using residual variances and common variances as weights are derived for comparison with the maximum likelihood and alpha solutions. It is shown that the covariance matrix Upsilon defined by maximum likelihood factor analysis is Gramian provided that all diagonal elements are nonnegative. Other methods can define a Upsilon which is non-Gramian even when all diagonal elements are nonnegative.