Reliabilities were high and comparable to those for the Wechsler Adult Intelligence Test, (WAIS-III).  For example, for both men and women ages 21-30 the mean section reliability is .81 and the total score reliability is .95.  Reliabilities were computed by the Kuder-Richardson-21 formula, which in the author=s experience yields values about .02 lower than Cronbach Alpha reliabilities. 

The comparable Wechsler III Performance sub-tests have a median split-half reliability of .83 and total score reliabilities ranging from .88 to .92 across various age groups (media.wiley).  The WAIS-III mean of test-retest reliabilities for the five performance sub-tests is .75 (range .67 to .81) in a sample of 100 16-29 year-olds.  For the total score, the reliability is .88 for this sample (Tulsky and Zhu, p. 58).

Thus, the present test reliability appears to be as good as or better than that of the widely used WAIS-III test.

Internal consistency

A few test section and total scores have minor correlations with age and education.  When these factors are controlled for, the five test sections for adults have a mean between-test correlation of  .45 (range .40 to .50) and all sections correlate substantially with the total score, with a mean of .75 (range .72 to .78).  This is interpreted to indicate that each section contributes equally a unique and valuable element to the total score. It is desirable to have low correlations between test sections and high correlations between test sections and the total score to which they are contributing, as this maximizes the value of each section as a unique contributor to the total score and maximizes its reliability.

These values are somewhat better than those for the early WAIS test.  For example, for a sample of 300 males and females ages 20-34, the mean of the correlations between the five WAIS Performance tests is .53 (range of .44 to .62).  The median correlation between these five tests and the total Performance I.Q. score is only .53 (range .44 - .62) (Wechsler, p. 100).

However, for the more recent WAIS-III Wechsler test, the mean of the correlations between the basic five performance tests is .47 (range .37-.60).  The mean of the correlations between these five and the total Performance I.Q. of which they are a part is .76 (range .68 to .79) (Tulsky and Zhu, 1997).

Thus, the present author=s internal consistency test data is close to that of this widely used current WAIS-III test, with a mean between-test correlation of .45 compared to .47 for the WAIS-III, and a mean of correlations between sub-tests and total score of .75 compared to .76 for the WAIS-III.

Regarding age, the only significant correlation for women was with Common Hand Tools (.27**).  For men the significant correlations were for Physics (.16**), Tools (.28**) and Total Score (.16*). 

Regarding education, no tests correlated significantly with education for men.  All did for women, but not strongly:  .10*, .10*, .11*, .23* .12* and .19** respectively for the five sub-tests and the total score.

This general lack of relationship between test scores and age and education is interpreted to indicate that the tests are measuring innate aptitude for learning ("intelligence") more than amount learned ("achievement"), and may be relatively "culture free".  Ethnic background was not assessed in the initial test takers due to an oversight but will be solicited in subsequent use to obtain data for checking possible minority group bias.

Validity

Other than content validity, the test currently has no other objectively established validity.  However, given its high reliability, it is expected to be as valid as the Wechsler Performance tests and similar spatial aptitude tests for predicting relevant behavior, such as success and enjoyment in vocations and hobbies requiring spatial aptitude.

Initial Norms and Report Format

Norms as of February 21, 2007 were for about 1250 persons tested over the Internet.  Norms as of this date were not yet large enough for children under 16 or adults over 60.

The educational backgrounds of the 1250 persons who had taken the test as of February, 21, 2007 are as follow:

 

Norms as of this sample were by gender and age group, in 10-year segments from 21 to 30, etc., except 16 to 20 for teens and young adults.  The top group was 51 to 60.  I.Q.s are based on a mean of 100 and standard deviation of 15.  The printed report provides all scores, percentile equivalents and a brief explanation of the general meaning of the scores as measures of aptitude rather than achievement.

Norms were to be increased as data is obtained from administration of the test to Internet customers, as has been done for the Kids I.Q. test, which is currently normed on several thousand children.

Sample #2, Updating Research Data

In September of 2007 all data available to date was analyzed.  The sample totaled 2,854, 1011 males and 1843 females.  The subjects ranged in age from 5 to 90.  Nationality data was available for about 1600 subjects and included persons from Australia, Canada, Hong Kong, India Ireland New Zealand, Pakistan, the Philippines, South Africa, the United Kingdom and the United States.  39 subjects were from "other countries". 

Alpha reliability coefficients were computed for each ten-year age group, e.g. for ages 20 to 29, 30 to 39, etc. separately for males and females.  These alpha coefficients were generally in the .80's and .90's.  The total score alphas were all in the 90's except for males in their 60's (.86). 

Mean raw scores for the five test sections and total score were plotted on a graph for each 10-year age group and for children aged 5 to 10.  The mean for each group beginning at group 30 to 40 was computed and plotted at the midpoint age for that group (e.g. age 35 for the 30 to 39 year old group.  These graphs were prepared for males and females separately.

For the section scores, the resulting curves rose steadily to age 25 and then remained essentially level to age 45.  After that they gradually declined so that the raw scores for the 60 to 69 age group were about equal to those for the 20 to 29 age group.  Of note was the more gradual slope of the Common Hand Tools score and its climb to a later peak age, the 50s for women and the 60's for men.

These curves were then used to determine estimated means for each specific age from 5 to 25.  Standard deviations for these ages were also estimated based on data averaged across age groups, e.g. 10 to 19, 20 to 25.  These estimated means were close to the actual means for each teen year, e.g. 14, 15 and 16 but provided a smoothly consistent rise that seems a more reasonable basis for future norm data.  The sample sizes were small for ages below 10 and the actual scores appear to be for very bright children.  Therefore, for this age range, 5 to 10, the smoothed curve means are much different than the actual means.

Similarly, standard deviations tended to vary from age to age, so mean standard deviations were computed.  The smoothed score means and mean standard deviations are used for I.Q. report computations as of late September, 2007, as these are judged to closely approximate the true values for the English speaking population likely to take the test over the Internet.

For example consider these actual and smoothed scores for the physics sub-test, ages 5 through 25:

Means and Standard Deviations for Physics Subtest, Males,

Actual and Smoothed, Ages 5 to 25

 

The total raw score (across all five sections) tended to rise steadily to the 50 to 59 age group and then decline slightly in a smooth arc.

Norms as of late September, 2007

Norms as of late September, 2007 will be by gender and age, for each specific age from 5 to 25 and in clusters of age (e.g. 30 to 39) above 25.  The total sample as of this norming is 2854.