The Bilingual Verbal Ability Test (BVAT): A Breakthrough in Bilingual Assessment - or Is It?
The test is designed to help in developing entry and exit criteria in bilingual education, to facilitate appropriate program placement and planning, and to assess a bilingual student's academic readiness.
The BVAT consists of the Comprehensive Manual, the Easel Test Book, different language-specific format test records, a computer disk (either Window or Macintosh format) containing a scoring and report writing program and a training videotape.
The BVAT is comprised of three individually administered subtests:
1. Picture Vocabulary: The subject is required to name a pictured object with the degree of difficulty increasing gradually. It is an expressive language task that involves word retrieval ability at a single word level and measures comprehension/ knowledge ("crystallized intelligence").
2. Oral Vocabulary
Synonyms: The subject is required to make a synonymous word association with difficulty increasing gradually. This task measures knowledge of word meanings ("crystallized intelligence") during an oral presentation of stimuli.
Antonyms: The subject is required to make an opposite (antonymous) word association with difficulty increasing gradually. This task measures knowledge of word meanings ("crystallized intelligence") during an oral presentation of stimuli.
3. Verbal Analogies: The subject is required to recognize the analogous relationships between two words and to find a word that fits the same relationship to a third word. This task measures verbal reasoning in increasingly more complex conceptual/ logical connections ("fluid intelligence").
The above subtests were drawn from the Woodcock-Johnson-Revised (1989) Cognitive Battery and translated into 15 different languages, presumably the most widely used languages in the USA: Arabic, Chinese (Simplified and Traditional), English, French, German, Haitian-Creole, Hindi, Italian, Japanese, Korean, Polish, Portuguese, Russian, Spanish, Turkish, and Vietnamese. The best practice for administration of the BVAT is by a trained bilingual examiner. However, when the bilingual examiner is not available, the Manual suggests a training for a Primary/Ancillary Team (pp. 51-59), and defines the role, limits, and scope of functioning of the "ancillary" examiner (who is not a translator, just a person with mastery of the targeted language) insisting that a "primary" examiner remains at the testing site and bear full responsibility for the accuracy of the procedure, as well as scoring and interpretation of the results.
The procedure includes administration of all the subtests in the English language first. Each item failed in English is re-administered in the native language. If the child gets that item correct in his/her native language, it is added to the score for that subtest, The overall subtest score is based on the child's knowledge/reasoning skills using both languages, thus reflecting the very nature of the bilingual ability. Time Time is up to 30 minutes and depends on a child's age and language proficiency.
There are two basic options for the BVAT interpretation: age-based or grade-based. In addition to standard scores, the relative proficiency index (the same indicator as the Relative Mastery Index in the WJ-R), and percentile ranks, the BVAT offers "instructional zones" index, five levels of English language proficiency (from Negligible through Very Limited, Limited, Fluent to Advanced), and aptitude/achievement discrepancies in relation to the WJ-R Achievement Tests. (The rationale for using the BVAT as an aptitude measure is based on the assumption that the BVAT score may be interpreted as "...the level of English language ability that would be demonstrated by the bilingual if all of that person's language abilities were available in English" (Manual, p. 3). All scoring is automated through the "Scoring and Reporting Program" software, which is a standard feature of the BVAT kit. The BVAT Comprehensive Manual contains the "Examiner Training and Practice Exercises". A training videotape, prepared by the publisher, accompanies the test.
According to the Comprehensive Manual, normative data for BVAT was obtained during WJ-R COG standardization (1988-89)
Reliability was calculated through the traditional (a split-half procedure corrected for length by the Spearman-Brown formula and resulted in the high .80s for the subtests and in the mid .90 for the clusters) and an innovative "alternate-form alternate-procedure" operation. Following a counterbalanced design (so-called "a two-way study"), the participants were tested in English and the gains were obtained by re-testing in Spanish (a translation). Then, the same participants were tested in Spanish (Bateria, 1996) and the gains were obtained in English (a translation). The Manual (p. 89) reports the median BVAT reliability for the 542 bilingual subjects as .84. Due to the lack of parallel forms in other languages the "two way study" with bilinguals was not carried out in any other language (Manual, pp. 85-86).
Content validity is crucial for a test translated into 15 different languages. The "comparability" of translation was attempted through an 8-step procedure described in the Manual on pp. 68-70. This appears as a very meticulous and thoughtful proceeding; still, native speakers of these languages may find inadequacies due to the enormous difficulty of the translation itself. Some of the 15 languages have deleted items (sometimes up to four items) because of the "untranslatable" nature of these items (Manual, Table 8-5, p.71). It is not clear what effect it may have on the scores obtained if they are based on norms that include all items.
Another aspect of the content validity is the issue of "uneven" complexity (from a relatively easy naming task in Picture Vocabulary to a much more difficult verbal reasoning task in Verbal Analogies). The authors tried to minimize this difficulty by introducing the "cluster concept" which is based on the "convergence" of three different aspects of verbal ability: receptive language, expressive language and verbal reasoning. The concurrent validity study was attempted on the English/Spanish bilingual population using as criteria measures eight well known tests of verbal abilities (Manual, p.74, Table 8-10). The correlation coefficient was within the range of .7 to .9 on average. A predictive validity study of the BVAT and a number of achievement tests produced a correlation coefficient ranging from .65 to .85.(Manual, pp.70-79). Construct validity was reported within the .7 to .9 range.
Publication of the BVAT is a significant event in the field of bilingual assessment. The Publisher and the authors should be complimented for their bold effort to measure the total bilingual competency in bilingual individuals. We have to understand, however, that this is only a first step of a long journey: The BVAT measures only a part of the child's cognitive-academic language (e.g., literacy skills remain beyond the scope of the test) and the content of the test is too narrow to make any long-term instructional planning or an educational placement decision based on this test alone. Item-by-item translation in a particular language may not satisfy some bilingual (in this language) professionals due to the regional variations and possibility of different ways of translating certain items. Sometimes the English word has a meaning that corresponds to several words in the targeted language: the synonym/antonym subtest is particularly vulnerable to this situation.
More important, however, is the fact that the level of difficulty may change significantly (from easy to difficult and vice versa) as the result of translation. Professionals should use particular caution with the Picture Vocabulary subtest, the weakest in the battery (in terms of its reflection of the cognitive skills and operations) and the most culturally biased. In my experience, some items in this subtest are too culture-specific (e.g. men panning gold) to be passed even by older and educationally advanced bilingual children. Hand-scoring is not available for this test: the Publisher assumes access to computers by all assessment personnel, which is not always the case.
The computer generated report available for the BVAT software users should be used as a template only for a qualitative and individualized interpretation of a student's performance. Restrictions related to norming (see above) should also be taken into consideration while interpreting the results of this test. The BVAT is the first attempt to create a standardized test for assessing the "common underlying proficiency" (Cummins) in utilizing cognitive-academic language. The BVAT goes beyond the existing practices of testing separately mastery of English and the child's first language. Probably for the first time in the history of psychometric measurements it provides a "holistic" overall estimate of a bilingual child's cognitive language mastery - an event long overdue in the field of bilingual assessment (Valdes & Fifueroa, 1997).
What is new and important, the BVAT includes other languages, not only Spanish. The BVAT is educationally relevant because it measures the cognitive-academic aspect of language proficiency needed for learning in schools. In sum, the BVAT is an effective (although not perfect!) instrument in bilingual assessment.
Comprehensive Manual, BVAT, Riverside, 1998
Blanc, M. & Hamers, J-A. (1989). Bilinguality and Bilingualism. Cambridge U. Press, New York.
Cummins, J. (1984). Bilingualism and Special Education: Issues in Assessment and Pedagogy. Clevedon, England: Multilingual Matters.
Cummins, J. (1996). Negotiating Identities: Education for Empowerment in a Diverse Society. Los Angeles: CABE
Lopez, E. (1997). The Cognitive Assessment of Limited English Proficiency and Bilingual Students. In: Contemporary Intellectual Assessment. The Guilford Press, NY.
Mohanty, A.K. & Perragaux, C. (1996). Language Acquisition and Bilingualism. In: Betty, Dasen, Saraswathi (Eds.) Handbook of Cross-cultural Psychology: Basic Processes and Human Development, Volume 2, pp. 217-253. Allyn & Bacon, Needhem Heights, MA.
Sattler, J. (1992). Assessment of Children. (Revised and Updated Third Edition). San-Diego.
Valdes G., & Figueroa R., (1994). Bilingualism and Testing: A Special Case of Bias. Ablex Publishing, Norwood,NJ.