Introduction-- The National Visiting Committee held its fourth on site meeting with the STEMTEC project PI's on April 10, 1999 at Amherst, Massachusetts. Members present were: Angelo Collins (Vanderbilt), Leo J. Hickey (Yale University), Paul Irish (Champlain Valley Union High School), Arnold Ostebee (St. Olaf College), and Barbara Tewksbury (Hamilton College). Katylee Hoover attended the meeting as representative of the NSF.
Overview and Recommendations-- The NVC feels that the new evaluators have made substantial progress in defining the goals of the project in concrete terms. However, the committee is deeply concerned that the evaluation plan still does not provide an adequate assessment of STEMTECís effectiveness in increasing the number and academic preparedness of undergraduates heading towards K-12 mathematics and science teaching programs. Development and implementation of appropriate evaluation mechanisms for these goals is essential and must be given a higher priority by the PIís. Since "STEMTEC's basic goals are to increase the numbers, diversity, and quality of new science and math teachers" (Mort Sternheim in STEMTREK, Winter/Spring 1999, p.2), the development of tools to assess these goals is overdue. Under the circumstances, the NVC recommends that one or more of the PIís reaffirms STEMTECís commitment to these goals directly in a dialogue with representatives of the NSF.
The NVC believes that STEMTEC was justified in having committed much of its early effort to the improvement of science and math teaching at the college introductory level by encouraging the use of innovative teaching methods. It is not surprising, therefore, that the PIís first concern has been with the superficial features of those modes of instruction--how they are implemented. However, the NVC believes that the PIís have not moved rapidly enough to define these innovative methods precisely, to determine the situations in which their use is appropriate, and to assess their effect. We have argued repeatedly, for example, that not all student-centered activity is valuable activity. The NVC strongly recommends that, during the summer recall meetings in July, STEMTEC instructors be asked to engage in substantive dialog about how content and understanding can be most effectively conveyed using innovative techniques, rather than focusing primarily on mechanics and the implementation of techniques. We urge the PIís to seek help from the evaluators immediately in preparing discussion and evaluation activities that will enable participants to focus on this deeper and crucial level of alternative pedagogy. This should be done by focusing on two crucial areas: first, what constitutes valuable activity (in contrast to merely the best techniques) in achieving the goals of conveying content and fostering understanding and, second, how an instructor can evaluate his or her success in reaching these goals. Given the current state of the evaluation report, it is essential that the evaluators be engaged immediately in preparing this aspect of the summer recall--even if this requires committing additional budgetary resources.
Most importantly, the NVC must express its misgivings concerning the prospects for a positive review of the STEMTEC program at the reverse site visit that will take place next spring. In order for STEMTEC to be viewed as a successful project, the NVC believes that the PI's must prepare an explicit action plan that concentrates on the central goals of their program (as set out under Sect. III.A.3. of the Cooperative Agreement between the STEMTEC PI's and NSF) and its implementation for the training and recruiting of teachers.
At present, the NVC feels that STEMTEC's first goalóestablishment of collaboration (Agreement, Sect. III.A.3.a) has been achieved and should no longer be an issue. Rather the plan should be specifically addressed toward the achievement of goals b, c, and d of Sect. III.A.3 of the Agreement, especially goal c, dealing with teacher recruitment and retention.
The committee feels that goals b and c, (which seek to improve the curriculum for, and to recruit and retain promising students into the teaching profession) will be the hardest to demonstrate. The action plan must include a detailed evaluation initiative that can convincingly link the changes made in STEMTEC courses to improvements in teacher preparation and recruitment. This will probably require the use of a standards-based instrument, such as NESN, the identification and tracking of STEMTEC students, and the identification and evaluation of effective instructional methods. Also, the NVC feels that the evaluation plan thus far lacks an initiative to evaluate the effectiveness of STEMTEC in increasing the number of students heading toward K-12 math and science teaching programs. Development of such an assessment tool, using baseline information, is essential.
On the positive side, it is the NVC's opinion that the case for STEMTEC's performance on its stated goal of recruiting math and science teachers from traditionally underrepresented groups (Agreement, Sect III, A, 3, c,vii) was considerably boosted by the initiative, under the direction of Sharon Palmer, on scholarship awards and the development of science education groups at Massachusetts colleges outside of the collaborative and we strongly encourage its continuation.
A final issue of concern to the NVC involves priorities for what will be the last three years of the project. The NVC further recommends that the PI's clarify their specific areas of responsibility for the operation of this grant, for instance, scientific content to Yuretich, pedagogy to D'Avanzo and Sternheim, operations to Sternheim, and evaluation to Feldman. In addition the NVC recommends that, because resources of funds and time are limited, it would probably be better for STEMTEC not to target additional advanced, college-level science and math courses at this time, but rather concentrate on the areas of evaluation and recruitment and retention of K-12 math and science teachers.
Leo J. Hickey
For the National Visiting Committee
10 May 1999