Introduction-- The National Visiting Committee paid a second site visit to the STEMTEC project on April 3 and 4, 1998. Members present were: Angelo Collins (Vanderbilt University), Leo J. Hickey (Yale University), Paul Irish (Champlain Valley Union High School), John W. Layman (University of Maryland), and Barbara J. Tewksbury (Hamilton College). Jack Hehn also attended as the representative of the National Science Foundation. During its visit, the committee had ample opportunity to discuss the program with the Principal Investigators, to interview the evaluation team at length, and to participate in a portion of the STEMTEC Introductory Meeting for Cycle II participants at Holyoke Community College. In addition, several committee members sat in on a few STEMTEC classes before the NVC meetings began.
In general, NVC members felt that this was a very positive visit. The committee expressed its satisfaction with the progress of the program to date and with the constructive and forthright manner in which the PI's have dealt with points and issues raised in the first NVC report as well as with the results of their own analysis. The NVC's focus during their second visit was on the overall goals and vision of the STEMTEC program and on the closely related issue of evaluation. Discussion of these issues and recommendations regarding them constitute the subject of this report.
Overview and Recommendations-- STEMTEC has now passed successfully through the initial phases of setting up of the cooperative, recruitment of the faculty, and the development of a number of individual courses that embody concepts directed toward what the PI's term "student active learning." With the program at this stage, the NVC is of the opinion that it is time for the PI's to clearly state their overall goals and vision for STEMTEC. This is especially critical in order to develop effective tools for evaluating its impact. Several times during our visit the questions, "What is a STEMTEC course?" and "What do we expect a STEMTEC teacher to be?" were raised, without clear answers being provided. Therefore, the committee feels that it is mandatory that the PI's provide the new participants in this summer's workshop with a statement of what they believe constitutes effective student-centered teaching as well as making it clear that participants are expected to work toward that goal rather than to simply change their courses in any way that decreases the time spent in standard lecture mode. STEMTEC participants receive significant benefits and they can reasonably be expected to make significant commitments to truly changing their courses. In order to do this they have to have a clear sense of what is expected of them.
… From our conversations with the PI's and other members of the project it is apparent that they envision that a complex dialogue among expert teachers, expert educators, and expert scientists/mathematicians will lead to an innovative program to stimulate student-active and inquiry-based learning that will, in turn, lead to the recruitment and retention of K-12 science and math teachers and also improve the college-level teaching of these courses. However, the NVC has seen little evidence that evaluation tools have been, or are being developed, that actually test the effectiveness of its methods. To cite several examples: what is effective student active learning, what are effective strategies in achieving it, and what are the indicators that these are being achieved by individual teachers or by the program as a whole?
… The NVC notes that STEMTEC participants sign an agreement to have their courses evaluated. Nevertheless,
only limited evaluation occurred last semester with some participants opting out when it came time to be evaluated.
The NVC therefore wants to emphasize that it considers that the evaluation component is mandatory for all
participants, as stipulated in their signed agreements, and that it is the responsibility of the PI's to see that this
takes place. Each course should have an evaluation that gives the professor feedback on the progress that he or
she has made in achieving the goals of a STEMTEC course and on what still needs to be done….