Spring 1999 STEMTEC Evaluation Plan

This draft of the Spring 1999 Evaluation Plan begins with a rationale for the approach we propose, then turns to the evaluation questions, data collection methods, and responsibilities for instrumentation development, administration, and analysis of surveys and observations. The evaluation will measure the degree to which STEMTEC faculty have implemented pedagogical practices in ways consistent with recommendations of the national standards organizations (1) and STEMTEC students= progress toward developing understanding of science and mathematics and recommended pedagogical practices.

Rationale for the Spring 1999 Evaluation Plan

-- Models for Teacher Preparation

National Science Foundation (NSF) Collaboratives for Teacher Preparation have as their goal to improve the preparation of science and mathematics teachers and to increase the numbers of science and mathematics teachers from populations underrepresented in the science and mathematics teaching force. Well-prepared is taken to mean teachers with a deep understanding of science and mathematics content and with the ability to develop the level of understanding called for in national standards in their students (through their use of pedagogies recommended in the national standards documents).

Achieving this goal requires that individuals intending to be teachers have the opportunity to develop the understanding of science and mathematics that the national standards prescribe for teachers and the ability to used pedagogies recommended for developing the understanding of science and mathematics in K-12 students called for in the national standards.

In conventional teacher preparation programs (Figure 1), the requisite knowledge and abilities are developed in course work and practicuums provided by schools of arts and sciences and schools or departments of education. In the STEMTEC model (Figure 2), responsibility for developing the knowledge and abilities falls to faculty teaching science and mathematics courses in colleges of arts and science. These are faculty who are themselves only just developing the understanding and abilities required to teach science and mathematics using the recommended pedagogies.(2) Figure 3 presents, in a tabular fashion, a comparison between the two approaches.

In addition to responsibility for developing content knowledge, STEMTEC science and mathematics faculty have responsibilities for recruitment of students into teaching and providing opportunities for students to practice recommended pedagogies with their fellow students. Furthermore, science and mathematics faculty are expected to make contacts with science and mathematics teachers in the geographic area to provide sites in which students can work with K-12 science and mathematics students.

 

 

Figure 1: Conventional Approach to Teacher Preparation

 

 

 

 

 

Figure 2: STEMTEC Approach to Teacher Preparation

  

 

Comparison of Conventional and STEMTEC Teacher-Preparation Approaches

 

Conventional

STEMTEC

Science and mathematics content knowledge development

In science and math courses presented by college and university faculty

In science and math courses presented by college and university faculty

Science and mathematics pedagogical content knowledge development

In education and social science courses

Exclusively in science and math courses present by college and university faculty

Recruitment and retention

Within standard recruitment efforts

College and University science and math faculty; STEMTEC administrators

Practical experience in K12 classrooms

Responsibility of Education Department faculty often in cooperation with K12 teachers

Exclusively in science and math courses presented by such college and university faculty

Induction (the first year of teaching)

Responsibility of K12 schools

Aspects of this support offered through STEMTEC administrators

 

Figure 3: Comparison Between Conventional and STEMTEC Teacher Preparation

 

Evaluation Plan - Spring 1999

Consistent with our analysis of the STEMTEC approach to teacher preparation, as illustrated in Figure 2, the Spring 1999 evaluation plan focuses on the development of content and pedagogy understanding of students enrolled in STEMTEC courses offered in the spring and on STEMTEC faculty classroom practices in their spring 1999 courses. (The information below refers to the Spring 1999 semester.)

-- STEMTEC Students

Survey data in four areas will be collected on all students in STEMTEC courses taught in the spring of 1999. The same data will be collected on all students who are receiving STEMTEC Teaching Scholarships. Data on these two groups of students will be compared to measure the effects of scholarship support on students= progress toward meeting STEMTEC objectives

Research Questions and Data Collection

1. Recruitment and Retention

-- What consideration had individual students in STEMTEC courses given to teaching as a career when they entered post-secondary education?

-- How did their experiences in STEMTEC courses and activities reinforce or modify their positions on teaching as a career?

-- What responsibility for recruitment and retention is being taken by STEMTEC faculty?

Are STEMTEC faculty encouraging their best science and mathematics students to consider teaching as a career?

What strategies are STEMTEC faculty employing to get students from under represented populations (in mathematics and science) to consider teaching as a career?

Data Collection

-- Information from one component of the survey of all students in STEMTEC courses taught in Spring 1999

-- Information from one component of the survey of all STEMTEC Faculty

-- Information from one component of the observations of a sample of STEMTEC Faculty

2. Science and Mathematics Content Knowledge Development

-- To what degree have students in STEMTEC Courses developed understanding of science and mathematics called for in national documents?

-- Is the level of understanding correlated with the number of STEMTEC courses the student has experienced?

Data Collection

-- One component of the survey of all students in STEMTEC courses taught in Spring 1999.

3. Science and Mathematics Pedagogical Content Knowledge Development

-- How often have students in STEMTEC Courses experienced pedagogical practices recommended by the national standards?

-- How do students characterize these practices, that is, how do students describe the pedagogical activity?

-- Do students find these practices successful in helping them develop their understanding of science and mathematics?

-- Do students believe that these methods are applicable in other disciplines?

Data Collection

-- Information from one component of the survey of all students in STEMTEC courses taught in Spring 1999.

4. Practical Experience in Post Secondary or K-12 Classrooms

-- How often have students in STEMTEC Courses been involved in teaching as a part of a STEMTEC Course?

-- What was the nature of that teaching experience?

-- Who did the students teach?

-- What did they teach?

-- How often have students in STEMTEC Courses been in K-12 classrooms as part of a STEMTEC Course? What did they do in the classroomB observe, tutor?

Data Collection

-- Information from one component of the survey of all students in STEMTEC courses taught in Spring 1999

-- Information from one component of the survey of all STEMTEC Faculty

-- Science and Mathematics Faculty

Because a basic premise behind the STEMTEC Model of Teacher Preparation is that teachers teach as they were taught, the congruence of the content taught in STEMTEC courses and the congruence of faculty members= pedagogical practice with national recommendations are essential questions for the evaluation of the STEMTEC Project. A detailed faculty-survey should also be conducted in Spring 1999 that addresses the recommended pedagogies and that is modified based upon feedback from the 6 February survey.

-- Is the content taught in STEMTEC Courses congruent with national standards?

This question will be answered through an analysis of tests and any other methods used by STEMTEC faculty to grade their students.

-- Do STEMTEC faculty practice pedagogy as recommended in national standards?

This question will be answered through the analysis of observations and interviews of STEMTEC Faculty as they use recommended pedagogies in their classrooms.

Data Collection

-- Information from the faculty surveys

-- Information from syllabi and faculty's student-evaluation instruments

-- Observations

Sample Selection

Using responses from the survey administered at the 6February meeting, faculty members who report they are using recommended practices and are reasonably confident in the use will be identified. The sample observed should include faculty members using all pedagogies sampled on the survey and at least one faculty member from each institution will be observed. The ratio of science to mathematics faculty in this sample should be about 5 to 1 (the approximate ratio of STEMTEC science to STEMTEC math courses.) Each faculty member should be observed at least twice during the course of the semester.

Using data from the observations, faculty who are using recommended pedagogies as defined in national standards, will be interviewed to learn how they developed understanding of the pedagogies and their evaluation of them.

-- Tasks

-- Develop student survey; conduct student survey

-- Collect Spring 1999 Course Syllabi

-- Collect Student-Evaluation Instruments and Methods used by Faculty, that is, the tests and information that indicate how faculty are assessing their students

-- Collect a Complete List of STEMTEC courses that have been offered assembled by Institution

-- Summarize data from the 6 February survey:

to identify faculty using recommended pedagogies with some confidence

to identify how faculty describe recommended pedagogies

-- Develop observation instruments; conduct observations

-- Develop interview protocol; conduct interviews

Endnotes

(1) The National Science Education Standards, the National Council for Teachers of Mathematics, and the National Board of Professional Teaching Standards (NBPTS) are the national standards organizations referenced within.

(2) An implicit assumption within the STEMTEC approach to teacher preparation is that college learners, after exposure to recommended pedagogies by higher education teachers (the science and math faculty), will then be able to implement the recommended pedagogies in kindergarten through high school classroom.