ASSESSING THE IMPACT OF TECHNOLOGY ON TEACHING AND LEARNING:
A CAMPUS-WIDE APPROACH

Lucinda Lea, Director, Office of Information Technology
Sarah Barlow, Professor of Biology
Sylvia Brace, Manager of Instructional Technologies
Middle Tennessee State University
Murfreesboro, TN 37132
(615) 898-2512 

Gina Roberts, Web Instructional Technology Specialist
University of Tennessee Knoxville
Knoxville, TN 37996

INTRODUCTION 

As an emerging major institution of higher education in the state, region, and nation, Middle Tennessee State University (MTSU) is committed to helping its students become educated men and women. Through five colleges and thirty-five departments, MTSU offers instruction in the basic and applied sciences, business, education, liberal arts, and communication. Even though the primary focus is on undergraduate education, Masters degrees in nine areas, the Specialist in Education, and the Doctor of Arts degree are also offered. The diverse student body of 18,000 consists of full- and part-time students, both traditional college-age and adult learners. The administration, along with 700-plus faculty members, is dedicated to providing MTSU students with a positive educational experience in a supportive campus environment.

From the decision in 1989 to provide every full-time faculty member with a microcomputer to the opening in 1997 of a new academic building equipped with forty-seven technology-based classrooms, the administration has established the support of instructional technology as a crucial part of the higher education equation. Since 1994, MTSU faculty have enjoyed a comprehensive level of instructional technology support featuring access to technology that includes a campus-wide network, sixty-one technology-based classrooms, more than sixty computer labs, and two multimedia development centers; training in the use of hardware and software; consultation with instructional technology staff; release time to develop technology-based instructional materials; and recognition of their instructional technology achievements.

The faculty at MTSU who use technology as a tool to enhance instruction now represent the mainstream, not just the early adopters. More than two hundred faculty have taught in a technology-based classroom, seventy-two have received release-time grants, and more than three hundred have attended on-campus workshops, conferences, and institutes related to instructional technology. Therefore, in terms of sample size and longevity, a study to evaluate and assess the effectiveness of using educational technologies should begin to provide relevant data.

A study of MTSU faculty was conducted during the spring 1998 semester to assess the status and effectiveness of instructional technology. Thirty-five percent of all MTSU full-time faculty participated in the study. Results have provided valuable information that will help determine measures for improving educational technology resources and services for MTSU faculty and students.

THE STUDY

When preparing for this study, we were confronted with the question, "How do we measure the effectiveness of instructional technology?" While we were tempted to compare it to traditional teaching, we realized our attempts would be futile as traditional teaching comprises many dynamic, immeasurable, and complex components. We decided that instructional technology at MTSU could best be evaluated by measuring its impact on the depth and breadth of content covered, student performance, and good teaching practices that were widely acknowledged as catalysts for improved learning. The good teaching practices used in the survey were adapted from the "seven principles of good practice in undergraduate education." (Chickering & Gamson, 1987) These practices include student interaction with instructor, student collaboration, student participation and feedback, and high expectation of student performance.

While technology will not transform poor teaching practices, can its effective use in the classroom benefit good teaching practices? Is technology being applied in MTSU classrooms in ways that facilitate these good practices? Does the use of technology in the various disciplines strengthen the foundation of quality education? The purpose of this study is to answer these questions and provide information that will assist with future instructional technology planning.

THE QUESTIONNAIRE

A four-page, four-part questionnaire was the survey instrument used in the study. The questionnaire was first tested by a sample group of faculty and deemed a reliable test instrument and was then mailed to all full-time MTSU faculty. 

The questionnaire begins by giving faculty an opportunity to express some general feelings and beliefs regarding instructional technology at MTSU. The first section summarizes the wide variety of instructional technology applications and resources available at MTSU and encourages faculty to respond based on their general impression or feeling. In this section of the questionnaire, faculty rate how much they agree with a variety of statements that measure their perceptions about various types of instructional technologies and their value in the classroom. Other questions ask faculty to rate their level of skill and knowledge using and developing the various types of instructional technologies. This section also gauges faculty opinions and attitudes about office computers, technical support, multimedia development centers, technology-based classrooms, instructional technology workshops, faculty development grants and awards, and instructional technology events.

The second part of the survey is the heart of the study. It measures the frequency of use of various instructional technology applications by MTSU faculty. The instructional technology applications include using computer applications to present lecture outlines or demonstrate specific concepts; using audio/visual equipment to display materials; communicating electronically with students; and assigning students supplementary use or development of materials such as Web pages, computer-assisted instruction modules, and computer-based applications. In this section, faculty evaluate the effect of these applications on the depth and breadth of content covered, student performance, and good teaching practices including interaction with students, student collaboration, student participation and feedback, and faculty expectation of student performance.

The third segment of the questionnaire gathers information about the projected future use and development of instructional technology by MTSU faculty. Faculty rate anticipated use of various types of instructional technologies within the next eighteen to twenty-four months.

The final portion of the questionnaire gathers basic demographic information such as years in higher education, years at MTSU, professional rank, tenure status, department, and discipline. In this section, an open-ended statement allows faculty to discuss why they feel instructional technology is or is not important. Another open-ended statement provides faculty with the opportunity to make additional comments.

SURVEY RESULTS AND MAJOR FINDINGS

The survey results lead us to several major findings. These findings will help MTSU faculty and administrators to better understand the state of instructional technology on the campus and to identify the resources that are needed to ensure its future.

1. Faculty Believe that Instructional Technology Is Essential

An overwhelming majority (97%) of the responding faculty agree with the statement, "I believe that the use of technology in the classroom can enhance student learning." With the majority of respondents giving comments, it is not difficult to understand why MTSU faculty value the use of technology in the classroom. Reasons vary from those as simple as "it gives variety," "it facilitates learning," and "it improves communication," to those as detailed as this one:

 "It has the potential for moving the old traditional education process from a teacher-focused inflexible structure to a highly flexible and integrated 'learning environment.' Without the complete integration of instructional technology into the education process this environment will not be achievable and what we call 'educational' institutions today will lose their current dominant position as providers of educational services. To me it is exciting but also it is a matter of institutional viability."

Many of the faculty responding to the survey note that instructional technologies will be required features of future teaching and learning environments. Comments relating this include the following:

"Instructional technology is necessary for education and to be educated in the 21st century. We [faculty] no longer have a choice. We cannot ignore the advancing use of technology in the classroom."

These faculty recognize that instructional technology is valued and will help to better prepare students for the workplace and for a life of continued learning.

2. Faculty Have a Variety of Needs Relating to Instructional Technology

While most of the responding faculty appreciate the value of instructional technology in the teaching and learning process, the survey data show that a number of factors can either promote or inhibit their ability to develop and use instructional technology. When faculty were asked if they had sufficient time to develop and adapt course materials for the use of instructional technology, seventy-one percent said they do not have time, indicating that this is the largest factor inhibiting the faculty member's ability to develop such materials. Comments include the following:

"Finding time to upgrade technology skills is difficult especially with so many changes, but it is necessary."

"It is very time consuming. I am not sure that the payback justifies the effort at times."

Another factor is the effect that the use of instructional technology has on tenure and promotion. Only fifty-four percent of the respondents believe such use positively affects the tenure and promotion process, even though the University encourages such creative activity.

The need for adequate office equipment for development of instructional materials clearly exists across the campus, but it is apparent that inequity with regard to equipment does exist among colleges. Thirty percent of the respondents felt that their office computer was inadequate for supporting the development and use of instructional technology applications.

Eighty-nine percent of respondents believe it is important that the University provide technology-based classrooms. It is strongly held that "more classrooms with computer-teaching potential are needed." There is frustration on the part of some faculty that there is no access to technology in the classroom in which they teach.

The fifth important factor is training, particularly Web-based training. Eighty-three percent of the respondents believe that the use of Web-based instructional materials enhances student learning; however, only thirty-five percent believe they have the skills and knowledge required for developing and using a Web-based learning environment.

These five factors -- time to develop, tenure and promotion rewards, adequate office equipment, technology-based classrooms, and Web-based training -- clearly reflect the primary needs of the faculty related to the development and use of instructional technology.

3. Instructional Technology Is Being Widely Used Across the Campus

The survey results show that faculty from all ranks and colleges are employing instructional technology tools both in and out of the classroom. They are excited about the possibilities for teaching and learning that the use of technology provides as indicated in the following comments:

"It is necessary training for our students! It raises attention levels and enhances the learning environment."

"It provides an interactive method for facilitating learning. It provides an improved method for student/faculty and student/student communication."

"It is in the real world; it enriches the experience of theoretical modeling; enables access to much more information/applications."

Survey results show there is no significant difference based on rank or tenure in the faculty who responded to the survey. There is a significant difference, however, in the percentage of response from the various colleges or areas, with only twenty percent of liberal arts faculty responding and sixty-three percent of the library faculty. The average response across colleges and areas is thirty-five percent.

4. Different Instructional Technologies Accommodate Different Teaching Practices

The primary purpose of this study was to see if technology plays an important role in the implementation of good teaching practices such as faculty/student interaction, student/student interaction, student participation and feedback, and high expectations of student performance. Survey results indicate that technology does indeed have an important role in improving the conditions for good learning in MTSU classrooms. Data also show that MTSU faculty have found that matching instructional technologies with the appropriate educational goals improves student learning.

Eighty-seven percent of the respondents indicate that the use of electronic communication such as email, Web-based bulletin boards, discussion groups, and listservs positively affects faculty/student interaction. While the majority of the responding faculty felt electronic communication also improved student interaction with other students, the survey data show that encouraging student use of instructional technology to complete assignments was the instructional technology application that best serves collaborative learning.

Eighty-one percent of the respondents say that the use of audio/visual equipment such as VCRs, visual presenters, and laserdisc players in the classroom benefits student participation and feedback. Most also agree that the use of audio/visual equipment allows them to cover more material and delve more deeply into instructional materials.

Results reveal that faculty underestimate the positive effect of technology on student performance. With each of the technology applications considered in the questionnaire, faculty report a positive effect on expectation of student performance (43% - 80%) and on actual student performance (56% - 87%), and in each case performance exceeded expectation.

Faculty agree that instructional technology is an important tool in accommodating a diversity of learning styles and in preparing students for life-long learning and citizenship. One respondent comments, "Technology gives the students an independent channel to obtain information and meets individual learning style differences."

Some of the survey comments reveal that technology makes the most difference when it creates changes or updates in instructional objectives. One professor comments, "It has forced me to rethink and reevaluate my instructional approach and the instructional material."

5. Faculty Instructional Technology Use Will Continue to Increase

The use of technology in MTSU classrooms, with the exception of audio/visual equipment, was essentially non-existent before 1994. At that time the use of technology as a teaching and learning tool began to be encouraged and supported on a University-wide basis. In the subsequent three and a half years, at least one-third of the faculty have been involved at some level in technology-based instruction.

The survey data reveal that the use of instructional technology will continue to grow perhaps more rapidly than before. More than eighty percent of those responding say that in the next two years they will encourage students to communicate electronically through email, Web bulletin boards, listservs, discussion groups, and news groups; they will encourage students to use supplementary materials such as Web pages, computer-assisted instruction modules, or other computer-based applications outside of class; and students will be encouraged to complete assignments, create presentations, or develop Web pages. The faculty overwhelmingly indicate they will use computer applications to present lecture materials and demonstrate specific concepts.

CONCLUSION

Results of the survey reflect faculty enthusiasm for the use of instructional technology and the belief of the faculty that its effective use in the classroom benefits good teaching practices; however, the study data reveal that from the faculty perspective lack of time and resources limits the development and implementation of technology in instruction.

MTSU's commitment to technology-based instruction must continue. The University has to develop creative and innovative means of providing financial resources for instructional technology. Creativity, innovation, and a keen eye for the future will serve educators well in planning and making decisions for the educational frontier that lies ahead.

REFERENCES

Chickering, Arthur and Zelda Gamson (1987) "Seven Principles of Good Practice in Undergraduate Education," AAHE Bulletin (March).

| Main | Faculty | Student | Instruments | Results | Comments | Papers | Slides |