4. Results and Discussion

The curriculum was presented to the subjects in two formats, over three cycles as proposed. The research instruments were used to gather data. The data was analyzed to address the four research questions as follows.

Research Question I.

Do adult learners using computer presentation acquire knowledge as effectively as learners receiving the same curriculum in a traditional classroom lecture format?

Results. Adult learners using CAI were found to acquire knowledge as effectively as users in a conventional classroom for those topics which involve learning specific, discrete facts. Learners using CAI did not learn as effectively for topics which are theoretical, require visualization, or hands-on skills.

Discussion. The knowledge gained with each instructional format was measured by administering an assessment tool as a pretest and posttest for each topic to each group, then comparing the scores. The scores indicate learning gains occurred for all six topics in each instructional format. For each topic, the gain in the classroom format met or exceeded the gain in the CAI format. Table 2 summarizes the test results (Appendix D, E).

Table 2
Pretest and Posttest Mean Scores for Classroom and CAI Format Instruction (n=29)

	Classroom Pretest	Classroom Posttest	Classroom Gain	CAI Pretest	CAI Posttest	CAI Gain
Terms and Concepts	49	86	37	60	83	23
Components	63	87	24	62	86	24
Wiring Diagrams	65	85	20	63	81	18
Safety	68	86	18	67	87	10
Measurements	58	85	27	61	79	18
Electro Static Discharge	45	95	50	41	85	44

The pretest provided a measure of knowledge prior to instruction. The mean pretest scores of the two formats from Table 2, when combined and arranged from highest to lowest, showed Safety having the highest overall existing knowledge (67.5), followed by Wiring Diagrams (64), Components (62.5), Measurements (59.5), Terms and Concepts (54.5), and Electro-Static Discharge (ESD) (43). The posttest scores were compared with the pretest scores to determine knowledge gain per topic. The mean gains of the two formats from Table 2, when combined and arranged from highest to lowest, showed an inverse relationship to the pretest scores. ESD had the highest gain (47), followed by Terms and Concepts (30), Components (24), Measurements (22.5), Wiring Diagrams (19), and Safety (14). For each topic, the classroom format gain met or exceeded the CAI format gain. The classroom format showed the greatest advantage in Terms and Concepts (gain difference 14). A smaller advantage was seen in Measurements (9), Safety (8), and ESD (6). Little or no classroom advantage was seen in Wiring Diagrams (2) and Components (0).

The pretest scores reflected the varying experience levels of the participants prior to instruction. Safety is a high priority in this company, and the company has an active safety program and regular training on related safety topics. The high profile of the safety program is reflected in the high pretest scores, indicating a high level of existing knowledge of this topic.

Wiring Diagrams and Components are both familiar topics to the learners, as they have hands-on experience in understanding components and interpreting wiring diagrams in their normal work day as electrical assemblers. The learners tend to gain knowledge of these two topics through normal work activities and informal conversations with their peers and supervisors in addition to formal training.

Pretest scores for Measurements were lower, reflecting that use of instruments and measurements is not part of the learner's normal work responsibilities. These tasks are usually performed by technicians, so the learner's familiarity with these tools is lower.

The Terms and Concepts topic is all theoretical, including definitions of units of measurement and use of formulae to determine unknown variables. The lower pretest scores reflect that the learners do not require this knowledge to perform their jobs on a daily basis.

ESD concerns the potential hazard static electricity poses to integrated circuits, and prescribes specific work practices to minimize this hazard. The low pretest scores reflect that a). the company had not provided this training in several years, so most of the learners had never received it, b). ESD is a relatively new topic to industry, and c). ESD was not recognized as a hazard to electrical components until the proliferation of the microprocessor in the late 1980's, making the likelihood lower that the learners may have received instruction in this topic at previous employers.

The inverse relationship between the pretest scores and gains indicates the instruction provided the greatest gains in areas where existing knowledge was low. This had the effect of bringing all learners up to a certain level of mastery of each topic.

The highest classroom advantage over CAI was seen in Terms and Concepts. This topic is greatly dependent on understanding and manipulating formulae. The classroom format offered a greater potential for interaction between instructor and learners, and opportunities to teach abstract concepts requiring visualization. The CAI format was limited to showing and explaining fixed examples, and did not have the technical sophistication to allow learners to experiment with various scenarios. Cuoco and Goldenberg (1996) proposed CAI for mathematics include the capability for the learner to manipulate formulae within the lesson, so they might "tinker with mathematical objects, just as they might tinker with mechanical objects".

Classroom instruction showed little or no advantage over CAI in the topics of Wiring Diagrams and Components. Wiring Diagrams is a symbol-based instruction, and CAI is particularly well suited to presenting symbols and allowing different methods of display and comparison. CAI also is well suited for Components instruction, as it has the ability to easily show cutaway views and animations. The CAI instruction included two animations showing how fluid level switches operate. The classroom version was limited to a static illustration, and the instructor had to explain the operation. Several of the CAI learners commented on the animation's ability to allow them to understand the operation of the devices.

The implications for using CAI to maximize knowledge gain would include taking advantage of the medium's ability to allow formula experimentation when teaching theoretical and abstract topics such as mathematics. This aspect would strengthen a CAI version of electrical Terms and Concepts as used in this study. Another implication is to include animation whenever possible to allow visualization of complex or unseen processes.

Research Question II.

Do adult learners using computer presentation achieve satisfaction with the delivery method as much as learners attending a classroom lecture?

Results. Adult learners using CAI were found to achieve higher satisfaction than classroom learners in the speed of instruction. CAI learners were found to have equivalent satisfaction with classroom learners in knowledge gain. CAI learners were found to have lower satisfaction than classroom learners in their belief of appropriateness of instruction format, job application, time and place of instruction, and instructor access.

Discussion. Satisfaction with the learning method was measured by a survey (Appendix C) completed by the learners following the posttest. Learners were allowed to keep the survey for several days to record their opinions. All surveys were returned, however not all learners responded to all questions.

The survey consisted of several questions for the classroom group only, several for the CAI group only; and the remaining questions were for all learners. Most of the questions were provided to measure learner satisfaction. A few additional questions were included at the request of the company to determine the learner's access to technology at home, and preferences for potential training topics and formats. These additional questions and responses were not part of this study. Table 3 summarizes the satisfaction survey results (Appendix F, G).

Table 3
Persons Indicating Satisfaction Level for Classroom (CR) and CAI Format Instruction

Question	Group	SD	D	N	A	SA	NA
2, 5	CR	1	2	3	15	8	0
2, 5	CAI	1	2	4	11	5	0
10	CR	0	0	0	19	8	0
10	CAI	0	1	0	16	8	0
11	CR	0	1	1	17	9	0
11	CAI	0	0	2	17	6	1
12	CR	0	0	1	18	8	1
12	CAI	1	2	1	8	9	1
13	CR	0	1	3	21	3	0
13	CAI	1	3	1	11	8	0
14	CR	0	3	2	8	7	7
14	CAI	0	2	0	8	15	0

Note. Questions 2 and 5 are identical except for wording identifying which group is answering the question. SD = strongly disagree, D = disagree, N = neutral, A = agree, SA = strongly agree, NA = not applicable.

---

Not all survey respondents answered all the survey questions. In the following discussion, all percentages refer to the percentage of people responding to that question.

In questions 2 and 5, the topic was the learner's belief that the format they used was appropriate for the topic. 79% of the classroom learners agreed or strongly agreed the format they used was appropriate, compared to 70% of the CAI learners. Question 8 surveyed learners about the speed of instruction. 93% of the classroom learners responded the instruction was given at an adequate rate. The remainder stated the speed was too fast. No one stated the classroom format was too slow. 96% of the CAI learners were satisfied with the rate. No one stated the CAI was too slow.

Learners were asked in question 10 about their satisfaction with their knowledge gain. 70% of the classroom learners agreed they increased their knowledge, 30% strongly agreed. 64% of the CAI learners agreed they increased their knowledge. 32% strongly agreed. Question 11 measured satisfaction with the pertinence of instruction to job requirements. 93% of the classroom learners agreed or strongly agreed the instruction would have direct effect on their job performance, while 88% of the CAI learners agreed or strongly agreed. Satisfaction with access to the instructor was clearly higher for the classroom learners. In question 12, 93% of these learners agreed or strongly agreed with the level of access, and no one expressed dissatisfaction. 77% of the CAI learners expressed satisfaction, with 14% expressing dissatisfaction.

Question 13 surveyed learners on satisfaction with the time and place of instruction. The intent of this question was to measure satisfaction with the setting - fixed time for classroom vs. flexible time for CAI, and group classroom environment vs. individual CAI sessions. 86% of the classroom learners expressed satisfaction with time and place, 4% expressed dissatisfaction. 77% of the CAI learners expressed satisfaction, 14% expressed dissatisfaction.

All learners were asked in question 14 to rate their satisfaction level with their ability to use a personal computer independently. When asked to respond to the statement "I feel comfortable using a personal computer on my own," 73% of the learners agreed or strongly agreed.

Learners expressed higher satisfaction with CAI's self-pacing ability. In the classroom sessions, the classes tended to proceed at the rate of the slowest learner, and due to time restrictions, some learners were inevitably left behind at the conclusion of instruction. The knowledge gain was found to be independent of instruction format, which suggests that the criteria used to select classroom or CAI for a given topic may be more influenced by other variables.

CAI learners expressed lower satisfaction with instructor access. In this study, the instructor was not visibly present with the CAI learners. Satisfaction may be increased if the instructor access is improved beyond telephone contact.

CAI learners were not as satisfied with time and place of instruction. While every effort was made to minimize distraction, the increased isolation of the learner may have been a contributing factor.

Implications for industry seeking to maximize satisfaction with CAI include taking advantage of CAI's self pacing to maximize potential for fuller knowledge, placing a high priority on instructor access, and providing a place for instruction that is welcoming, yet free of distraction and isolation.

Research Question III.

How does the cost of computer presentation delivery compare to classroom lecture in terms of learner and instructor time?

Results. The overall cost of CAI in this study, in terms of learner time, was found to be about 15% higher than the classroom instruction. However, when examined by topic, the cost was found to be higher for theoretical topics, and lower for fact-based topics.

Discussion. Learner time is that consumed by each learner to receive the instruction. This was documented in order to indirectly measure the cost of instruction, as this cost was directly proportional to the time required. There were no other costs incurred such as handouts, books, and supplies. Learner time was documented for each training session in each format. Table 4 summarizes the number of learners having elapsed instruction times in certain time categories (Appendix H, I).

Table 4
Number of Learners to Time Required for Classroom (CR) and CAI Format Instruction

Time in Minutes	Session 1 CR	Session 1 CAI	Session 2 CR	Session 2 CAI	Session 3 CR	Session 3 CAI
6-10						1
11-15						2
16-20						2
21-25		1		1		4
26-30				3		5
31-35					9
36-40		1	1		1	1
41-45		5		2		1
46-50	10	3	9	4	8	3
51-55	19	4	12	3	2
56-60		11	7	11	9	4
61-65		11		1
66+		4		4		6

Note. Each training session comprised two training topics:
Session 1 comprised Terms and Concepts and Components.
Session 2 comprised Wiring Diagrams and Safety.
Session 3 comprised Instruments and Measurements and Electro-Static Discharge.

---

The classroom sessions indicate a close distribution, as many learners shared a class having a common duration. Variations do exist within the same session, as each session was offered multiple times in order to accommodate all learners; and each session varied in time. The CAI sessions resulted in widely varying time requirements, as learners were able to use as little or as much time as needed. A summary of time comparisons is shown in Table 5.

Table 5
Learner Time (in minutes) Required for Classroom (CR) and CAI Format Instruction

	Session 1 CR	Session 1 CAI	Session 2 CR	Session 2 CAI	Session 3 CR	Session 3 CAI
Total	1,543	1,744	1,570	1,819	1,389	1,592
Mean	53	60	54	63	48	55
Median	55	60	55	57	48	40

Note. Each training session comprised two training topics:
Session 1 comprised Terms and Concepts and Components.
Session 2 comprised Wiring Diagrams and Safety.
Session 3 comprised Instruments and Measurements and Electro-Static Discharge.

---

The data indicates the total minutes of CAI instruction always exceeded the total minutes of classroom instruction. The company estimates their cost of a learner's time at $25/hour, thus the labor cost for 4,502 minutes of classroom learning was $1,875, the labor cost was 5,155 minutes of CAI was $2,150.

An unexpected finding was that the CAI time data was affected by several CAI learners who required substantially more time than others to complete the instruction. Classroom learners who required more time had no such option. Thus, the comparison of total minutes is skewed in this respect.

The mean (n=29 per format group) is likewise affected by these outliers in the data. The median calculation minimizes the effect of the outliers, and shows that for most learners, the CAI format required slightly more time for sessions 1 and 2, and substantially less time for session 3.

The implications for using learner time effectively for industries implementing CAI include allowing great latitude for learners in time allowed, yet defining a method to detect a point of diminishing returns for those learners who reach a point of using large amounts of time to complete a topic.

Research Question IV.

How does the cost of computer presentation delivery compare to classroom lecture in terms of curriculum development and delivery?

Results. The overall cost of the CAI in this study, in terms of development and delivery, was found to be higher than classroom cost in a 2.8:1 ratio. 96 hours of CAI development was required to create three hours of actual CAI instruction, yielding a development to presentation time ratio of 31:1.

Discussion. The curriculum was prepared by first developing the CAI format instruction. This required 96 hours to convert existing curriculum notes to HTML format, prepare the illustrations, link all the pages, and load on the server. This resulted in an estimated 3 hour CAI program, or a ratio of 31 hours of development per hour of instruction.

Once the CAI instruction was complete, each screen was printed on a separate page, and each illustration converted to an overhead transparency. These were punched and bound to serve as the lecture notes and graphics for the classroom instruction. This method ensured consistency of content, and required three hours to prepare. Table 6 summarizes the time required for development, delivery, and administration of the curriculum.

Table 6
Development and Delivery Time (in hours) Required for Classroom and CAI Format Instruction

	Classroom	CAI
Planning	6	6
Curriculum development	3	96
Proofing and testing	3	6
Facilities and equipment	2	7
Prepare forms	3	3
Schedule learners	4	4
Orientation	5	9
Pretest	5	5
Instruction	18	5
Posttest	5	5

---

The company estimates their cost of the developer's time at $50/hour, thus the classroom development cost was $2,700; the CAI cost was $7,300. However, it should be noted that as the CAI was developed first, the entire curriculum development was charged to CAI, except for the three hours used in printing and transparency preparation.

There were no other significant costs incurred in the study, as all equipment, facilities, and supplies were owned by the company.

One advantage of CAI is that initial development costs are recovered quickly as the number of users increases. The 96 hours of curriculum development was an initial investment and will serve an unlimited number of learners.

A second advantage is instruction time required. The five hours shown for CAI instruction consisted of many brief interactions with the learners to ensure the equipment was operational and they could begin the lesson, after which they could work independently. The 18 hours of classroom instruction shown is the actual time the instructor taught the classroom learners.

One implication for using CAI development and delivery time effectively for industry include choosing CAI for topics which do not have frequent content changes, and can be reused many times; to maximize the return on the development investment. Another implication is that any CAI topic should be developed as fully as possible prior to use, in order to minimize subsequent maintenance time.

Summary

CAI was found overall to be as effective as the classroom for fact-based learning, but not as effective for those topics requiring critical thinking or mathematical problem solving. The users expressed high satisfaction levels in CAI, especially with the self pacing and navigational freedom aspects. The time required by learners to use CAI was slightly higher overall, yet when examined by topic was found to be lower for those topics involving fact-based learning. Development time was found to yield a 31:1 ratio to instruction time. Much higher ratios are used in the popular press, with a 100:1 ratio a common rule of thumb. The CAI used in the study was kept as technically simple as possible, with little emphasis on cosmetic appearance, which may explain this lower ratio. This study yielded these recommendations for industry:

1. Evaluate the type of instruction desired. Fact-based learning is a clear candidate for CAI. Theoretical, abstract learning can effectively use CAI if it capitalizes on its strengths in formula manipulation, experimentation and animation.

2. Provide an appropriate environment. A comfortable area free of distractions and significant access to the instructor will maximize learner satisfaction.

3. Implement a system to identify those learners who may not have a learning style conducive to CAI, and route them into classroom sessions. Monitor progress of all learners to detect those who encounter trouble.

4. Maximize the return on investment by developing and rigorously testing the CAI curriculum prior to release, and then seek to maximize the number of users.

CAI is not an automatic replacement for classroom instruction, but if developed and implemented thoughtfully, it can yield successful results.