Abstract
This article presents results from an evaluation survey among students studying at the NKI Electronic College. The discussion is based on a view that in spite of the fact that dramatic changes in technology for learning are taking place, concepts, theories and didactic practice from distance education should not be overlooked when designing solutions for flexible teaching and learning in the future.
This article presents results from an evaluation survey among students
studying at the NKI Electronic College (Nettskolen). (The complete
report [9] is published as: http://www.nki.no/ekko/for_alle/fagartikler/evaluation/evaluati.htm.)
The Electronic College is mainly a metaphor for the administrative,
technological and didactic structures and courses and programmes
offered via Internet and WWW by NKI Distance Education. This evaluation
study was one of a series of continuous surveys among NKI distance
education students and specifically carried out as an activity
within the EU Leonardo Online Training Project (MMWWWK - Multi
Media World Wide Web Kernel for distance education).
In connection with the developments in information and communication
technologies during the last 10 to 15 years some writers argue
that distance education has gone through a number of generation
shifts from correspondence education to today's systems based
on the new technologies, see e. g. [1], [4], [2]. It is also argued
that these developments wipe out differences between types of
organisations, such as distance teaching institutions and traditional
educational institutions [3], and that in this new
situation the concept and theories of distance education are no
longer relevant [2].
In our view concepts, theories and didactic practice developed
within distance education, will be of value also in the future,
specifically in connection with developing and organising part-time
education adults. Thus, also that a merger of forms not necessarily
is as easy as many tends to believe. We believe that educational
institutions in connection with further and continuing education
for the adult working population, face challenges from target
groups that have very different needs from young people studying
full-time at campus. Theories and experiences from 'traditional
settings' are not easily transferred to another pedagogical reality.
This view has only been strengthened through our work on computer
conferencing, computer mediated communication and teaching on
the Internet during the last 10 years.
The evaluation reported in this article aimed at looking into the views of NKI Internet students on some of these issues, such as freedom and flexibility, individual learning or group learning, presentation of learning material, access to resources on the net and tutor interaction.
NKI is one of the largest non-governmental educational institutions in Norway. NKI comprises The Polytechnic College, NKI Distance Education, The Academy of Commerce and Industry and The NKI Publishing House. A further presentation of NKI is available at URL: http://www.nki.no
NKI Distance Education offers more than 60 distance education programmes and more than 300 courses, mainly in technical and vocational subjects. It has about 15,000 students. An extensive network of local partners around the country is established to offer face-to-face classes to support the distance students. The distance education tutors are part-time employees recruited from business, industry, schools and colleges around the country.
The Electronic College is designed so that distance students can
do all their communication online. In contrast, many of the internationally
renowned implementations of 'electronic colleges' are actually
using computer conferencing as a supplement to correspondence
courses or on-campus teaching. The philosophy of the NKI Electronic
College has always been to offer programmes that are independent
of time and space through a system that facilitates flexible communication
for administrative, social, and teaching purposes.
Since the first courses were delivered in 1987, one may distinguish
between three stages of technological developments. The first
system based on the 'EKKO computer conferencing' system
developed at NKI, was in operation from 1987 to 1993. The second,
which was Internet based, started in January 1994. The first and
second stage experiences are documented in an OECD report [5]
and in several articles available at http://www.nki.no/ekko/for_alle/fagartikler/.
This paper focuses on the third stage system that was introduced
with the first web-based courses in 1996.
Table 1 gives an overview of courses and programmes included in
the survey.
The Information Network Programme is designed specially
for distance education on the Internet. The text books are in
print form, but the study guides are only available as web-material.
E-mail for one-to-one communication and Listserv for many-to-many
communication are the two main channels for communication. All
courses have individual start-up times and unpaced progression.
Exams are offered twice a year for each of the courses.
The Information Technology Programme is available as evening
classes, correspondence courses, and courses on the Internet.
Both textbooks and study guides are in print form. The courses
have no course home pages, but the students have access to the
college home page and e-mail for one-to-one communication and
Listserv for many-to-many communication. All courses follow a
semester plan with start in September and February and have paced
progression. Exams are offered twice a year for each of the courses.
The Information Technology for Teachers Programme is designed
specially for distance education on the Internet. The text books
are in print form, but the study guides and some of the readings
are only available as web-material. E-mail for one-to-one communication
and Listserv for many-to-many communication are the two main channels
for communication. A pilot group of 21 students working as primary
and secondary school teachers in the Oslo region took part in
the programme. The courses followed a fixed semester plan, had
paced progression and two scheduled face-to-face meetings. Internet
access and support was provided by the Oslo School Authority and
the pilot project was funded by the Ministry of Education.
Table 1. Programmes and courses offered during
the survey period.
| Programmes and courses |
| ||
| Information Network Programme | |||
| Internet | |||
| Data communication | |||
| Network administration | |||
| Multimedia network | |||
| Information Technology Programme | |||
| Introduction to information technology | |||
| Project management and development of
information systems | |||
| Introduction to programming | |||
| Systems analysis | |||
| Object oriented programming | |||
| Project economy | |||
| Databases | |||
| Management of local area networks | |||
| Strategic use of information technology | |||
| Project work | |||
| Information Technology for Teachers Programme | |||
| Internet for teachers | |||
| Information technology | |||
| Courses not included in a programme | |||
| Java-programming | |||
| Tutor in distance education |
From experiences during the last ten years, we have changed views
and strategies concerning some important aspects of the delivery
of courses on the net, e. g. based on some expectations concerning
student backgrounds, access to equipment and answers to questions
concerning attitudes and preferences of how, when and where to
learn. Our strategies for future developments have partly been
based on subjective experiences of personnel with varying responsibilities
within the organisation; such as developers, academics, tutors,
counsellors, IT support staff and administration, as well as on
formal evaluation studies, discussions with other institutions,
students and tutors. Thus, the study is an attempt to cover some
issues which we felt needed more coherent and updated analysis
before completing the structure and design of the WWW kernel for
distance learning in co-operation with our EU partners:
The questionnaire was distributed to students enrolled in one or more courses offered via the Internet during one full year. Because of a fault in the selection programme, students who had completed or cancelled their studies were excluded. After one reminder we received 64 usable questionnaires back (43 percent response rate).
This article concentrates on results of specific importance for our internal discussions on didactics of Internet teaching.
Reasons for taking the course are mainly related to present job needs, increase possibilities for changing job and increasing formal competence. Technology based (Internet based) studies could be seen as attractive in itself, especially for students choosing information and communication related subjects. None of the students said that interest in this specific form of teaching was the main motive. A large majority of the students had access to computer and Internet both at home and at work and also had an e-mail address before enrolling. It is also worth noting that nearly 70 percent of the students report that the fact that the courses were offered on the net, had some or great significance for their decision to enrol. To see whether Internet studies really attracts new student groups, we asked if the students would consider taking the course as full-time, part-time or correspondence study. (Some of the NKI courses and programmes are, in fact, offered via "the full range" of study forms.) The answers demonstrated very clearly that full-time study was not at all probable (3% on the positive side), part-time study ticked positively by 23% while correspondence study was considered probable by 41%. The answers illustrate that studying via Internet attracts new groups, but also that many could have considered enrolling for traditional correspondence study.
The technological developments have created and array of different
ways of organising teaching and learning. At one extreme of a
continuum of different forms of studies presented as distance,
decentralised, flexible or open learning, one might find a form
which we categorise as 'individual flexible distance learning'
which allows the individual student complete freedom as to when
to start, where to study and which pace to follow. At the other
extreme we find the model of 'the extended classroom'
which requires the students to study as a group, often also that
they meet regularly at a certain place at certain times. Studies
offered on the Internet may be organised as either form or anywhere
on this continuum. From our first pilot trials of offering courses
based on computer mediated communication, we saw that these new
technologies represented a qualitatively different development
from most other media solutions presented during the last 25 to
30 years concerning the possibilities to support open and flexible
learning of high quality.
In the beginning our view was that efficient information technology
based teaching including inter group support, academic and social
communication presupposed fixed start and progression schedules.
This lead to 'electronic' teaching and learning becoming much
less flexible than distance teaching based on traditional media.
Since then we have changed opinion and NKI has changed strategies
to meet what we experience as the needs of the market.
The value of this strategy is clearly demonstrated in the students' answers. Nearly 90% of the respondents look upon 'free starting time' as an advantage. The preference for 'free progression plan' is nearly as great with 80% ticking on the positive side. This is probably due to the fact that some students actually experience that they have a need for a push from the school or some strict requirements to be able to give priority to their studies. E.g. in an early survey some students answered "free pace of study" for both 'the largest advantage' and 'the largest disadvantage' of correspondence study [7]. In an earlier study one student wrote: "If electronic distance studies had been more flexible, it would have been a superb choice". And our conclusion on this point was: "...it is a major challenge to develop methods and organisations in distance education based on computer conferencing systems which take care of the distance student's need for autonomy and flexibility." [8, p. 92]
Table 2 gives an overview of different didactic elements and the
students' attitudes towards their value for their learning as
measured by the question: "The teaching contains different
elements. Try to assess these concerning value for your learning"
Interpretation of the information presented in Table 2 is not necessarily easy - as the students answer with reference to their actual experiences in the NKI Internet courses. This means that as NKI has chosen to design the Internet courses, whether based on WWW learning material or not, with great emphasis on flexibility, courses structured into study units with assignments for submission to be commented on by a 'personal' tutor, collaboration and communication with fellow students will receive less attention. Thus, in a way this can be seen as a 'self-fulfilling prophecy'. On the other hand, with some few exceptions there seem to be small differences between students with different experiences, e. g. between students studying programmes which have emphasised conferences and group based progression during the semester.
Table 2. Didactic aspects of the teaching and
their value for learning (Percentages, N=60).
| Very important | 1 | 2 | 3 | 4 | 5 | ||
| Content quality/relevance | 65 | 17 | 8 | 3 | 2 | ||
| Quality of the study guide | 55 | 28 | 15 | - | 2 | ||
| Quality of assignm. for subm. | 37 | 52 | 10 | 2 | - | ||
| Reading/studying | 57 | 33 | 10 | - | - | ||
| Work with assignm. for subm. | 50 | 42 | 8 | - | - | ||
| Tutor's comments | 45 | 28 | 20 | 7 | - | ||
| Academ. comm. with fellow st. | 17 | 17 | 30 | 27 | 10 | ||
| Social comm. with fellow st. | 8 | 7 | 32 | 25 | 28 | ||
| Individual comm. with tutor | 33 | 33 | 23 | 8 | 2 | ||
| Academic conferences | 9 | 20 | 39 | 20 | 12 | ||
| Social conferences | 3 | 12 | 25 | 27 | 32 | ||
| Collaborative learning | 8 | 23 | 28 | 20 | 20 | ||
| Individual flexibility | 55 | 23 | 15 | 5 | 2 |
Generally, we may note that the students emphasise aspects known from individual distance education, such as quality of the study guide, individual reading/studying, work with assignments for submission, tutor's comments, individual communication with their tutor and individual flexibility as most important for their learning, while the specific aspects of computer mediated communication, such as social and academic communication with fellow students and social and academic conferences as well as collaborative learning seem to be regarded as less important. Concerning the first mentioned aspects, it is of course important to realise that these also can be more efficiently organised in distance learning systems applying the Internet and WWW for distribution of material and communication.
The electronic or virtual environment offered to the students
may be used or taken advantage of in different ways. The use may
be passive as when reading or active when writing
to fellow students or tutors or contributing to conferences. Passive
use may help the student in his or her individual learning process,
but does not contribute to the social system of learning, to other
students' learning or to the construction of a social climate.
The students' actual participation in the communication was also
measured by a number of different variables.
It seems very clear that the students make little use of the communication possibilities that are organised for them as part of the Internet/computer mediated communication courses. This fact might either indicate that these possibilities are of relatively little importance, that the actual use satisfies the needs they have for communication for efficiently completing their studies, or that the organisation or content of conferences is unsatisfactory. Writing to the tutor and to the administration and reading in academic conferences are the most frequent ways of using the communication possibilities.
Table 3. Actual use of resources/means for learning
during the study programmes (Percentages, N=60/59).
| Very much | 1 | 2 | 3 | 4 | 5 | ||
| Write to the tutor | 15 | 15 | 20 | 40 | 10 | ||
| Write to fellow students | 2 | 2 | 12 | 37 | 48 | ||
| Write to academ. conf | 2 | 2 | 10 | 27 | 60 | ||
| Write to social conferences | 2 | - | 12 | 15 | 72 | ||
| Write to the administration | - | 8 | 17 | 32 | 43 | ||
| Read in academic conf | 5 | 12 | 13 | 27 | 43 | ||
| Read in social conferences | - | 2 | 10 | 24 | 64 |
Among the courses included in the evaluation there were some with
learning material on the WWW and some applying only e-mail and
conferencing facilities. We asked the students studying courses
that did not include material presented on the WWW, whether they
would see it as an advantage to have the study material presented
on the WWW. On a 5-graded scale from 'absolutely an advantage'
to 'does not matter' half the students ticked the middle
alternative with the rest distributed on both sides.
Students on courses involving WWW material were somewhat more
positive towards presenting the material on the WWW than were
students of Internet courses not including this type of course
presentation. However, the enthusiasm was not very large in any
of the groups.
We have internally discussed whether the students would prefer
receiving the WWW material also in printed form. The answers showed
that the views differed a lot. Different views may have resulted
from different experiences and needs in different courses. This
question should be further examined.
We also asked about opinion on the importance of 'presenting
the material on the WWW', of 'Internal hyper-links'
and of 'hyper-links to external material'. Most students
ticked on the positive side for all three aspects of WWW materials.
It seemed that the use of hyper-links to external resources was
the facility most appreciated by the students.
Table 4. The actual use of WWW facilities. (Percentages,
N=33).
| Very important | 1 | 2 | 3 | 4 | 5 | ||
| Use of internal hyper-links | 24 | 15 | 42 | 9 | 9 | ||
| Use of external hyper-links | 24 | 21 | 24 | 15 | 15 | ||
| Search in external resources | 27 | 18 | 36 | 6 | 12 |
Again, we may note that there are large differences in how much the students have actually used the WWW facilities. Differences in use may, of course, be related to how much of the course/programme the students have actually completed. The number of students was too small to really examine such relationships.
There is hardly any doubt that demands on teachers can be very
large and teacher workloads may become very heavy in courses offered
on the Internet. Cost efficiency of distance study is normally
based on the fact that large investments in developing study material
can be justified through less interaction and rationalisations
resulting from large scale teaching. Our experiences does not
support this. Originally we believed that distance teaching based
on computer mediated communication could be based on learning
material more similar to that used in on-campus studies. According
to our experiences this seems not to be the case. Thus, Internet
based distance teaching may become expensive because developing
high quality materials demand resources and student expectations
of teacher involvement tend to become quite high.
The average turn-around time for assignments was 3.1 day, according to the students varying a lot from tutor to tutor. The students' view on the quality of the tutors' work and satisfaction with the turn-around time experienced varied a lot. Some students argued that the tutors should put more effort in facilitating and moderating activities in the academic conferences, and the open answers indicated that one should stress the necessity for giving quick response to student answers and assignments.
The questionnaire ended by asking: "Would you recommend studies
on the Internet to others who were qualified and interested in
the subject?" It was encouraging that a large majority of
the students say that they would recommend Internet studies to
other people (only 5% ticking on the negative side).
The general impression from the evaluation study was that the
students were reasonably satisfied with their experiences as Internet
students with NKI Distance Education. It generally seems that
the students valued the aspects of study that have been emphasised
in distance education systems, such as material for individual
self instruction, two-way communication with enthusiastic tutors
supplying individual support and personal comments and guidance.
The students were not so concerned about social and academic group
communication. The reason may be either that these aspects are
not considered so necessary for their learning, or that they have
taken so little advantage of the possibilities that they really
do not know, or that the possibilities and advantages are not
exploited fully by NKI administration and tutors.
Concerning the use of the WWW we need to experiment more in different
courses to find out how the Internet resources can be applied
in better ways. Presenting material on the WWW is probably not
the most important development. The big challenge is to use the
WWW and its resources for stimulating meaningful activities including
links to external resources based on a course presentation structure
which makes the learning more interesting, efficient - and effective.
Future evaluation should focus more on how to use the Internet and WWW for didactic purposes with an emphasis on the aims, objectives and teaching methods of each specific course.
[1] Garrison, D. R.: Multifunction microcomputer
enhanced audio teleconferencing: Moving into the third generation
of distance education. Int. J. of Innovative High. Ed.
3(1),26- 29. 1986.
[2] Garrison, D. R.: Understanding distance education.
A Framework for the Future. Routledge, London. 1989.
[3] Grepperud, G.: SOFF and the National Distance
Education Network. In: George, J. W., Nylehn, B. & Støkken,
A. M.: Distance Education in Norway and Scotland. Experiences
and Reflections. Pp. 36-52. Edinburgh: John Donald Publ. 1996.
[4] Lauzon, A. C & Moore, G. A. B.: A fourth
generation distance education system: Integrating computer-assisted
learning and computer conferencing. The Am. J. of Dist. Ed.,3:1,
38-49. 1989.
[5] Paulsen, M. F. & Rekkedal, T.: Technology
for adult learning in Norway Including a case study on the NKI
Electronic College. In Adult Learning and Technology in the
OECD Countries. OECD. 1996.
[6] Paulsen, M. F. & Rekkedal, T.: Computer
conferencing - a breakthrough in distance learning or just another
technological gadget? I: Sewart, D. & Daniel, J. S.: Developing
Distance Education. Papers to the 14th ICDE Conference,
Oslo 1988. Oslo: ICDE. 1988.
[7] Rekkedal, T.: Tekniske studier. Korrespondanseundervisning
og klasseundervisning. Delrapport 2: Brevskoleelevene. Stabekk:
NKI. 1978.
[8] Rekkedal, T.: Recruitment and Study Barriers in the Electronic College. In: Paulsen, M. F. & Rekkedal, T.: The Electronic College. Selected Articles from the EKKO Project. Bekkestua: NKI/SEFU. 1990.
[9] Rekkedal, T. & Paulsen, M. F.: The Third Generation NKI Electronic College. A Survey of Student Experiences and Attitudes. An Evaluation Report Written for the Leonardo Online Training Project http://www.nki.no/eeileo/. 1997.