The Background for the Development of NADE's Quality Standards for Distance Education
Paper to the 1994 EDEN Conference:
Human Resources, Human Potential, Human Development: The Role of Distance Education.Tallinn, Estonia, 6 - 8 June, 1994
Norway was the first country to regulate distance education by a separate law (in 1948). This Act has been in operation from 1949 to 1992, and has had considerably influence on the development and status of distance education in Norway.
Recently an amendment has been passed that integrates distance education into the Norwegian Adult Education Act. At the same time an extensive accreditation scheme has been terminated and the Correspondence School Council, which was in charge of the accreditation on behalf of the Ministry of Education, has been closed down.
This paper report and give a summary of the experiences and areas of concern that have brought about this amendment and how NADE has approached the challenge of assuring and developing quality in distance education.
In the education system, distance education - for as long as it has existed -has been a relatively marginal phenomenon. Thus, all over the world spokesmen for distance education have had to battle the whole time to get it recognized as an equivalent form of education. Even the big, state distance education institutions that have been established in the last twenty years have encountered the same scepticism from traditional academic milieu and have had to fight a continuous battle for recognition.
The private correspondence schools have usually been even more subject to suspicion from the established education system and from the authorities. This sceptisism has partly been based on the sheer fact that the schools were privately owned, but may also have been a result of some private schools conducting pedagogical and business practices that rightly could be critisised.
Attempts have been made to solve this problem in various ways.
Some have found it most appropriate to let the market punish those who do not measure up. Here the idea is that in the long run, quality will pay off, and substandard measures and institutions will disappear of their own accord. This reasoning is, in our opinion basically correct, but one problem is simply that indivdual distance students scattered over a large area, have little or no contact with each other. Thus, it is possible to earn money by taking advantage of people's credulity without giving the customers much opportunity to hold the seller responsible. Even incidental and short-lived ventures of this sort can have a destructive effect on the general reputation of distance education.
Thus, the valid institutions have usually been interested in measures that can assure a certain level of quality. It has often been an important factor when organisations are formed that they should help set standards for education and business practices. Thus, in many cases correspondence school organisations have prepared guidelines for "good correspondence school practices" in the form of a Code of Ethics. It has also been common that new members had to be accepted by the remaining members and that departures from agreed standards could lead to exclusion from the organisation. This applies to both national and international organisations.
The Norwegian Association for Distance Education (NADE) was established in 1968. In the first charter the compilation and enforcement of rules for good correspondence school practices were listed as one of the aims of the association. These rules were immediately drawn up. They have not been amended since then.
NADE's rules for good correspondence school practices were relatively concise. In addition to requiring loyalty within the organisation and respect for the authorities' decisions, they included provisions relating to the minimum amount of information that should be provided in school catalogues and provisions regarding responsible advertising and sales methods. They also stipulated that the institutions should make use of competent professionals as course authors and teachers, monitor their work, and conscientiously follow the students' progress and motivate them to complete their studies with good grades.
Given that we in Norway have had quality control through state accreditation of all correspondence courses, NADE's rules have been more important ethically and commercially than pedagogically.
Some organisations have gone a step further by establishing accrediting schemes and separate institutions for accreditation of schools and curriculums. The best known of these is the National Home Study Council in the USA, which established its Accrediting Commission in 1955. This was approved as an accrediting body by the federal government in 1959. In the Netherlands a foundation for the inspection of correspondence schools was established in collaboration between the correspondence schools and the authorities as early as 1947. It was converted into a public, independent inspection scheme in 1973. In Great Britain they also got a semi-public, voluntary scheme for accreditation when the Council for the Accreditation of Correspondence Colleges was established in 1969. In this instance the authorities have gradually withdrawn, so that the CACC is now an independent, non-profit institution.
The idea of establishing supranational accreditation schemes has been aired now and then, but has not gained sufficient approval. One possible international scheme was discussed within the International Council for Correspondence Education (ICCE) at the close of the 1960s. In recent years the political integration of Europe has made it more opportune to consider a solution at the European level, and it has thus been advocated that a European accreditation scheme patterned on the national schemes be established. So far this idea has not gained very broad support.
The third solution, which has been chosen in a number of countries, is quality control through government legislation.
Since the last world war many countries have regulated distance education, usually by means of special laws for this type of education. Norway was the first to do so in 1948. Other countries that have followed suit include Italy, Denmark, Belgium, Spain, West Germany, France and the Netherlands.
One common feature of most of this legislation is that it defines private distance education as a separate area. It does not concern itself with distance education from public institutions, and it usually does not tie in with legislation relating to other kinds of private schools. In most cases the main aim seems to be to safeguard consumer interests - thus, this is explicitly stated in the German Gesetz zum Schutz der Teilnehmer am Fernunterricht (Act of 1976 relating to the protection of the student in distance education). It is reflected in provisions relating to contract terms and sales methods, but most laws also include provisions that are supposed to help assure the professional and pedagogical quality of the courses that are offered. The policy instruments here may include the accreditation of the individual courses and curriculums, qualifications requirements for teachers and administrative personnel and inspection schemes. In many cases separate institutions or bodies have been established to exercise the state control.
There may be reason to question how successful the regulation of private distance education has been. To be sure, it is possible to point to the positive experiences of many countries. In particular legislation seems to have an immediate effect in getting some substandard activities removed, at any rate in the cases where the control is obligatory. Public inspections usually also lead to an improvement in quality at most institutions, but some negative experiences have also been reported. Some argue that an emphasis on minimum standards may result in a mediocre level of quality. It can also have a negative effect if the requirements introduced are too detailed. Quality appraisals and pedagogical attitudes keep evolving, and rules and criteria can easily lag behind the development of pedagogy and technology and becoming barriers for progress in the field.
In university circles one can hear playful self criticism to the effect that "a university is an institution that does research on everything, except its own activities." We shall not state any opinion about this statement, but it may well be correct that systematic evaluation of the quality of the education in the traditional educational system has had a relatively low status, both in Norway and in other countries.
At the same time, it is correct from our standpoint to point out that the evaluation of education has played an important role in the activities of most modern distance education institutions.
There are many reasons why the evaluation of education or programmes has been taken seriously in distance education circles. Some of them are:
- The way distance education has been organised, there are fewer possibilities for direct feedback from students to those responsible for the education. This means that those who plan, develop and implement the education have to formalise their work with the aim of providing information about how the education works. This is not as necessary in systems where the teachers, who are often responsible for both planning and execution are in continuous contact with their students.
- Distance education is normally organised in a system characterized by a division of labour among a number of "expert groups." Formalised knowledge about this process thus becomes more necessary in order for the information to affect system changes.
- Distance education is often initiated on the basis of defined political resolutions and objectives concerning looking after needs in the society, e.g. increasing the availability of education for defined groups. Thus, the evaluation of results will be desirable in order to determine the extent to which the political intentions are realized.
- Distance education is an area characterized by new developments, both as regards pedagogical solutions, media and technology utilisation, and organisation. Various models and systems are tried out, and information is needed about their effects and effectiveness.
- In many situations distance education has been developed in a tradition of systems thinking and educational technology. In this tradition systematic feedback, evaluation and revision and/or further development all play an important role.
- We can probably also add that both public and private distance education have had a need to develop and "demonstrate" quality because in some situations they are battling with a perception of being only second-rate. As mentioned in the introduction, this is not just limited to private institutions.
Thus, it is typical that most large distance education institutions have established separate departments or institutes for research and evaluation, or have formalised their quality improvement and quality assurance work in other ways as soon as they were established. Examples of this are Britain's Open University with its Institute of Educational Technology and FernUniversität in Germany with its Zentrales Institut für Fernstudienforschung (Central Institute for Distance Education Research) and Zenter für Fernstudienentwicklung (Centre for Distance Education Development). Norwegian distance education institutions, such as NKI and NKS, have also carried out systematic research and evaluation work for many years. To unite efforts this work has been organised in the Centre for Distance Education (SEFU) in cooperation with the State Institution for Distance Education (NFU). (In this context we ought to mention that in 1992 SEFU a report on theoretical areas of concern and practical experiences related to the evaluation of distance education.)
Concerning quality work in business and industry, there has been an increasing emphasis on requiring that those responsible for the processes also should take responsibility for quality control and quality developmentment. This should also be the case in education. One example of this position is Mary Thorpe's book on evaluation of open learning and distance education emphasizing the necessity of letting the practitioners take an active role in the evaluation processes. Thus, the book becomes a kind of manual for "practitioner-based" evaluation. In this context the practitioners include all of those who are involved in the courses that are offered: administrators, marketing experts, professional staff, teachers, tutors, textbook authors, office personnel and receptionists. The idea is mainly that if the evaluation is really to have a positive impact on the improvement of quality in the educational system, the practitioners must understand the importance of taking part in the evaluation processes. The results of the evaluation processes have two main target groups - the practitioners themselves and current and future students. Just as with quality assurance in general, Thorpe argues that the evaluation of education ought to be carried out as a group activity that brings together personnel from various parts and levels of the organisation.
In many areas of the society - in industry, public administration and the service sector - we have witnessed a rapidly growing interest in and emphasis on matters of quality in recent years. In industry, quality control is a well-known phenomenon, i.e. manufactured goods are inspected and then either approved or rejected on the basis of well-defined specifications. A high percentage of errors means that production is not cost-effective. Thus, quality control gives rise to a need for quality assurance - i.e. routines and systems that can ensure that the manufactured goods meet the specified quality standards.
A firm that can document an effective quality assurance system, will be more easily able to inspire confidence as a supplier of goods. This is one of the reasons why national and international standards for quality assurance have been drafted. The international standards do not define the products' quality, but describe the requirements that ought to be made of the firms' quality systems. The quality specifications are set by the individual firm, or jointly by the firm and a contract partner.
The standards that exist have been developed in connection with production-oriented industry. However, they are also being increasingly employed in service-oriented activities and are being revised and supplemented with these activities in mind. Often the use of quality standards is combined with ideas about total quality management. Total quality management is usually associated with an extensive effort to focus the whole organisation and its mode of operation on the users' needs, with a continuous monitoring and improvement of the quality of the organisation's performance.
Public educational institutions usually base their activities on academically oriented attitudes and concepts of quality and are simultaneously subject to the authorities' productivity and efficiency controls. This is also true of institutions that make use of distance education, although these institutions may tend to be a little more market-oriented, especially if they offer business-oriented studies. Independent distance education institutions may find it even easier to base their activities on strong trends and requirements in the business community. They often provide training programmes for firms. They are also usually very concerned with satisfying their customers' - i.e. the pupils' and students' - needs for good, effective training. Thus, more and more often we confront the question of whether, and if so how, quality standards and ideas about quality assurance and total quality management can be employed for institutions that provide distance education.
This is by no means problem-free. Among other things, the concept of quality itself is far from unambiguously defined, neither in a business nor an educational context. Quality control and quality standards in production-oriented industries presume that the products must meet particular specifications or quality requirements. In an academic tradition quality will more likely be regarded as something that distinguishes itself as better than average. Quality in education can take a number of different forms, and it is difficult to define or specify, but also in pedagogical matters there are trends that put great emphasis on measurability and specified goals for the education. We have also mentioned that some of the legislative or voluntary controls of quality in distance education are traditionally based on specified requirements for pedagogical and business practices.
In the service sector and in total quality management thinking, it is usually the users' needs that are given priority. Quality is defined as fitness for purpose, and this fitness is determined by the users' needs and the way they experience the service in question. The goal is to meet the customer's needs in the most cost-effective way. We can generally regard this emphasis as a parallel to a student-oriented way of thinking in an educational context. The requirements of the education may also be to achieve the students' goals in the most efficient way, but the students' goals may have to be corrected on the basis of other ways of defining quality in education. The user perspective must be supplemented with academic standards.
Freeman has described some of the problems involved with applying existing standards for quality assurance when preparing materials for education. The basic problem is that the processes that are usually employed in the production of learning material do not comply with the condition in the standards concerning the specification of requirements and the testing of the elements in the product on the basis of unequivocal specifications. Instead, quality educational products are usually developed according to an iterative model with drafts that are evaluated by experts and revised until the results are satisfactory. Thus, the question is whether it is possible to develop ways of producing learning material that meet current standards, or whether other kinds of standards are required for the processes that are employed in education and the development of learning material.
Although basic questions of this sort regarding the use of quality standards in education must be faced, there is hardly any doubt that quality thinking can be employed and can also be fruitful for educational institutions. It is possible to describe the processes and functions that are involved in an institution's activities. It is also possible to define certain specific and measurable quality requirements for these processes and their results whether these requirements are based on good judgment or on more systematic knowledge about the activity. It is possible to develop systems for assuring and improving quality on the basis of these requirements and standards. A sensible use of these quality improvement systems will most likely improve the quality of educational institutions.
An extensive attempt to establish guidelines for quality in an educational context is the SATURN organisation's quality guide for open learning and distance education. This guide takes its point of departure in the efforts at standardisation and total quality management in the business community and attempts to transfer these intentions to educational activities. Thus, it attempts to create a terminology, to define key areas and to provide guidance in the form of recommendations and check lists. SATURN's quality guide regards teaching from the perspective of the roles of various actors: providers of information, developers, deliverers, corporate customers and individual learners. Requirements and guidelines are linked to the following key areas:
For each area it specifies what the various actors ought to do in order to assure quality (one general recommendation for each area), and more detailed checklists are provided in the form of questions.
Though it can be argued that systematic evaluations and quality improvement work have got a late start in the traditional educational system, a number of activities that aim for quality improvements have been initiated in recent years. This work has consisted partly of political and organisational changes and partly of work specifically associated with school appraisals and the quality of studies.
The quality of the educational system has been on the agenda in recent years. Among other things, an OECD report from 1988 relating to educational systems and policies helped stir up the debate. Government studies from the last four or five years have also influenced the debate and stimulated the work.
The Quality of Studies Committeethat was appointed to take a special look at the quality of studies in higher education has also had an impact on the ongoing development.
This committee proposed a number of measures aimed at improving the quality of higher education. Among other things, the committee proposed that a pilot project should be started with a national evaluation of a selected set of courses being offered based on a combination of internal and external evaluations. In evaluations in general and the pilot project in particular, the committee points out that both internal and external evaluations have quality improvement and not control as a goal. The experiment that the Quality of Studies Committee proposed was initiated in 1992 under the direction of the Norwegian Institute for Studies in Research and Higher Education. More than 40 institutions and/or local departments are participating in this evaluation of economic-administrative studies, including several distance education institutions. External expert groups with international participants have presented evaluation reports based on the institutions' own internal evaluation and visits to the institutions.
An important example of active experimental activity in recent years is the National Council for Upper Secondary Education's (RVO) school evaluation project. The initiative was taken in 1987 with a pilot project at six schools in two counties during the period from 1990 to 1992. On one of the introductory pages in the basic document it reads: "School evaluation as a new concept entails that the individual school has greater responsibility for systematic evaluation..."
This quote is also pertinent to the distance education institutions' responsibility for their own quality pursuant to the Norwegian Adult Education Act. School evaluation and quality improvement in the public education system must also be considered in light of the central authorities' increasing interest in efficient organisation and management by objectives.
As for the attempts of public bodies to stimulate quality improvement, it should be noted that the Norwegian Executive Board for Distance Education at University and College Level (SOFF) has put considerable emphasis on the institutions' responsibility for evaluating the individual projects. Also in this context the importance of evaluation for quality improvement in individual institutions and projects has been specified, and emphasis has been given to the exchange of experience and expertise between institutions and between projects.
As mentioned, the Act of 12 November 1948 concerning Correspondence Schools was the first law in the world in this area. Actually it was passed as a result of a government study originally aimed at establishing a state correspondence school.
According to the Act concerning Correspondence Schools, a separate council, the Correspondence School Council, was supposed to help the ministry with the supervision of the correspondence schools. In the first years the Correspondence School Council's supervisory work consisted mainly of subject related and pedagogical reviews of individual courses prior to accreditation by the ministry. The Council also conducted inspections at the schools in order to appraise an institution's total activities. According to an amendment in the Act in 1969, not only the individual courses, but also the schools were required to gain accreditation.
Through the Act concerning Correspondence Schools and the work of the Correspondence School Council, the authorities demonstrated that private institutions were to have the main responsibility for carrying out correspondence education, while the public authorities wished to monitor the quality and eventually provide economic support according to particular rules.
There is hardly any doubt that the Norwegian Act concerning Correspondence Schools, the work of the Correspondence School Council and the ministry's accreditation of correspondence courses and correspondence schools have had a positive impact on the development of correspondence education in Norway. It would also be correct to argue that there are few examples of institutions that have operated with insufficient quality or doubtful business practices in the period that the Act concerning Correspondence Schools has been in effect in Norway.
The accreditation scheme has ensured that each individual course was evaluated by independent consultants before it could be put on the market. As a result the schools have established routines for quality assurance through the choice of authors, the editing work and in some cases the use of independent consultants before the courses are submitted to the Correspondence School Council for evaluation. Naturally the schools have wanted to avoid the delays and negative rating associated with not having prepared their material sufficiently before the application for accreditation was submitted. The evaluation by independent consultants has provided feedback that has affected the actual development of the courses, so that changes and revisions could be carried out before the course was put on the market.
Another factor that has had a positive impact on quality assurance is the limiting of the accreditation to a period of three to five years. In this way Norwegian correspondence schools have had to establish systematic routines for revision, reorganisation and updating of the course material and the development of new courses.
In addition to the accreditation of individual courses, the Correspondence School Council has been responsible for supervising the schools. This has mainly been accomplished through inspections at the schools (originally at least once a year) and through mandatory information that the schools were required to submit, e.g. statistics and annual reports. In recent years the Correspondence School Council conducted a more thorough review of the individual schools' total activities with lengthy and extensive evaluations based on internal reports from the institution in question, meetings with key personnel and external analysis by the Correspondence School Council's representatives. The final reports from these evaluations of institutions dealt with marketing and economic systems, tutoring, support and counselling, qualifications for full and part-time employees, the quality of individual courses and study programmes that are offered, turn around time of assignments, experimental activities and administrative routines, i.e. all the factors that affect the total quality of the educational programme that is offered.
The Correspondence School Council's instructions also stipulated that the council should work to see that correspondence school education was as professionally and pedagogically up to date as possible. Among other things the council was supposed to motivate the schools to do research and development work. In connection with this the schools cooperated with the Correspondence School Council on training and development among the employees of the correspondence schools.
The Course Committee, a liaison body between NADE and the Correspondence School Council, held regular courses and conferences for employees of the correspondence schools. Some of these projects ended up in practical handbooks for quality improvement, e.g. a manual on the development of correspondence courses. The Correspondence School Council also took the initiative to a liaison with NADE concerning the development of a distance education programme for distance tutors in order to ensure that smaller institutions with few resources would also have an opportunity to give their teachers satisfactory training. The collaboration on tutor training had its point of departure in the fact that the two largest distance education institutions, NKS and NKI, had previously developed study programmes of this sort.
It is likely that the authorities' follow up, supervision and economic support have played an important role in ensuring that distance education in Norway, which so far has mainly been represented by the independent distance education institutions, is recognized nationally and internationally for its high level of quality and competence. The situation is also a result of the institutions' willingness and ability to carry out systematic quality improvements, partly through research and development work. We can safely ascertain that the Norwegian Act concerning Correspondence Schools has had a positive impact on the development of distance education in Norway.
The repeal of the Norwegian Act concerning Correspondence Schools, the placing of the approved, independent distance education institutions under the jurisdiction of the Norwegian Adult Education Act, and the delegation to the institutions of independent responsibility for their quality improvement work are each in many ways the result of a natural development.
This change is partly a result of certain weaknesses in the old accreditation and supervision scheme and partly a result of the relatively high level of competence that the independent distance education institutions have attained. Furthermore, the technological and pedagogical developments in distance education and recent ideas about how quality improvement work should be organized and implemented in general have also tended to support the change. Last but not least, we should mention the close cooperation and trust between the institutions and the ministry that have been established in recent years. All of these factors are closely related and the natural result is that evaluation and quality assurance have primarily become the institutions' own responsibility.
Some weaknesses of the accreditation scheme under the Norwegian Act concerning Correspondence Schools can be related to factors involving the question of objectivity and how the institutions might utilise the findings of the Correspondence School Council's consultants. The question of objectivity cannot be resolved in any particularly easy way. It is not possible to guarantee that external appraisers will be able to delve sufficiently deeply into the course to give a reasonable appraisal or that they will be more objective than internal appraisers. Another problem is that those responsible for a course may loose their own initiative in the quality appraisal. For example, it is not very promising for an institution's quality improvement work if they regard an external approval as a guarantee that the product meets the quality requirements that they themselves must and ought to be making.
Due to questions of competence and objectivity, the Correspondence School Council's consultants had to be recruited from institutions that did not offer distance education. This meant that in many ways they were not very well qualified to evaluate correspondence courses and correspondence school education. Often they were poorly qualified to appraise what distance education entails on the whole and how the material in an individual course should function as one element in the total study programme. They could easily end up evaluating the actual course material by itself and comparing it directly with text books that were often developed for other target groups.
There is little doubt that some consultants also started out with negative attitudes to distance education: "My field cannot be taught through a correspondence course." In some cases the consultant has had a more teacher-oriented attitude to education as opposed to the student and/or study-oriented attitude, which has characterized distance education for a long time. These factors may have made it difficult to accept the teaching philosophy of the course.
Whereas the consultants appointed from other institutions could be regarded as objective, it is clear that many, particularly in recent years, were not. The lack of objectivity could be because other institutions regarded the distance education programme as a competitor. The recruiting base for certain types of education has been scanty at times, and some institutions may have been slow to adapt to demands from students or the authorities for less traditional and more flexible study programmes.
It should certainly also be mentioned that external appraisals may cause those responsible for the course to pay more attention to the consultant's appraisals than they do to the students' needs. In some cases, in fact, it turned out that the consultant (and the Correspondence School Council) demanded changes in the course that the schools' course development personnel, on the basis of their knowledge of the field and of distance education, regarded as detrimental to the quality of the course.
During the 20 years that the Norwegian Act concerning Correspondence Schools was in effect, the schools underwent an extensive competence building. Since 1970 some of the schools have conducted significant research and development work, either individually or in collaboration, which has resulted in internal competence building and external recognition. Under these circumstances, the external supervision of details seemed a little unnecessary.
Simultaneously, the technological development has rendered the Act concerning Correspondence Schools and the accreditation scheme somewhat outmoded. There are a few main reasons for this. The institutions that came under the Act concerning Correspondence Schools have evolved from "correspondence schools", which conducted education based on printed material and two-way communication through the mail, into "distance education institutions" that employ computer technology, new media and communications technologies in a number of different ways. The accreditation scheme was not adapted, and could not easily be adapted, to this new situation. The act, the accreditation scheme and the economic assistance system tended to preserve the status quo rather than attend to the schools' actual situation and development needs. At the same time, other types of institutions were able to conduct distance education completely outside the control of the authorities by means of technology that was not regulated by law and authorization schemes.
Taken together, these factors have brought the authorities to the conclusion that distance education should be regulated as a kind of adult education, and that accredited distance education institutions themselves, in collaboration with the authorities, are best able to take responsibility for quality control and quality assurance.
After the public regulation of distance education in independent institutions was integrated into the Norwegian Adult Education Act, effective 1 January 1993, the responsibility for ensuring the quality of the learning material, the teaching and the practical implementation of the study programmes has been delegated to the individually approved distance education institutions. NADE has been requested by the ministry to prepare guidelines for quality standards in distance education.
The previous accreditation scheme was strongly focused on accreditation of the individual printed course materials on the basis of a subject related and pedagogical as well as an evaluation of gender equality. In addition, an individual school was accredited i.a. on the basis of requirements as to the competence of its staff. The Correspondence School Council conducted inspections of the schools: their marketing, pedagogical practices and other factors that were deemed important to the students. When the amendment was passed it was specified that the evaluation of quality ought to have a broad basis. In the documents related to the bill this has been expressed as follows:
In the autumn of 1992 a working group appointed by NADE's executive board carried out an introductory study of this matter. NADE's Standing Committee on Quality, which was established by the executive board on 18 December 1992, has continued this work. The main points in the Standing Committee on Quality's mandate are formulated as follows:
NADE's Standing Committee on Quality is supposed to be the association's expert body in quality matters. It is supposed to work on matters involving quality criteria, quality standards, and quality assurance and improvement in distance education. The committee can offer opinions to the executive board, propose measures, and, when requested by the board, take responsibility for implementing measures aimed at promoting quality improvement work among the association's members. The committee submits an annual report concerning its activities to the executive board.
In consultation with the Ministry of Education, Research and Church Affairs and NADE's executive board, the Standing Committee on Quality is supposed to formulate and update criteria and guidelines for quality and quality assurance in accredited, independent distance education institutions. The committee is supposed to function as the association's liaison body and adviser to the ministry in matters concerning quality and quality assurance. Based on the criteria and guidelines, the committee can advice the ministry in individual cases.
The members of the committee are appointed by NADE's executive board on an individual basis. The committee as a whole should have a broad and versatile expertise in matters concerning distance education. The executive board appoints the committee's chairman. NADE provides a secretary to the committee.
The Standing Committee on Quality is supposed to assist the individual member institutions in taking care of their responsibility to provide quality distance education and to furnish the ministry with a general basis for evaluating whether the quality assurance at the independent distance education institutions is satisfactory. The committee is supposed to promote an awareness of quality that is based on requirements for professional-pedagogical, ethical and organisational aspects of these institutions' activities.
The following main tasks should be attended to:
It should be noted that NADE's Standing Committee on Quality is meant to be neither an accrediting body nor an inspection body equivalent to the former Correspondence School Council.
The individual distance education institutions differ greatly among themselves in purpose, type of activity, resources and size. It is therefore difficult to devise quality standards that are equally applicable in spite of these differences. NADE's standards are supposed to be recommendations and must give the individual institution sufficient freedom to define quality requirements on the basis of its own circumstances and possibilities. At the same time they must establish certain minimum requirements that are expected to be met if the institution is to be able to maintain a justifiable level of quality.
The quality standards have both an internal and an external function. Internally at the individual institution, they are supposed to serve as guidelines for the institution's own quality improvement work. They should not relieve the institution of the responsibility for defining its own quality improvement policy and quality goals, nor should they prescribe in detail how the quality improvement work should be carried out. However, they should help the institutions become more aware of their responsibility for quality in various areas, serve as an aid in the institutions' self-evaluation and define the agreed standards of NADE, which an accredited institution is expected to meet.
In this context the previous rules for good correspondence school practice are insufficient. As previously mentioned, they are very concise and general and do not go into any detail about quality standards or criteria. Though the new guidelines must also be relatively general, there is a need for a greater degree of detail and specification.
By establishing more specific standards, they also automatically acquire an external function, and thereby contribute to the specification of quality standards that are relevant for distance education under the direction of other institutions besides the independent ones. This applies to the members that are under a different jurisdiction (e.g. the universities), and also in principle to institutions that are not members of NADE. In this way we hope that NADE's quality standards can have an impact on the quality of Norwegian distance education in general.
Moreover, the standards should provide information and guidance to the ministry in its evaluation of the independent institutions in connection with accreditation and any complaints that may arise. When the ministry is to evaluate the accreditation of institutions, the decision must be made on the basis of a comprehensive evaluation. That means that individual departures from the guidelines for quality standards must be acceptable without doing away with the accreditation altogether. The ministry must also consider how well the standards suit the activities of the individual institutions. Thus, the ministry must employ the standards in a concrete evaluation of the individual institution's special features in dialogue with the institution itself.
The working group and the committee have approached the quality improvement and quality assurance work from different angles and have considered quality requirements, evaluation systems, checklists, etc., which have been utilised in different contexts. We did not want to bind the institutions to a particular method of approach in their quality improvement work (e.g. based on ISO standards). There ought to be room for some diversity of thought and approach without forgoing the necessity of establishing certain common quality standards.
We chose to take our point of departure in a matrix of problem areas for evaluation that is described in a report from Lund University. The report describes a model for the evaluation of a professional field or an institution. One step in this model is the institution's self-evaluation, and the model designates nine areas for this self-evaluation. The areas are determined by a matrix in which one evaluates students, teachers/courses and organisation in terms of conditions and constraints, processes and results, respectively. This yields a total of nine boxes, whereas the institution itself determines which questions should be evaluated for each area.
In the work with the quality standards, we have altered this matrix so that it is better adapted to distance education. We have divided the distance education institutions' activities into four main categories. Each of these main categories is again divided into four phases. These are combined in a matrix of 16 elements, which we have called quality areas. For each of these quality areas certain factors have been specified, which can or ought to enter into the institution's evaluation of its own quality. These factors may be more or less important for different institutions and types of activities. In some cases other factors besides those included here may also be relevant.
| Conditions and Constraints | Implementation | Results | Follow up | |
| Information and Counselling | 1 External constraints Organisation Partners |
2 Channels Content |
3 Student body Other results |
4 Evaluation Customer relations |
| Course Development | 5 External constraints Organisation Target group Staff Partners |
6 Supervision, co-operation,follow-up and guidance of authors Choice of media Formative evaluation |
7 Course description Materials meeting requirements Teaching aids |
8 Evaluation Customer reactions Updating and/or revision |
| Course Delivery | 9 External constraints Organisation Students Materials Teachers Partners |
10 Two-way contact Teaching and guidance Exams and tests |
11 Students' achievements of goals Course completion Learning results |
12 Evaluation Customer relations |
| Organisation | 13 External constraints Organisation Partners |
14 Management Communication Future orientation |
15 Achievement of goals Financial results Repute |
16 Evaluation Reporting |
The quality standards that have been specified are grouped and numbered according to areas and factors that have been included in the matrix. Although a factor may be important for quality, it is not always possible or desirable to define general quality standards that are linked to that factor. Hence, quality standards have not been specified for all of the factors, and the number of standards also varies a great deal among the different areas. (Appendix 1 gives as an example Quality Area 6: Course development - Implementation with factors and quality standards.)
Sometimes expressions like "should" or "must" are used in the quality standards. In these cases the standard is meant to expresses a requirement that an approved institution is expected to meet. These standards have therefore been printed in bolder type. On the other hand, when the expression "ought" is used, it means that the standard is not regarded as an absolute requirement. In a comprehensive evaluation of an institution's quality, however, the extent to which the institution meets all of the quality standards that are relevant to its activities will be a significant factor.
NADE's quality standards are recommended standards, and in most cases they are formulated rather generally so that they can be applied in very different institutions. We have not tried to formulate precise specifications of quality in the different areas. In an actual institution it is obviously possible and desirable to go further with the specifications than we have gone here. The individual institution must therefore specify its own quality requirements and quality assurance and improvement systems. In that sense NADE's quality standards should be regarded as a point of departure and a guideline for the institutions' quality improvement work.
Starting with the NADE standards, the institution ought to go through the sixteen quality areas with an eye to its own activities. First the specified quality improvement factors ought to be evaluated. How relevant are they to the activities? Have any factors been overlooked? Thereafter, the individual quality standards can be considered. If they are relevant to its activities, the institution will often be able to formulate the requirement more concretely and specifically in relation to its own organisation and way of working. It will also be possible to define requirements for aspects of the quality of the activities that NADE's general standards do not touch upon.
The institution ought to also evaluate the organisation of quality improvement measures and systems and/or routines for quality assurance. Many of them will choose to formulate a comprehensive quality assurance system with comprehensive documentation where a single person is responsible for the system. NADE's quality standards do not make any specific requirements about how the quality improvement work should be organised and documented internally in an organisation. Nevertheless, this is a very important aspect of the institution's quality improvement policy.
The institutions will be subject to some external requirements for documentation, primarily in connection with reporting to and supervision by the Ministry of Education, Research and Church Affairs. It will be an advantage for both the institution itself and the ministry if quality improvement measures and quality assurance can be documented in a well-arranged and easily accessible form. However, more specific rules for this have not been devised.
A temporary version of the quality standards was sent out for a hearing to NADE's member institutions in April 1993, with a 1 June deadline for comments. Based on these comments, the Standing Committee on Quality has revised the standards into their current version. The Standing Committee on Quality has recommended that the current version of the quality standards be put into effect for a trial period and thereafter revised on the basis of the experiences gained during this period of testing.
If we consider the historical development of forms of quality assurance and quality control of distance education in Norway, we may perhaps conclude that the new scheme attempts to combine different solutions and find a balance among many different policy instruments. Since 1948 Norway has had a strict regulation of independent distance education. It has been forbidden to market correspondence school education without state accreditation of each individual course that is offered. In practice, however, this prohibition has not been strictly enforced in recent years. The amendments of 1993 mean that private commercial distance education is now permissible without state accreditation. At the same time, the state accreditation of courses will be phased out.
What remains of the previous regulation is the state accreditation of institutions. This accreditation is simultaneously tied to the possibility of receiving government grants so that only accredited schools have the right to economic assistance for their activities. Accredited schools must submit a report on their activities, and the ministry has the authority to conduct inspections.
As a supplement to this "mild" form of quality control by the authorities, the development of agreed NADE standards as a basis for the institutions' own quality assurance is being supported. Over the years a significant mutual trust has developed between the authorities and the independent institutions with their joint body, NADE. This is the basis that allows this form of coordination between voluntary and statutory schemes to function.
* This paper draws on the monograph by Ljoså, E. & Rekkedal, T.: From External Control to Internal Quality Assurance published by The Norwegian Association for Distance Education, 1993.
Act of 12 November 1948 concerning Correspondence Schools (amended by Act of 7 February 1969)
Act of 28 May 1976 no 35 on Adult Education (amended by Act of 19 June 1992 and provisions, Ministry of Education, Research and Church Affairs (KUF), 1993)
Allen, R (1969): International accreditation, or a federation of regional and national accrediting agencies. I Erdos, R (red): Proceedings of the Eighth International Conference of the International Council of Correspondence Education, s 282-296. Chicago: ICCE
Amdam, R P og Bjarnar, O (1989): En bedrift i norsk skole. NKS og fjernundervisningen i Norge 1914-1989. Oslo: NKS-Forlaget
Barache, J (1988): Total Quality policy in distance education as a general process of innovation. Epistolodidaktika 1988:1, s 11-29
Bjørndal, I, Tiller, T og Simmons, H (1991): National aims and local freedom. A school evaluation project in Norway. Oslo: RVO
Brevskolerådet/Norsk Brevskoleforbund (1975): Brevkursproduksjon i lys av systemtenkning. Oslo: NKS
Brevskolerådet (1977/1989): Veiledning for brevskolelærere. Revised edition by NKI Forlaget
Council for the Accreditation of Correspondence Colleges (udatert): Accreditation & Re-accreditation.
Freeman, R (1991): Quality assurance in learning material production. Open Learning, Vol 6, No 3, s 24-31
Gesetz zum Schutz der Teilnehmer am Fernunterricht (Fernunterrichtsschutzgesetz - FernUSG) vom 24. August 1976 (BGBI. I, S. 2525)
Guri, S (1987): Quality control in distance learning. Open Learning, Vol 2, No 2, s 16-21
Helgesen, H (1945): Skolen for alle. En ny vei i opplysningsarbeidet. Oslo: AOF
Helgesen, H (1969): Statutory provisions relating to Norwegian education by correspondence. In Erdos, R (red): Proceedings of the Eighth International Conference of the International Council for Correspondence Education, s 259-280. Chicago: ICCE
Henderikx, P (1992): Management and promotion of quality in distance education. Open Learning Vol 7, No 3, s 34-41
Ingham, E (1991): Voluntary accreditation in Europe? Epistolodidaktika 1991:1 s 78-82
Interparlamentarische Gesellschaft für Bildungsmedien (1992): Dokumentation: Europa-Tagung Fernunterricht. Bonn
Jordell, K Ø (1992): Nasjonal evaluering av økonomisk-administrativ utdanning. Premisser og prosess. Rapport 3/92. Oslo: NAVF
Kaiser, C (1973): Inspection in correspondence education now and in the future. Epistolodidaktika 1973:1, s 13-17
Karow, W (1980): Privater Fernunterricht in der Bundesrepublik Deutschland und im Ausland. Schriften zur Berufsbildungsforschung 58. Hannover: Herman Schrödel Verlag
Karow, W and Storm, U (1975): Criteria for the evaluation of distance teaching courses by state authorities. E. Ljoså (ed.): The system of distance education. Papers to the 10th ICCE International Conference, s 72-83. Malmö: ICCE
Ljoså, E (1982): State support and control of correspondence education. Epistolodidaktika 1981:2, s 41-45
Ljoså, E (1991): Legislation in Europe - the Norwegian experience. Epistolodidaktika 1991:1, s 54-64
National Home Study Council (undated): Facts about the National Home Study Council
Nilsson, K-A (1992): Utvärdering för kvalitet. En handledning för grundutbildningen. Pedagogisk utvecklingsarbete vid Lunds universitet nr. 92:179
Norges standardiseringsforbund (1991): Utvalg av Norsk Standard: Kvalitetssystemer. NS-ISO 9000 serien. Oslo
Norsk Forbund for Fjernundervisning (undated): Regler for god brevskolepraksis
NOU 1988:2: Med viten og vilje
O'Donaughue, M, Lundgren, U og Kogan, M (1988): Reviews of National Policies for Education, Norway. Paris: OECD
Ot. prp. (Report to the Odelsting) nr. 36 (1948): Om lov om brevskoler
Ot. prp. (Report to the Odelsting) nr. 55 (1991-92): Om lov om oppheving av lov 12 november 1948 om brevskular og om lov om endringer i lov 28 mai 1976 nr 35 om voksenopplæring
Presswood, R E (1972): The accreditation of correspondence colleges in Britain. Epistolodidaktika 1972:1, s 3-7
Rekkedal, T (1989): Lærer i fjernundervisning. Bekkestua: NKI
Rekkedal, T (1991): Evaluation concepts and practices at NKI, Norway. I Schümer, R (red.): Evaluation concepts and practice in selected distance educational institutions. Hagen: FernUniversität
Rekkedal T (1992): Evaluering ved noen utvalgte fjernundervisningsinstitusjoner i andre land. I SEFU: Evaluering av fjernundervisning. En analyse av teori og praksis ved noen sentrale institusjoner på feltet. Bekkestua: SEFU
Robinson, B (1992): Applying quality standards in distance and open learning. EADTU-News nr. 11, s 11-17
Rådet for videregående opplæring (1988): Skolevurdering. Et grunnlagsdokument
SATURN (1992): SATURN quality guide for open and distance learning (pilot version)
Schuemer, R (red): Evaluation concepts and practice in selected distance educational institutions. Hagen: FernUniversität
Scriven, M (1975): Evaluation bias and its control. Occasional papers series 4. Kalamazoo: Evaluation Center, Western Michigan University
SEFU (1992): Evaluering av fjernundervisning. En analyse av teori og praksis ved noen sentrale institusjoner på feltet. Bekkestua: SEFU
Skår, D (1981): State support and control of correspondence education. Epistolodidaktika 1981:2, s 32-40
SOFF (1991): Evalueringsbilder. En veiledning ved evaluering av fjernundervisning. Tromsø: SOFF
St. meld. (Report to the Storting) nr. 43 (1989-90): Mer kunnskap til flere
Studiekvalitetsutvalget (1990): Studiekvalitet. KUF
Thorpe, M (1988): Evaluating Open & Distance Learning. Harlow : Longman
Vennemann, M (1991): Legislation in Europe - the German experience. Epistolodidaktika 1991:2, s 29-32
Weinstock, N m fl (1976): Les cours par correspondance du secteur privé en Belgique. Brussel: Centre National de Sociologie du Droit Social
Wenting, H (1991): The good and bad points of the Dutch legislation. Epistolodidaktika 1991:1, s 65-69
6.1 Supervision, Management and Cooperation
6.2 Follow Up and Guidance of Authors
6.3 Choice of Media and Learning Material
6.4 Evaluation of Product under Development
6.1.1 The project manager should make the participants in the development project aware of the conditions and constraints that have been specified for the project.
6.1.2 The project manager ought to see that the development work follows a fixed plan with respect to time, resource input and other conditions. He/she should also report departures from this plan and implement corrective measures.
6.2.1 The institution should make relevant competence requirements of authors, consultants and others who are brought into the development process.
6.2.2 In accordance with Norwegian law and contract legislation, the institution ought to have clear agreements with and contractual obligations to authors, consultants and others.
6.2.3 The institution should give authors, consultants and others necessary guidance and training regarding aspects of distance education in order to assure quality in their work.
6.3.1 The institution ought to be able to justify the choice of media and use of existing learning material on the basis of the programme's goals and the students' needs and qualifications.
6.4.1 The institution ought to evaluate course material during the development process. The following factors ought to be considered:
The material ought to be evaluated by at least one person who is qualified in the subject matter in addition to the author before it is put into production.