Introduction

This guidance material is based on the material created by the E-course Quality Working Group in 2012 and last updated in 2024. The aim of the guide is to provide support in the design and creation of new e-courses and in the improvement of existing e-courses.

This guide builds on the ADDIE Instructional Design Model, which consists of five phases:

• analysis – analysis of needs, target group and context;
• design – the formulation of learning objectives, the selection of types of media to be used, the design of the structure of the course and its content, and the design of the learning process;
• development – preparation of an e-course, including learning materials and learning guides;
• implementation – the use of an e-course in a real learning process with real learners;
• evaluation – an evaluation of an e-course and its learning process, which is not a one-off activity but a process that takes place throughout the ADDIE model.

Creating a course can be individual or teamwork. In addition to subject experts, we recommend involving educational technologists, study designers, multimedia or IT specialists.

1. Analysis

The analysis phase is the basis for planning the learning process based on the ADDIE model. This phase  differs for the course included in the curriculum and for the continuing education course, as the course in the curriculum has, to a large extent, already been analysed.

Steps in the analysis phase:

• analysis of needs and formulation of general objectives – Determining the place and relevance of the course within the curriculum, its relationship with other subjects, the necessary prior knowledge, and formulating the general objectives of the course;
• analysis of conditions – – Examining the available time, budget, and resources, identifying existing materials, determining who will teach, assessing the technological tools needed and available as well as considering the potential size of the learning group.
• analysis of learners or the target group – Understanding who the learners are, their interests, prior knowledge, experience, learning skills, ability to handle technical tools, age, language proficiency, etc.;
• content analysis – formulating the learning outcomes of the course and the necessary prior knowledge. Based on this information appropriate learning activities and technological tools will be selected in the future.

Let’s take a closer look at each of these four steps.

1.1. Analysis of needs and formulation of general objectives

By analysing needs, you will find out what learners already know or are able to do in this field, and what they should know or be able to do after completing the course. When it comes to a subject in a curriculum, the needs, objectives and learning outcomes have already been mapped in the curriculum.

The needs analysis includes the following activities:

• identification of the necessary knowledge, skills and attitudes;
• assessment of the actual situation;
• prioritising the questions, themes, and problems that the course should primarily focus on.

Different methods can be used to analyse needs:

• literature reviews and interviews with experts– to identify the necessary knowledge, skills and attitudes;
• observations and surveys– to assess the actual situation;
• tests – to evaluate existing knowledge;
• group discussions – to assess the needs and expectations of learners, to identify learning goals and expected outcomes.

Learning outcomes are designed to provide learners with a clear overview of what to expect and what to focus on during the course. These outcomes should be formulated with a learner-centred approach, detailing the knowledge, skills and attitudes the learner will acquire. Additionally, the outcomes must be measurable, describing not only the skills gained, but also the context in which the learner must be able to apply these skills.

First, the general objective of the course must be formulated, which is then followed by formulating the learning outcomes. Clearly defined objectives and learning outcomes help the author of the course in deciding on the content of the course, its sequence and the effectiveness of the learning process, and in choosing the appropriate learning environment, teaching and assessment methods.

1.2. Analysis of conditions

By analysing the conditions one examines the circumstances under which the course will take place.

• What are the time, budget and other resources available to develop and carry out the course?
• What materials already exist?
• Who will be teaching?
• How is the online and face-to-face learning combined? Is hybrid learning used? Is the learning process managed by the teacher or is the course passed independently?
• Which technological means are needed and which of these are already available?
• Which online learning environment is used? What opportunities and limitations exist?

The remaining activities in the analysis of conditions relate to people (both learners and teachers), materials and technology:

• Do they have the needed technical and learning skills? If necessary, the course must also include learning about selected technologies and methods.
• Is there a team to provide technical and methodological support for the staff and learners (educational technologists, learning designers, tutors)?
• Are there people who can provide technical assistance, for example, to create video or audio materials?
• Do teachers need any training?
• Is it possible to use existing teaching materials, task descriptions, tests, etc.?
• Which materials can be adapted or translated, which ones still need to be prepared?
• How to keep track of copyrights (is it necessary to ask for permission, buy materials, etc.)?
• What digital tools can be used in the course?
• What software tools are available for creating course materials (for designing materials, composing tasks and tests, etc.)?

It is important to analyse how much time it takes for learners to cover the course content and how it fits into the time period during which the course takes place. Is it an intensive course, whether participation in this course is the main activity for the learners during a certain period of time (two-day course, one-week course, etc.), or whether the activities are scattered over a longer period of time (a semester, two months).

Creating a course is a labour-intensive process, and if teachers lack the necessary skills, some tasks may need to be outsourced. Developing complex digital learning materials often requires specialized software for graphics, audio and video processing, and animations. In such a case, it is important to determine whether free software is available or if a paid solution is needed. We recommend exploring educational material repositories, where you can either access resources for free or purchase various teaching materials and components, such as illustrations, animations, video clips, etc., for a small fee.

By the end of this stage, the planned time period and volume of the course, the budget, the software to be used, and the team of performers (teaching and support staff) as well as their relevant skills, must be identified.

1.3. Analysis of learners and target groups

The analysis of the target group is particularly important in continuing education and non-formal education. The aim is to create an understanding of future learners, i.e. to learn about their:

• background data – age, location, employment, work experience;
• motivation – learning goals, expectations, link between the course and their work;
• ability to learn – previous level of education, experience, special needs, availability of time to study, etc.;
• prior knowledge– skills and experience in a subject-related field.

The analysis of the target group answers the following questions:

• Who are the potential participants? How heterogeneous is the group?
• What are the needs, skills and prior knowledge of the learners? E-courses may provide an opportunity for educational institution to reach new target groups. In order to make decisions about the prior knowledge and skills required for the learners to complete the course, the target group may be surveyed.
• What is the maximum group size that can be effectively taught under these conditions?
• What types of technical equipment do the learners use (computers, access to the internet, necessary accessories, etc.)?
• What are the learners’ skills when it comes to using these tools?
• Do course authors need to develop the learners’ learning skills at the beginning of the course?

1.4. Content analysis

Based on the analysis of learners, course objectives and desired course content, it is advisable to create a detailed content plan. The course content should align with the course volume and not be overly extensive. If e-courses contain too much material, learners may struggle to keep up. Therefore, one should only include topics that directly help learners to achieve the learning outcomes, rather than everything the author knows about the subject. The structure and presentation of the course content significantly impact the learners’ experience.

In particular, the content must be:

• of  appropriate volume;
• corresponding  to the level of study and academic standards;
• relevant and clearly presented;
• building on prior knowledge and skills to enhance and reinforce learning;
• logically structured and sequenced;
• interactive;
• updateable in terms of both pedagogical methods and subject matter.

Once you have a comprehensive understanding of the course content, it is important to consider how to prioritise and organise it. The concepts and principles introduced to learners early on should facilitate, or at the very least not hinder, their subsequent learning.

There are several options for sequencing the content:

• thematic ranking – suitable for material covering related topics that can be taught in any order;
• chronological order – events, activities or discoveries are presented in the order in which they occurred.
• spatial order – the content is presented place by place, e.g. speaking of the human body starts from the head and moves down to the toes;
• concentric circles – the positions to be described have a common part with the following positions (e.g. a management course where the sequence of content is as follows: manager, manager and his/her team, team within the organisation, organisation in the country);
•  causal order – content is presented as a chain of causes and effects, ideal for teaching how to investigate and identify relationship (e.g. in meteorology);
• structural logic – content follows the logical structure of the subject matter;
• problem-focused sequencing – content introduces problems and encourages learners to find solutions and interpretations;
• spiral sequencing – concepts are revisited throughout the course, each time at a more complex level (e.g. overview – more in-depth study – analysis);
• reverse order –the last step is learned first, then the penultimate, etc. This option is suitable for teaching decision-making and interpretation of results. E.g. for chemical tests (test results, conducting the test, selection of an appropriate test).

The content of the course is usually developed by teachers, lecturers, trainers, etc., with support from technical experts.

Quality requirements for the e-course in the analysis phase

The course meets the needs and opportunities of the target group.

The course has clearly defined objectives and learner-centred learning outcomes.

The course content is designed to enable learners to achieve these outcomes.

2. Planning the learning process

The learning process in an online environment varies according to the tools used. Attention should be paid to how to plan, organise, coordinate, prepare, rank and supervise learning tasks and activities effectively. In this chapter, we will pay attention to the various aspects of planning the learning process to create an e-course.

Planning the overall structure of the e-course, learning activities, scheduling and technological tools occurs simultaneously. Among other things, the ways to assess the achievement of learning outcomes will be considered.

When selecting pedagogical approaches and tools, it is essential to consider the need for learners to plan their time. For instance, clarity is needed on which course components require online asynchronous (at different times) or synchronous (real-time) activities, which components are taught face-to-face in a classroom, which are learned independently in the digital environment or elsewhere.

The outcome of the design phase of the learning process is a comprehensive understanding of the e-course and time limits for various activities, ensuring the course is completed on time.

2.1. Pedagogical structure of the e-course

Choosing an appropriate pedagogical strategy is crucial for planning the learning process. The author(s) of the course must determine the best ways to utilize the available teaching methods based on the level and subject area of the course. The effectiveness of the learning process depends on both the learning materials and the activities designed to achieve the established learning outcomes.

The pedagogical structure of the e-course must consider the fact that learners have more freedom to plan their time, making independent learning skills essential. It is important to note that the workload for teachers may increase in e-learning. E-learning is not just about sharing teaching materials, creating tests and tasks, it encompasses a holistic learning process.

To ensure a diverse learning experience, it is recommended to use different learning methods, which we will introduce in the following chapter.

2.1.1. Pedagogical approaches

Current pedagogical approaches emphasise  enhancing learner engagement, motivation, autonomy and responsibility. There is also a strong focus on integrating digital tools, as they support modern learning methods, improve the efficiency of the learning process, and offer flexibility to both learners and teachers. The goal is to support learners rather than lead them – the learner should be self-directed, and proactive in finding solutions. The most common approaches and methods that can be applied to the learning process in an e-course are described below. Different learning methods can be successfully combined within one course.

Trialogical approach: The principle of trialogical learning consists of the joint work of learners in creating learning artifact with a specific focus (e.g. creating or supplementing learning material). The emphasis is on creating something new. For this purpose, various activities are implemented and technological tools are used. E.g. create a concept map, wiki page, project documentation, design prototypes, create tasks that are implemented in real life, compile a database with the resulting summaries, diagrams and conclusions. There is extensive use of digital tools suitable for co-creation and co-writing (e.g. online collaborative tools or smart apps).

The activity is organised around shared knowledge objects (idea, phenomenon, rule, principle, purpose, topic). Initially, each learner can contribute and study the topic individually, and then the results of individual creative work are used in joint activities (the opposite may also be the case). It is essential that no created object of knowledge would be rendered useless and forgotten in future activities. Hence, all created objects must be further used/improved/developed in consecutive activities.

Problem-based learning: Problem-based learning requires learners to analyse certain problems or cases, to think practically, to make decisions, and develop solution plans. Generally, problems fall into two categories:

1. Simple problems, where there is only one solution, e.g. a large proportion of mathematical and natural science problems, where formulas are applied to find solutions; language learning, where rules are followed, etc. Simple problems can be solved, for example, through online exercises and tests with automatic feedback.
2. Complex problems, where there may be multiple solutions or where there is no definite solution (it must be found and discovered by learners themselves). E.g. decision making, error search and error correction, case studies, problems requiring strategic action, design problems, solving dilemmas, etc.

Learning activities can be selected according to the type of problem. For simple problems, task-based learning is appropriate. For complex problems, methods such as research learning, project learning, group discussion, simulations, game-based learning are effective.

A problem or case typically describes a real relevant situation that interests learners, relates to their experiences and learning goals, and is sufficiently complex to warrant analysis. The problem can be created and offered by both the teacher and the learners themselves. The problem must focus on activities and the construction of new knowledge that is needed for future success in the labour market, solving problems in real terms, rather than theoretically. The knowledge acquired is integrated into the problem itself, not on the subject being taught. Learners are instructed to direct their own activities (alone and collaboratively) so that they contribute to solving the problem.

Different forms of group work or discussion are used, rather than lectures – instead of transferring new knowledge, they are constructed by the learners themselves. The teacher has a supportive and mentoring role. Digital tools support the organisation and execution of group work (discussions, agreements, deadlines, division of work) in the collection and sharing of information, etc. This can be done using the opportunities within the learning environment (forum, webinar tool, wiki) or digital communication channels such as Skype, MS Teams, Zoom, Google Meet, Messenger, etc.

Exploratory learning: Exploratory learning allows students to assume the role of a scientist, solving problems and studying phenomena. The learning approach mirrors the activities of researchers: posing research questions, formulating hypotheses, planning experiments, and proposing solutions to investigated problems. Instead of relying on teacher dictation, learners discover and construct knowledge themselves, taking responsibility for their result. For exploratory learning, digital tools can be applied to data collection (databases, tables, sensors, photo or video cameras), processing (e.g. spreadsheet applications), co-operation (project management software – Trello, Basecamp, Fleep, common cloud applications and documents – Google Drive, OneDrive), presentation of results (presentation, video, model, video, animation, blog, website, e-portfolio) or sharing (repositories, portals, websites).

Project-based learning: Project-based learning is a learner-centred approach resulting from problem learning, based on three principles of constructivism: 1) learning is context-based, 2) learners actively participate in the learning process, and 3) learners achieve their goals through social interaction and exchange of knowledge and understanding. In the case of project studies, it also starts by presenting a problem, as in the case of problem and research studies, and the main focus is on achieving a common goal through cooperation. Unlike other methods, project-based learning produces a tangible output (newspaper, magazine, poster, website, video, instructional material, model, prototype, product, service, etc.) that is shared with and/or demonstrated to fellow learners or more general audience. Digital tools can be implemented in project-based learning at virtually  any stage: brainstorming and selecting ideas, analysing opportunities and conditions, planning and sharing tasks and stages of work, organising ongoing cooperation and information exchange, and preparing and presenting the final solution.

Project studies typically involve a wide range of subjects (integration) and last for a longer period.

Flipped classroom: An inverted classroom is a method of conducting the learning process where learners independently acquire new information, while contact time is used for deepening understanding through learner-to-learner and learner-to-teacher interactions.

The teacher’s role in this approach is the selection, creation, and organisation of the information to be acquired (e.g. learning video, interactive learning material, podcasts, illustrated texts, presentations), the creation of suitable learning assignments for meetings, and the provision of guidance and personal feedback during the learning process. Learning assignments can be created in an online learning environment (e.g. tests, exercises, discussions, group work) or conducted on a face-to-face basis according to needs and opportunities.

Game-based learning: Playful learning is the use of games and simulations in the learning process to enhance the achievement of learning outcomes. The selected games must have a specific purpose and link to the learning outcomes of the course. In the game, specific goals are set or problems are solved by performing certain activities. Risks are taken, mistakes are made and learned from. A safe environment has been created in the game, where testing and failure is safe, so it is possible to try different solutions. Games increase the motivation and help learners apply what they experience in simulations to real-life situations. A game (e.g. role-playing game) can take place in both virtual and real environments. You can use a variety of quizzes (Jeopardy, Who Wants To Be A Millionaire), virtual reality and simulation environments, serious games (e.g. starting and running your own business, building and planning a city, saving energy, programming games, travelling in the past, etc.).

Playful Learning  (gamification): Playful learning means adding certain elements to the learning process (points, levels, learning marks, rankings, competitions, achievements, rewards). The aim is to increase motivation and refresh learning activities. Some elements of gaming are integrated into online learning environments, e.g. open badges, but in most cases the teacher has to figure out the system and rules to play by. A wide variety of digital tools are suitable for this: spreadsheet programs, special leaderboard creation tools, tools for creating and designing learning signs, etc.

2.1.2. Use of learning methods at the stages of the learning process

Choosing learning methods wisely involves considering the objectives, the desired learning outcomes, subject specifics, and the learners’ characteristics and experiences with various methods. It also requires evaluating the conditions of the learning environment, and the teacher’s own skills and experience with these methods. In fact, the choice of methods reflects teachers’ understanding of learning, their role and the role of learners in the learning process. The range of methods used depends on the teacher’s existing pedagogical arsenal, the expansion of which is the professional duty of every teacher. When planning feedback and evaluation, it is reasonable to consider the extent to which the learning methods align with the proposed assessment methods.

Learning methods can also be grouped according to the structure of the learning process, taking into account at which stage of the learning process it is more appropriate to use one or the other method.

Depending on the pedagogical approach, the stages of the learning process may be intertwined or take  place in different sequences (e.g. in case of problem-based learning, the acquisition of new information takes place during the task-solving process).

At the beginning of the learning process it is important to use learning methods (introduction and warm-up tasks) that help learners to tune in to learning, attract interest, set goals or review prior knowledge (e.g. brainstorming, definition of basic concepts). The aim of familiarisation and warm-up exercises is to help learners to get to know each other, create positive group atmosphere, promote relaxation, break down social barriers and typical thinking patterns, restore energy, and connect with both the subject and the group. Familiarisation exercises foster a learning community with shared understanding, responsibility and vision, where members support each other throughout the learning process. This is particularly important for e-learning.

In the new learning phase learners acquire new information by reading a text, listening to or viewing material, or experiencing something firsthand. At this stage, it is crucial to help learners deepen their understanding of the new material and connect it with what they already know. Meaning is actively created rather than passively received. New knowledge becomes ingrained when it is concretised with examples and associated with prior knowledge. For example, the process of creating connections occurs when new knowledge is applied to problem-solving. Different tasks should be tackled, using knowledge in various contexts.

Some examples of possible individual tasks:

• written tasks (preparation of a term paper, answering questions, formulating summaries, reflecting on what has been learned, supplementing the material);
• tests (retrieval practice, where the answers are checked by the learners themselves or by a computer; graded tests, preparation of questions and tests);
• participation in online discussions;
• practical work.

At each stage of the learning process, in addition to individual learning methods, group working methods can also be used. In group work, learners work together in small groups for a common goal. In the case of group work, all members of the group shall be jointly responsible for carrying out the task. Teamwork requires cooperation, the exchanging of skills and knowledge, and the division of labour in the performance of tasks.

Group work can have different objectives and the most appropriate group working method should be chosen to achieve each objective:

• presentation of new material, acquisition of new information (seminar, mosaic);
• supporting the development of analytical skills (discussions, case analyses, simulations);
• shaping social skills, promoting cooperation, shaping democratic habits and learning democracy (group discussions, press conference, symposium, discussions, debate, 5-minute production);
• highlighting and recognising different perceptions, views and attitudes, contributing one’s experience to group knowledge (academic dispute, role-play, interview, line formation);
• generating new ideas, problem solving skills (brainstorming, case analysis, open space method);
• iseenese tundmaõppimise võimaluste loomine õppijatele (iseseisvuse tunnetamine, oma tugevuste ja nõrkuste tundmaõppimine).creating opportunities for learners to learn about themselves (perceiving their independence, discovering their strengths and weaknesses).

Examples of group work:

• A private forum is used, where members of the group post their thoughts about the problem. At the end, one of the group members summarises the posts and submits it on behalf of the group.
• Using the group work tool, all group members write their thoughts in a common document. At the end, one of the members of the group summarises the opinions, indicating in brackets after the opinions and comments who the author of the section was.

In learning environments forums, wikis, chat rooms, webinar tools, etc., can be used for group work. A variety of online collaboration tools can be used.

In the reflection phase a holistic picture of learning is created, what has been learned is used in new connections and new situations, one’s attitude towards it is shaped, and perceptions are changed. Reflection is necessary for learners to embrace new knowledge. New knowledge is more entrenched when used and linked to prior knowledge. This requires empowering learners to express new knowledge and ideas in their own words, as well as assigning tasks where learners can creatively apply new knowledge and skills. An important task in the reflection phase is to activate exchanges between learners so that they can compare their new knowledge and understanding with those of other learners. The reflection process could contribute to the development of a personal relationship between learners and learning outcomes, which can be supported through learning records (e-portfolio), study diaries and other tasks. It is also important to create a complete picture, place new knowledge in the existing knowledge system (production of summaries, concept maps and schemes, answering questions, completing sentences, compiling a study diary and a digital skills development map).

2.1.3. Development of general competences

When designing the e-course, it is advisable to consider that not only the subject area but also the general competencies need to be developed, namely:

• learning skills 
• social skills and civic competences
• initiative and entrepreneurship
• mother tongue skills 
• foreign language skills
• digital competence 
• mathematics competence and basic knowledge of science and technology
• cultural awareness and competence

The best way to support the development of learners’ learning skills and digital competences is through e-course.

Learning skills are skills that, regardless of the field of study, have a direct impact on the effectiveness of learning. For example, learning skills include functional reading, concentration, organisation of knowledge, self-motivation, time planning, finding additional information, independent work and self-reflection skills. As the environment around us is changing at an ever-increasing pace, adaptive and self-management skills, social skills, critical thinking skills and creativity are also important learning skills. Although most people acquire a large part of them with a simple life experience, they are still skills that can also be developed fully consciously and purposefully. For example, it is possible to increase the ability of learners to read functionally by giving them the task of marking an important text or by allowing them to write a brief summary of the text they have read. When developing learning skills, it must be taken into account that different methods are suitable to different learners and there should be freedom to make choices. For example, the development of time management skills is not supported by providing a strict timetable and imposing penalties for exceeding deadlines. It is much more effective to allow learners to create their own individual schedule at the beginning of the course and to motivate them to follow it in the future.

Digital competences is necessary for participation in today’s digital society. Educational institutions in the Member States and candidate countries of the European Union will be able to develop learners’ digital competences on the basis of the Digital Competence Framework for Citizens (DigComp). Learning through an e-course supports the development of digital skills to a very large extent in all areas of competence of the learner’s DigComp model:

1. Information and data literacy – searching, browsing, filtering, evaluating and managing data, information and digital content.
2. Communication and cooperation in digital environments – communication using digital technologies, sharing data, information and digital content, active citizenship, cooperation and polite behaviour in the digital environment, management of digital identity.
3. Digital content creation – development and adaptation of digital content, copyright and licenses, programming.
4. Digital security – protection of digital devices, protection of personal data and privacy, protection of health and well-being, environmental protection.
5. Problem solving – solving technical failures, selection of digital technologies, innovation through digital solutions, assessment and development of digital competence.

The creators of the e-course could take into account the above areas and develop the digital competences of learners throughout the learning process. For this purpose, correct and high-quality learning materials are created, diverse learning activities are planned, using different digital tools and allowing learners to be active content creators.

2.1.4. Designing interactions between learners, building a learning community

In face-to-face learning learners have the opportunity to meet in one room, they often know each other and are able to jointly solve problems that arise in learning process. On the e-course teachers can organise peer-to-peer interaction and joint problem-solving between learners using digital communication tools.

As a rule, joint discussions between learners about course topics do not arise spontaneously, but with the initiative and support of the course provider.

In order to get acquainted with each other and facilitate the emergence of a learning community, suitable activities can be planned for the course. One option is, for example, to let all participants write a self-promotion in the course forum. At the same time, it is possible to start the course immediately with (thematic) group work, which forces learners to contact at least some of their coursemates. Of course, there are still other opportunities, and many of them can always be used at the same time. In any case, it should not be forgotten that this type of planned activity requires proper instructions and support from the teacher.

During the course, it is recommended that discussions, peer review of homework, debates, group works and other techniques are used that guide learners to communicate with each other and support each other in acquiring the subject.

2.2. Assessment and evaluation planning

When designing an evaluation, it is important to decide which evaluation methods and strategies to use. The assessment must be based on learning outcomes and must be planned before preparing study materials. It is necessary to determine which type of assessment is required, when the evaluation will take place, and which assessment methods (test, project, task, portfolio of works, etc.) are used.

Key issues indeveloping an evaluation strategy:

• Is the achievement of all learning outcomes measured?
• How is it assessed (with which activities)?
• When will the assessment take place during the course?
• What are the evaluation criteria?
• Are evaluation models or rubrics used?
• Who is the assessor (the learners themself, automatic assessment, the teacher, the fellow learner)?
• What feedback is given to the learner (scores, comments, model responses, etc.)?

Most e-courses include a variety of tasks that are given to learners to complete at regular intervals throughout the learning process. The obligation to complete tasks for a limited period determines the pace of study during the course, motivates learners to work continuously throughout the learning process, and helps them get feedback on their academic success. It must be borne in mind that tasks are only effective if students receive constructive feedback. Reporting a score or a number of points only reduces the usefulness and motivational value of the task.

Two evaluation activities are distinguished: performance or summative assessment and formative assessment. The summative assessment of learning is a tool used to determine the level of knowledge of learners and to determine the achievement of learning outcomes. Formative assessment for learning is a tool that helps learners learn more effectively and teachers analyse the learning process.

The following assessment options are available for online learning:

• self–assessment– tests and assessment rubrics for self-reflection have been added to the learning materials;
• computer assessment – usually a test of multiple choice questions, as well as all other types of questions that can be automatically assessed using a computer: short answers, matching, ranking, filling in gaps, solving crosswords, single-response calculation tasks. An increasing number of interactive multimedia exercises, self-tests and assignments are available in ready-made format. Automatic evaluation can be used in both self-assessment and graded evaluation;
• tutor assessment – written assignments (essays, reports, projects, problem solutions, etc.) that are assessed and given feedback by the instructor. This is usually the most basic form of assessment, the disadvantage of which is the high workload of the instructor, which is why it is desirable to combine different methods of assessment.
• peer assessment – this includes informal comments given to fellow students in group work and formal feedback to individual assignments.

The above methods of assessment may also be used in combination within the course.

Learners should be given an overview of the objectives, modalities and criteria of the assessment. Criteria can be used to create an evaluation model or matrix that makes the assessment more transparent and allows for self-assessment. Evaluation models and rubrics can also be created directly in a digital learning environment where it is supported, e.g. Moodle, Google Classroom, etc. Learners are only informed individually about their own learning outcomes; non-personalised learning outcomes can be shown to all learners.

2.3. Selection of digital tools

The choice of digital tools depends on learning objectives, content and activities. To select them, we recommend involving educational technologists, learning designers or other competent colleagues.

When choosing digital tools, account must be taken of:

• availability,
• ease of use,
• opportunities for active inclusion of learners,
• interactivity,
• adaptability,
• suitability for cooperation,
• compliance with standards.

Today’s Learning Management Systems contain a variety of tools and options for structuring the course, presenting content, communicating, performing learning activities, assessment and managing the course.

a) Other online tools can be used to create materials in addition to learning management systems. At the planning stage, it is necessary to consider which media are used to create educational materials.

b) Communication tools – have a major impact on the emergence of a sense of community and the establishment of academic relationships. The effective use of these tools may depend on whether or not a particular learner can complete the course. Therefore, course planners must devote a great deal of attention to the use of communication tools. Communication tools are divided into two classes:

• synchronous tools – real-time tools (web seminar facilities, chat room);
• asynchronous tools – tools used at different times (forum, e-mail);

c) Cooperation tools – exist in online learning environments (forum, task tool that can be used for group work, wiki). In addition, social software (e.g. wikis, blog, Trello, YouTube, Google Docs, etc.) can be used as collaborative tools.

During the planning phase, it is necessary to plan different parts of the learning process at the same time and to take into account the learning workload for learners. 1ECTS (European Credit Transfer and Accumulation System) means 26 (twenty-six) hours of study. In a blended course, learners need to be given a comprehensive overview of what is happening in both face-to-face and e-learning. The following table helps to plan the learning process as a whole.

The calculation of the time spent on learning is based on the time spent by the average learner on various activities, which in the auditorium and in e-learning must give the time prescribed by the ECTS.

Quality requirements for the e-course in the planning phase of the learning process

The course syllabus or programme of study is based on the requirements of the educational institution.

The prerequisites or prior knowledge and skills required for students to participate in the course are presented.

Course learning activities and assessment principles support the achievement of learning outcomes.

The principles for assessment and feedback are provided to learners.

The study materials and activities of the course correspond to the volume of the course.

The development of learning skills is supported (students are directed to reflect on what they have learned, designing of time management skills is supported, etc.).

Technological tools support the implementation of the learning process.

The principle of integrity has been taken into account in the design of the course so that the learning process is reflected in the online learning environment as a whole.

3. Development of the course

Course development is the process of creating a comprehensive course (educational materials, tests, assignments, study guides, etc.) using digital tools. To this end, it is advisable to involve experts from several fields. In universities and vocational training institutions, academic staff are provided with methodological and technical support in the preparation of the e-course by educational technologists, learning designers, multimedia and information technology specialists. A teacher’s assistant in creating and testing e-courses is an educational technologist or learning designer who:

• provides recommendations on how to prepare and design learning materials;
• assists in choosing appropriate methodologies and technologies for courses;
• assists in the technical design of the course;
• helps to plan learners’ support during the different stages of the course;
• helps to improve courses based on feedback received on the course.

When creating an e-course and using the creations of other authors, the Copyright Act must be taken into account. Usually, most of the e-courses are completed as a result of teamwork rather than by a single author. The course must include everyone’s contribution. Each material must bear an unambiguous and visible label indicating the author(s) of the material. For large-scale use of the works of other authors (translation, etc.), permission must be sought. Copyright and license rights must be protected and any restrictions on the use of materials created by third parties (including learners) must be respected. As already stated above, not all educational materials need to be created by the author of the course, open materials equipped with a corresponding license (e.g. Creative Commons) may be used. You may also refer to licensed materials available online, but make sure they are available throughout the course. There is an obligation to refer to the name of the author of the work used (if indicated in the work), to the name and source of publication of the work, and then it is permitted to:

1. cite and reference a lawfully published work to a motivated extent, in compliance with the obligation to correctly convey the thought of the work being cited or referenced as a whole;
2. the use of a lawfully published work as illustrative material for educational and scientific purposes on a motivated scale and provided that such use does not pursue commercial purposes;
3. reproduction of a lawfully published work for educational and scientific purposes on a motivated scale in educational and scientific institutions whose activities do not pursue commercial purposes.

3.1. Development of learning materials

When developing learning materials, care must be taken to ensure that they are suitable for use in self-study by learners. This allows learners to choose a suitable time and place to study. Self-learning materials may include part of the course’s core material, but they can also provide a valuable addition to subjects that are recommended but not mandatory to complete the course. Self-learning materials may be tailored to the needs of the course or programme, but each set of materials must be comprehensive, with specific objectives and measurable results. In the case of self-learning materials, interactive methods must be applied as much as possible so that learners can self-assess their progress towards learning outcomes.

Educational materials can be compiled using already existing materials that can be found in different repositories.

When developing educational materials, it is recommended to follow the following principles:

• Different media, digital tools and solutions are used to present educational materials: text-based materials (e.g. consumables, guides), slides (as support for auditory learning), illustrations (drawings, images, graphs), audio materials, video materials, integrated solutions (e.g. slides combined with audio or video), animations, web references, etc.;
• The learning content takes into account the needs and specificities of learners (including learners with special needs). The specialised equipment and software needed to participate in the course is provided to all learners (e.g. test versions of the programmes, technical equipment to be lent out);
• The learning content is reliable, accurate and up-to-date. The identity of the authors and the educational institution is clearly stated; reference is made to the works of other authors; the educational materials bear an indication of its latest update; all web links and videos are referred to in the course work;
• The learning material can be used with various browsers (Mozilla Firefox, Safari, Google Chrome, Edge) and operating systems (Microsoft Windows, Linux, Mac OS, iOS, Android). If there are specific technical conditions for the use of learning materials, they must be included in the course (e.g. in the Study Guide);
• The learning material can be used both on a computer and on a smart device;
• The learning material is linguistically correct;
• The learning material is suitably divided into smaller parts.

The following provides more precise and specific recommendations for creating text, graphics, audio, videos, animations, presentations, and tests.

3.1.1. Text-based learning material and its design

Text is the most common way to communicate information. It is more understandable to the reader (learner) when written text follows the recommendations of drafting, structuring and designing it. Digital learning material is mostly interactive text that contains web links and is presented in HTML format (websites).

Interactive texts allow to use learner-activating techniques (e.g. automatic feedback questions, the appearance of explanatory texts when the cursor moves to the picture) and embedded content that illustrates text and keeps learners active (e.g. YouTube video). Nowadays, fewer and fewer long linear texts are being read, and in learning, shorter articulated interactive text is preferred, combined with illustrative and explanatory material (text, figure, video, audio, interactive questions).

In the e-course, textual learning material is usually presented using LMS tools, like page or book, which contain an editor that can be used to design the learning material. Text is also often transmitted as files: materials created in presentation programs and text editors, distributed e.g. via PDF, DOC, PPT or EPUB formats in repositories or learning environments.

It is recommended that learning materials are created in a generally accepted format, for which the learner does not have to incur additional expenses, or the course author/executor provides the necessary special software to work with the materials. If the costs of acquiring specific software are to be borne by the learner, this must be clearly explained to the learner already before the start of the course (hidden costs). The most well-known text file formats are TXT, DOC, DOCX, RTF and ODT. These formats do not guarantee that the design of the text is preserved if you open them with other programs. To distribute texts in authentic form (particularly if they contain tables, drawings and images), the PDF format (Portable Document Format) is suitable as it preserves the original design of the material and can be viewed with all the most common operating systems, mobile devices and e-readers. PDF files can be edited with special software.

Suggestions for drafting and articulating easy-to-read and grasp text:

• the title generates interest and reflects what is said in the text;
• the introductory paragraph summarises the main content and the spirit of the text;
• lists are used;
• short paragraphs are used;
• the present form and the specific way of speaking is used;
• emphasis is made by using bold and/or italics, emphasis is placed on individual words rather than longer blocks of text;
• text underlining should be avoided unless it is a web reference;
• for external web links, preferably the title of the linked material, the author and, if possible, the full web address (URL) are marked;
• typing blocks of text with capital letters is avoided.

The following recommendations should be taken into account when drafting the text:

• when choosing a font, preference is given to the sans-serif style, which is easier to read on the screen (e.g. Calibri, Verdana, Arial);
• left or parallel alignment should be used to provide a contact point for the eye and to improve legibility;
• external online references are usually blue in colour and differ from internal references in the same learning material;
• similar content elements are designed in the same way (e.g. lists, rules, definitions, examples, tasks, theorems);
• background images and bright background colours should be avoided;
• the colours of the text and the background are distinguished in strong contrast (black on a white background);

The text, drawn up in accordance with these principles, attracts the attention of the learner and helps them to better understand the material. Failure to comply with the principles may hinder the learner’s work with educational materials (especially students with low vision or colour blindness).

Some learners may want to read lecture materials on their computer, without using the Internet, or on paper. This may also be necessary if the use of online materials is disrupted. Therefore, learning materials should be designed in such a way as to enable the learner, if desired, to store or print the learning material on their computer with minimal effort.

3.1.2. Creating graphics

It is recommended that textual learning material are combined with graphics. Graphics (drawings, images, graphs) can present the content you learn, illustrate it, catch the attention of learners and help them to better understand complex relationships. The use of graphics in educational materials is particularly useful for learners with a visual memory. It is important to make sure that the graphics are fit for purpose, include explanatory text, if necessary, and refer to its author. Design must take into account well-established design principles, must not dispel the learner’s attention or increase the workload. When adding graphics, you must also enter an alternative text (ALT) that is visible when the graphics are not displayed or cannot be seen (e.g. visually impaired users using a screen reader).

The most common browsers support image formats such as JPG, PNG and GIF. The last two allow the use of transparent backgrounds, such images can be placed on a coloured background so that the image’s own background does not stand out. JPG (or JPEG) is suitable for photos, as it allows you to keep a beautiful and detailed image even with a very small file size. However, this format is not suitable for graphs and drawings, where the use of, e.g. PNG or GIF format is recommended.

To better distinguish graphics from text, it is recommended to leave free space between text and image (for example, 5–10 pixels or 1–2 letterspaces). It speeds up reading, while at the same time giving the material space. When adding graphics to the educational material, attention should be paid to the size of the image file: if the learning material with illustrations is viewed via the Internet, then very large and high-resolution images are not suitable, the size of the images should be within the limits of 1–2 MB – then the material page opens quickly and smoothly. If images are added to tests or assignments that are taken in real time (e.g. exam tests, Olympiads, assessment tests), image volumes could be even smaller, preferably below 100 KB, as the network load on one web page will increase several times and the image will be loaded more slowly. If for some reason a large image size and high resolution are necessary, then the size of the image must be specified separately in the additional information.

3.1.3. Creating sound

Sound or audio recordings can be used to add explanations and sound samples to the educational material, but also to present the necessary educational content (podcast, audio lecture). The audio material is particularly suitable for a learner with a hearing memory or visually impaired learners. Creating audio material is not difficult, but it is harder to change than text. In language learning, the use of sound helps you learn pronunciation and get used to the sound of language.

To use sound recordings in educational materials and to listen to them smoothly, it is important to monitor the file formats used. The most common audio recording formats are MP3 and WAV, which can be played by all computers and mobile devices.

MP3 is a packaged file format, making the file in this format of small size and able to quickly move on the Internet. When creating an audio file in MP3 format, you need to make sure that the minimum transmission speed is 64 kbit/s for a speech and at least 128 kbit/s in other cases. The higher the transmission speed, the higher the quality of the result, but the bigger the size of the audio file.

WAV is an unpackaged and high quality audio recording format that is good to use for storing original material. It is cumbersome for distribution on the Internet.

Professional audio recordings, dictaphones, smartphones and built-in microphones can be used to create sound recordings. There are various applications for recording sound:

• Sound Recorder (Windows Free App)
• Garage Band (Mac OS default app)
• Audacity (Windows, Mac OS, Linux)
• Vocaroo https://vocaroo.com/  (online software)

When recording a sound clip, attention should be paid to finding a suitable tone of voice and manner of speech (appropriate pace, diction). The recording must be done using a proper microphone or headset (headphones and microphones) and make sure that there is no disturbing background noise.

The following common software can be used to process sound recordings:

• Audacity (free) https://www.audacityteam.org/ (Windows, Mac OS, Linux) – audio recording and editing (cutting, etc.) is easy and requires minimal effort
• Sound Trap https://www.soundtrap.com/ ( Windows, Mac OS)

The good sound recording for learning purposes, is:

• functional (purposeful, expedient, working, etc.);
• short enough (up to 10 minutes) for the listener to be able to follow;
• controlled by the listener (must be able to stop or scroll forward/back if desired).

Audio tags can be uploaded to cloud applications that make it easy to play them directly, share them as a link, or download them as a file (such as SoundCloud, Dropbox, OneDrive, Google Drive). SoundCloud is already built in and it is convenient to attach clips to the website. On smart devices, you often need to install an additional app to listen to the clip.

3.1.4. Creating a video

Video material can provide learners with experiences and knowledge that they would otherwise not have access to (e.g. the service situation in different cultural spaces, the presence of an expert, a visit to an underground mine, etc.). Video is an excellent tool to monitor processes over time, either accelerated or slowed down, and is particularly suitable for learners with a visual and auditory memory. The creation and modification of video material is time- and resource-consuming and may impose specific requirements for computer hardware and software, assuming that the creator of the material is familiar with video editing software. This is why video can be used for topics where the content is changing slowly.

When creating video material, four stages are distinguished:

• planning,
• recording,
• editing,
• distribution.

Planning

The creation of a video recording starts with planning, supported by the following questions:

• Is it necessary to stage any situation in order to illustrate the learning content?
• Is it necessary to film the performer and/or any activity, process, equipment, objects, etc.?
• Is it important to record what is happening on the computer screen?
• What is important when filming, what should the cameraman (the filmmaker) pay attention to?

During the design process, a scenario (frames, texts, captions, graphics, etc.) corresponding to the objectives will be developed to help decide which equipment and software to use when creating a video.

Recording

You need to have enough time to film the video and find a place where the necessary lighting conditions, background for frame compositions and possibilities for recording high-quality sound are available. Video footage can be created without professional equipment, but it is definitely necessary to have a photo or camcorder and a tripod. A smartphone or tablet can also be used to film, but the quality of the recording produced with them is not equivalent to that provided by a professional camera. Where possible, it is a good idea to seek the help of specialists (e.g. educational technologist or multimedia specialist) and to involve colleagues or learners who take care of different tasks (lighting, sound, subtitles, etc.).

Most devices can use automatic settings, but before shooting you should set the camera’s white balance and monitor the focus and aperture during filming. If necessary, several takes are made of the scene, from which the best will be selected during the assembly process.

It is possible to record what is happening on the computer screen with special screen recording software or special hardware (e.g. Echo360, Panopto), which records all information sent to the video output of the computer. Before creating a screen video, it is worth thinking carefully about the activities and explanatory text that are important for recording and practicing everything a couple of times. To create a screen video, the following software is suitable:

• Camtasia Studio: https://www.techsmith.com/camtasia/  (paid, Windows, Mac OS)
• Panopto: http://panopto.com (paid, Windows, Mac OS)
• Snagit: https://www.techsmith.com/snagit  (paid but affordable, free trial 30 days, Windows, Mac OS)
• ScreenPal (former Screencast-o-Matic): https://screenpal.com/ (free of charge in limited amount)
• Mango Apps Recorder https://workplace.mangoapps.com/mango-recorder  (free of charge in limited amount, making screenshots and screen recordings)
• Screencastify https://www.screencastify.com/m (free)
• OBS Studio https://obsproject.com/  (free, available for different platforms)

For example, BlackMagic ATEM Television Studio or ECHO360 Lecture Capture can record a signal sent to a video output. If you want to show what is written on a tablet or paper in a video recording, an interactive plaque (such as Smartboard, Wacom, Promethean), a document camera or an interactive pen (Papershow, eBeam, Ben Q) may be of help.

Video editing

In most cases, the recorded material needs to be edited (frame cutting, background sound, graphics, special effects and captions) using special software.

The Clipchamp app is recommended for Windows 365 users and is usually installed by default (if not, you can download it from Windows Store). It allows basic and convenient video editing and can also be used online (https://clipchamp.com/en/). Other free programs may also be used, such as Filmora (Windows/Mac OS), iMovie (Mac OS), Shotcut, OpenShot Video Editor, Da Vinci Resolve, online tools like WeVideo (https://www.wevideo.com) and Vimeo (https://vimeo.com/features/video-editor) and YouTube (https://www.youtube.com/creators/youtube-create/) will also do the job.

For more professional video editing, more complex software is used:

• Adobe Premiere Pro: https://www.adobe.com/ee/ (Windows, Mac OS)
• Avid Artist Suite: https://apps.avid.com/avid-everywhere/products/artist-suite/  (Windows)
• Final Cut Pro: https://www.apple.com/final-cut-pro/  (Mac OS)

The means of processing a screen video are usually the same as the one they were recorded with. The most common means of recording and processing a screen video are:

• Camtasia Studio: http://www.techsmith.com/camtasia.html
• ScreenPal: https://screenpal.com/
• Snagit: Snagit: https://www.techsmith.com/snagit
• Panopto: http://www.panopto.com

It should be taken into account that the time required to process the video is much longer than the recording time, as it is often necessary to review the material several times. The more precise and thoughtful the preliminary work (scenario) is, the shorter the processing time.

Add subtitles

If video content needs to be presented in other languages (translation) or as an additional medium for people with hearing disabilities, subtitles can be added to the video footage. They can be added using most video editing programs, but in this case it will be difficult to remove them later. It’s smarter to use a special subtitle creation software that allows you to create a separate, integrated subtitle file, which when imported, will place the subtitles automatically to the right position in the video. In addition, in this case, the viewer will have the opportunity to choose subtitles in their own language. Examples of free subtitle creation software are Subtitle Workshop (http://subworkshop.sourceforge.net), VisualSubSync (https://www.visualsubsync.org/home ), or Aegisub (https://aegisub.org/ ). In addition, 3Play https://www.3playmedia.com/services/captioning/ and Amara software (https://amara.org/en/) can be used, which are supported by the Vimeo  platform.

If videos are uploaded to YouTube, subtitles can be created directly there. YouTube has a video editing editor called YouTube Studio, which allows you to add subtitles, comments or process it like in a standard video editing software.

Distribution of the video

Before rendering a video work file into a film, attention should be paid to the metadata, dimensions, volume, duration and format of the video clip.

The video clip must include the key metadata: title, author, creation time, purpose or context that can be presented in titles. The materials of other authors (music, pictures) used in the video must be correctly referenced.

Video size refers to the dimensions of the video in pixels and the data volume of the video file in megabytes. The dimensions of the video are directly related to the volume of data in the video – the higher the resolution and duration of the video, the more data it takes every second. The recommended minimum dimensions for the video are 1280 x 720 pixels.

The duration of the video must be minimal in order to present the topic to be addressed. The length of the video clip for learning purposes should be no longer than 10 minutes. In the case of a longer video lecture, it is worth considering dividing it into shorter parts.

In order to distribute a video file online, the video must be stored in a format that can be viewed in any operating system. The most widely supported and suitable format for online distribution is MP4.

It is recommended to distribute video files through video repositories so that they can be viewed on different devices without downloading. The most well-known video repositories are:

• YouTube: http://www.youtube.com
• Vimeo: http://vimeo.com

The video file added to the repository shall be shared as a link or put with embed code directly into the learning material.

3.1.5. Additional materials

The study materials include references to additional materials supporting the content of the course (books, databases, web references, etc.). The course presentation must provide information on the resources made available to learners and the channels for accessing them.

When organising the course, it must be ensured that participants have access to all the digital resources needed to complete the course. The educational institution’s rules on access rights to journals and databases may specifically define the requirements regarding access rights for learners outside the educational establishment’s network. Online learners do not need to receive full access rights immediately, but it is important that they have access to all sources related to a particular course. The course presentation must provide information on the resources and channels available to learners, with the necessary passwords (e.g. temporary user ID to access material from the University’s intranet) to be provided securely to learners.

3.1.6. Tasks, tests

The task guides shall clearly describe the individual activities of the learner or the activities assigned to the group. It may be a report, an essay, a project, a study, an analysis, writing a thesis, etc. The topics of the tasks (compulsory or optional), recommendations for the organisation of the work, references to materials, deadline and manner of submission, evaluation criteria, etc., must be indicated.

Online learning environments include a variety of tools designed for learners to perform homework individually or in a group. For example, a assignment tool, a forum, a workshop, or a wiki can be used. The teacher provides a description of the task and instructions for completing the task, sees the answers provided, evaluates and gives feedback.

One way to check the learner’s knowledge is the use of retrieval practice tests, in which the learner is automatically informed about whether the answer was correct or incorrect, the correct answer or reference is provided, and the necessary comments are provided. The most time-consuming element when preparing such tests is the substantive creation of the tests, together with a suitable assessment system. In the future, the testing of a large number of students will entail minimal work costs.

Tests can be either for retrieval practice or for graded assessment.  Retrieval practice testsserve as self-checks for learners. They can be performed an unlimited number of times and the test results are not evaluated or recorded. A results test can be performed the number of times specified by the teacher, and the results of the test are assessed and recorded.

When creating tests, it is usually possible to create different types of questions: e.g. question with a computed answer, question with combined answers, filling gaps, jumbled sentence, alignment, multiple-choice question, long answer question, short answer question, true/false question, etc.

3.1.7. Learning guides

The course must have learning guidelines that provide clear instructions for learners to complete the course, describing the objectives of the subject or course, topics, activities, schedules, mandatory requirements, assignments and evaluation. At the same time, learning guidelines do not have to duplicate the information provided in the syllabus, but rather supplement it.

When presenting a general pedagogical approach and using tools, it should be taken into account that learners can prepare and plan learning activities. For example, it should be clear which components of the course require online or synchronous communication, which components are taught in an auditorium or otherwise, etc.

Course Learning Guidelines generally include:

• a description of the learning process, which includes an overview of online and face-to-face activities and independent work;
• schedule of subjects/periods/modules of the course, including the dates of the face-to-face training (at the auditorium and in e-learning); the time limits for the submission of the tasks and the dates of the tests and examinations;
• a more detailed description of the subjects/periods/modules of the course:
  • what educational materials should be read (by chapter, page);
  • what kind of independent work is required for a certain period of time;
  • the tasks to be carried out;
  • which activities are mandatory to participate in. If group work has been organised during the course, also describe the responsibilities of each group during that period;
• description of tasks. Describe in greater detail the individual tasks of the learner or the tasks assigned to the group. It may be a report, an essay, a project, a study, an analysis, writing a thesis, etc. Bring out
  • topics of tasks (mandatory or optional);
  • recommendations for the organisation of work;
  • references to materials;
  • deadline for submission;
  • the mode of reporting;
  • assessment criteria;
• mandatory requirements. Indicate the requirements required to pass an evaluation/examination or to obtain a certificate attesting to completion of the course. E.g. to take part in seminars, discussions and webinars, to write a term paper on a given topic and to take a test at the end of the course;
• guide to the use of the online learning environment
  • hardware and software requirements;
  • introduction of the learning environment tools used during the course and instruction on how to use it.

3.2. Use of existing materials

Learning materials don’t always have to be created on your own. For many of the topics we teach, our colleagues have already produced remarkably good materials that can also be used as part of the course being created. The desire to use such materials immediately raises two questions:

• Where can I find such materials?
• Can I use them?

The first question is easy to answer: on the Internet, of course. But the material found in search engines is not mostly learning material. On the Internet, however, there are special databases created to store learning materials and learning objects, e.g. repositories.

A learning object is a digital resource that is reusable in different learning contexts and supports learning. When creating learning objects, the aim is to re-use them. Therefore, learning objects are designed to use as little prior knowledge as possible and are not strongly interlinked, so that they can be placed more easily in the context of the course being created in new courses.

The most well-known international repositories for learning objects:

• Merlot (Multimedia Educational Resource for Learning and Online Teaching); http://www.merlot.org), one of the most well-known and comprehensive repositories, contains thousands of peer-reviewed teaching materials, access by type of material, search terms and fields.
• Wisconsin on-line (http://www.wisc-online.com), in particular teaching materials developed by WTCS (Wisconsin Technical College System) and made available for use by all interested parties.
• MIT Open CourseWare (http://ocw.mit.edu ), is also study material from a specific university (the Massachusetts Institute of Technology) that is made available to all interested parties for learning and use.

When using existing materials, you must make sure that you have the right to use these materials. It is useful to consult Creative Commons licenses https://creativecommons.org . If you want to search for Creative Commons license protected materials, use the search portal https://search.creativecommons.org.

3.3. Technical execution of the course

There are a number of different ways to create an e-course. The easiest way for a teacher is to realise the course in some comprehensive online learning environments. There are a number of e-learning environments in use: Edmodo, Canvas, Schoology, ILIAS, Google Classroom, but the most common is the Moodle environment.

The term ‘online learning environment’ covers all opportunities for managing the learning process, from the delivery of course learning materials to learners to the conduct of a comprehensive learning process. Within an educational institution, the online learning environment can be linked to a wide range of existing systems, e.g. the registration system for learners’ subjects.

Online learning environments are used:

• to present the teaching materials of the course, which may include text, images, videos and audio;
• for communication between learners and teachers, and between learners through a forum, mailbox, chat room or webinar tool;
• to facilitate learning by using search systems, dictionaries, photo databases, references to online materials;
• assessment of learners (tests and tasks);
• administration of the course (management of learners and learning outcomes, statistics on learners’ activities).

3.4. Course testing for smart device

The fact that the course has to work on a computer is self-evident. It is advisable to bear in mind that about half of the learners participate in e-courses via a smart device. The smart device must open educational materials as well as other learning content and carry out different activities, e.g. tests, tasks, forums. The course must be tested preferably with a smart device that uses both iOS and Android operating systems. If it becomes evident during the test that not all course activities are feasible with a smart device or the content viewable, information about this must be included in the learning guides.

3.5. Testing the course

The purpose of piloting the course is to determine whether the teaching methods and materials meet the stated objectives. Piloting makes it possible to identify the strengths and weaknesses of the course, to understand what technical and organisational problems arise during the course, and to improve the course according to the needs and wishes of the learners.

Since piloting is an important part of creating a course, it must be done throughout the course creation process for each step. Once the course is ready, a separate pilot should be carried out on the course as a whole in order to answer the question of whether the course meets its objectives. There are many methods for comprehensively piloting the course, which can be used alone or in combination with each other. The most common methods are as follows:

1. Review of the course with a more experienced colleague. Together with a colleague experienced in e-learning or an educational technologist, the main part of the course (materials, assignments, tests, guides, planned group work, etc.) is systematically reviewed;
2. Piloting with target groups. Some prospective learners will be selected who will receive specific tasks to test the course before starting the course with a large group. The number of testers represented could be 2–3 and they could be from different backgrounds (e.g. one experienced in e-learning, another a beginner; one strong in the subject, the other weak, etc.). This will provide random feedback on whether the tasks are understandable, whether the test questions are also unambiguous from the learner’s point of view, etc. Helping methods may be used: for example, sit in the same room as the tester and ask him to complete some part of the course while thinking out loud. The same can be done in e-learning via an audio connection, etc.;
3. Use of a dedicated pilot team. When the course is conducted for the first time, a special group will be selected, who, among other tasks, will be asked to evaluate the course from the perspective of the target audience and to provide comprehensive feedback to the course creators. For example, teachers with a strong pedagogical background, who are able to assess the effectiveness of teaching materials, tasks, etc., and provide adequate feedback with a view to the future target group, are suitable as a pilot group for courses;
4. Test a fully online course initially using combined methods. I.e., although the course is planned entirely with e-learning tools, use blended learning with a smaller group for the first time to get better feedback during the course.
5. Testing with a smart device. If you add something new to the course, be sure to use smart devices to check on how the result looks and whether the course is technically operational.

During course testing, feedback will be received on the following components:

• the structure and design of the course;
• content – its suitability, relevance, compliance with prior knowledge and organisation;
• effectiveness of teaching – learning performance within the time planned for the course, learners’ activity, attention and expression, usefulness of discussions, acquisition and adequacy of materials;
• use of technology – positive aspects, problems, perception of the technology used;
• suitability for use on a smart device;
• communication – opportunities to communicate with other learners and teachers, quality and quantity of communication;
• tasks – their usefulness, difficulty, time intensity, efficiency in receiving feedback;
• tests – their relevance, difficulty, feedback;
• support for learners – tutor assistance, technology, library and computer classroom services, access to learning materials;
• teacher – their leading role, organisational ability, preparation, enthusiasm, openness.

Testing is particularly necessary if the course is to be used in large groups, where changing the course during the course of study is particularly difficult.

Quality requirements for the e-course during the development phase

The course is well structured and easy to use.

Teaching materials are presented using appropriate media (e.g. text, images, animations, audio, video, etc.).

Teaching materials correspond to good practices of digital learning materials.

Learning materials support interaction between the learner and the learning content.

The study materials have been prepared in accordance with the terms of use of the works of other authors.

References are made to the works of other authors used in the preparation and illustration of teaching materials.

The course has learning guidelines for the whole learning process, giving an overview of the activities of face-to-face learning in the case of blended learning.

The use of the learning environment does not require the learner to purchase additional paid software.

The course is tested before being used in a real learning process.

The course can be completed on common smart devices.

The course is technically operational (links open, necessary tools are working, online literature referenced from the course is available).

4. Conducting the course

Once the e-course and its teaching materials have been created, it is necessary to use the e-course skilfully in the learning process. In the case of e-learning, it has been identified that one of the key factors to increase the rate of completion of the course is the support provided by the course provider to learners. Depending on the number of learners and the volume of work, a support person or tutor may be involved in the course. A tutor/support person is a teacher’s assistant, to whom the teacher may delegate some of their activities. A tutor can be, for example, a teaching assistant, a methodologist, a master’s student, a doctoral student or the best learner of the course. The tutor should have a clear overview of their role, responsibilities and participation rate in this course. The teacher communicates with the tutor to the extent necessary to prevent information gaps.

Teachers and tutors have to play different roles in e-learning:

1. technical – computer and learning environment-related assistance for course participants;
2. managerial– management of the learning process;
3. social – creating a supportive atmosphere between course participants;
4. pedagogical – drawing attention to important materials, managing independent work and providing feedback.

Technical role

The aim of technical support is to ensure smooth functioning of the learning environment throughout the course. Learners are advised on technical issues or referred to specialists to solve certain problems. Technical support must be available on a daily basis.

Technical activities of the course provider:

• Explains what kind of hardware and software is needed to take part in the course.
• Guides learners on how to open different file formats.
• Provides information about the learning environment and how to log in to the course.
• Provides tips on how to navigate the learning environment and how to use the tools found within the learning environment.
• Helps with technical problems.
• Makes the necessary changes to the learning environment during the course.
• Uses course management tools.

Organisational role

The organisational activities of the teacher or tutor begin already in the process of developing and preparing the course and continue during the course.

Organisational activities of the course provider:

• Monitors the progress of learners in the course and their participation. If possible, it will help to eliminate the causes behind ‘losing’ learners.
• Ensures smooth and planned running of the learning process.
• Manages the learning process – sends reports about the progress of the learning process, information about the presentation and progress of homework, reminders.
• Organises real-time chats with a chat room or webinar tool.
• Organises asynchronous forum discussions on topics related to the content of the course or not.
• Organises group work that forces learners to communicate and cooperate with each other.
• In the case of group work, divides learners into groups and explains the procedure for carrying out group work.
• Creates private forums for groups.
• Offers participants the opportunity to learn at a speed that suits them.
• If necessary, extends the time limits for homework.
• Explains the manner in which homework is to be presented.
• Creates a simple and clear correspondence structure.
• When using different forums, explains the function of each forum to learners.
• Helps solve the learners’ organizational problems.
• Asks participants for feedback on the content of the course and the use of the online environment.
• Asks for an assessment, recommendations and feedback on its activities.
• Ask for suggestions for further improvement of the course.

From an organisational point of view, it is very important to constantly monitor the activity of participants in the course and engage with those who are ‘lost’. Although the advantage of e-courses is that the learner can choose a suitable time to visit the course, it can also be accompanied by worries and fears. This also implies the need to support, encourage and give feedback to learners on what has been achieved.

Social role

The aim of the social role of the teacher and tutor is to contribute to the formation of a common atmosphere for the learning group, and to the learner feeling like their participation is awaited during the course, who can freely express their wishes and opinions.

Communication between e-course participants can be facilitated by the teacher’s choice of tasks and activities. The teacher sets a good example when interacting actively with students. When conducting the course using the forum, it is important that the teacher themself actively participates in it. For example, when asking all participants to introduce themselves to the course, it is recommended that the teacher introduces themself first. In addition, the teacher responds to the forum’s posts, thus supporting and encouraging learners during the course.

Social activities by the course provider:

• Introduces themself in the course and adds their picture.
• Adds an audio or video clip to the course, where they welcome learners and briefly introduce the course.
• Sends a welcome letter to the forum at the beginning of the course.
• Asks learners to introduce themselves.
• Introduces appropriate methods for written communication during the course.
• Quickly responds to letters from learners so that learners do not feel that they have been left alone in the learning environment.
• Sets up a dedicated forum for non-formal social interaction between learners.

Pedagogical role

In a pedagogical role, e-course learners need to be supported in carrying out tasks, moderating discussions, evaluating and providing feedback to learners, and evaluating the learning process.

Pedagogical activities by the course provider:

• Facilitates interaction between the learner and the learning material.
• Introduces different topics and provides guidance on which questions/aspects are most relevant to the topic.
• Supports learners in linking existing knowledge and course content.
• Knows valuable sources and references them.
• Searches for more information and refers to the most suitable materials.
• Promotes the sharing of resources and other content by learners.
• Constantly updates and improves supporting materials.
• Provides recommendations for the critical evaluation and selection of the information found.
• Organises discussions on various topics.
• Moderates the discussions.
• Supports students in carrying out tasks.
• Assesses students and gives them feedback.

When moderating discussions, the teacher:

• Initiates discussions on various topics.
• Guides discussions with observations, comments or questions.
• Monitors the understanding of learners and intervenes with their own comments if they fail to understand.
• Summarises the discussions.
• Assesses learners for their participation in the discussions.

When learners are performing tasks, the teacher shall:

• Suggest which ideas and strategies should be used for learning (e.g. selection of basic ideas, extension of links, illustration and schematic design).
• Provide recommendations for working with learning guidelines, working independently with teaching materials (e.g. suggests using systematic reading techniques), participating in group work and discussions, writing written papers (e.g. recommendations for structuring the work, proposing some ideas, referring to useful sources, reviewing the original version and giving recommendations for improvements, etc.).
• Monitor the progress of learners’ tasks, assisting and supporting when necessary.

In the assessment and feedback of learners, the teacher:

• Provides learners with the score or number of points received for the task.
• Informs learners of whether the answers were correct or wrong, without giving the right answer.
• Provides the correct answers in case of wrong answers.
• Substantiates the learners’ answers (why the answer was correct or wrong) or specifies.
• Guides learners with questions and comments.
• Provides additional information to learners.

Questions of plagiarism may arise during the evaluation, which need to be addressed by both online and traditional learning institutions. It is claimed that it is easier to identify plagiarism in educational establishments where electronic submission of works is the norm, since in such a case, in addition to online sources, the similarity of the proposed work to those of current and previous learners can be verified. Plagiarism detection systems, such as Ouriginal, can be used for this purpose.

Quality requirements for the e-course in the performance stage

The learner will receive comprehensive support in the learning process, which will help them to successfully complete the course.

The course follows a (planned) schedule.

Active participation of the learner in the learning process (communication between peers, the emergence of learning communities, etc.) is supported.

Students will be given feedback on their strengths, weaknesses and progress in this course.

Learners have been informed about the learning results (grades, points).

5. Assessing the course

Depending on the material covered by the course, periodic updates of the material reflected in the course may be necessary (e.g. based on new scientific findings). Also, in the course of teaching, ideas often arise on how to build the course in a pedagogical way. With this in mind, the course must be planned as easily as possible from the start, and a plan should be made how to make innovations for oneself. The plan could reflect which parts of the course will be changed and supplemented in the future, and what solutions will be seen. The plan should reflect the consideration of further changes according to the choice of technology (for example, in the case of a video lecture, the teacher is aware that in the case of a minor substantive change, the whole video lecture will have to be re-edited in a worst-case scenario, etc.).

It is recommended to review the e-course in full at least every three years: make additions and corrections, critically review the methods used, etc.

5.1. Documentation

The general recommendation is to document the entire life cycle of the e-course, from course planning to ex-post evaluation.

• Documentation allows for the analysis and assessment of the life cycle phases of the e-course. On this basis, it is possible to plan the resources of the next e-course better;
• Documentation allows you (if there has been a longer time gap in the meantime) or your colleague, course support person, etc., to start the created course with little difficulty.
• Documentation makes it possible to supplement and improve the course as a whole, as the strengths and weaknesses of the course and ideas for the future are recorded;
• The documentation shall reflect the time at which any part of the course has been last amended.

DThe collection of feedback from learners is also part of the documentation process. The results obtained shall be taken into account and any corrections, additions, etc., shall be made.

The teacher should analyse the level of satisfaction with the course over the years, and whether that satisfaction can be attributed to the changes made.

5.2. Methods of assessment

Through evaluation, the possibility of further development of the course is ensured. The evaluation of the e-course can, again, be based on the ADDIE model and an evaluation given of the analysis, design, development and carrying out stages of the course. The content, methods of implementation and interactivity of the course could also be taken into account in the evaluation. For this purpose, the following methods may be used:

• external evaluation – external reviewers review the course, providing feedback on development work and providing an overall assessment of the course. External evaluation may, in some cases, be benchmarking with other analogous courses, or expert evaluation, which provides feedback from experts (e.g. a quality label process).
• internal assessment, where:
  • during the course, data on how learners use e-learning tools is collected and analysed;
  • documented feedback will be provided during the monitoring and evaluation process, on the basis of which the course can be improved and modified.

Different methods can be used for internal assessment:

• course testing;
• traditional questioning of learners about a subject – a survey at the end of the course or module, where the learners can assess the subject/course and make suggestions for improvement;
• use of the feedback forum during the course. The use of feedback forums means, in particular, that more information about the course performance is obtained from people working with learners, such as teachers and tutors. Such feedback helps to interpret objective statistical indicators and feedback from learners better. At the same time, ethics must also be taken into account when monitoring the views expressed by learners in online forums and the use of data collected on web use;
• monitoring of the learning process – statistical data collected in the e-learning environment (number of letters sent, time spent by learners in the e-learning environment, etc.), monitoring the behaviour of learners, analysing the results and drawing conclusions from the information received;
• discussions with colleagues – introduction of e-courses to colleagues, familiarisation with other courses, etc.
• self-reflection. The simplest method of self-reflection is a survey, where it is very common to answer questions during self-reflection: what was the purpose of creating the course, what were my achievements during the course creation (learning experiences, etc.) and what was the result. Methods of self-reflection allow for more in-depth self-reflection (e.g. matrix analysis, which is the basis for the quality label process, etc.).

Regardless of whether external or internal evaluation is used for the assessment, evaluation is primarily a data-gathering process that highlights the strengths of the course and draws conclusions on the necessary changes that in turn allow for improvement of the course.

5.3 e-course evaluation process

5.3.1 Level of self-assessment

Self-assessment shall be carried out by the applicant itself, using the self-assessment form. The objective of this level is to raise the applicant’s awareness of the quality requirements for the e-course and to motivate the participants of e-courses to analyse their e-courses. The self-assessment submitted by the applicant for the Quality Label is used as an additional material for the course evaluation.

5.3.2 Organisational level

The goal at the organisational level is to obtain an assessment by the organisation and learners of the e-course applying for the quality label. A review form has been developed for evaluation, filled in by a person selected by the organisation (e.g. programme or curriculum manager, head of study, etc.) and approved by the applicant’s direct manager. The review of the organisation shall be submitted by the applicant for the quality label.

5.3.3 Expert level

The aim of the expert level is to provide feedback to the author of the course and to decide whether the e-course will be awarded a quality label. The expert evaluation shall be carried out on the basis of the evaluation form after the quality label.

Quality requirements for the e-course during the evaluation phase

During the course, notes are taken for the further improvement of the course.

There is a course feedback system available (general assessment of the course or receiving feedback from learners (including from an e-learning perspective).

---

The translation of the original text from Estonian to English and Ukrainian is supported by the Ministry of Foreign Affairs’ Development Cooperation Project No. 76-2021-A