Introduction
-
OER developers “marketing” their products.
-
Faculty, students and other potential users of OER assessing whether particular resources suited their requirements.
-
OER users assessing and informing others on whether these resources had met their requirements.
-
The educational benefits claimed for OER in the literature.
-
MERLOT’s evaluation criteria for OER content.
-
Merrill’s first principles of instruction.
Fitness for purpose
Unbundling fitness for purpose
MERLOT’s evaluation criteria
-
Content is assessed in terms of the correctness, significance and currency of the concepts/principles and their match with the learners’ characteristics and learning needs.
-
The potential teaching/learning effectiveness of the materials is judged by the inclusion of learning objectives, information on how the materials can be used by specific learner groups, the design of the materials and how these can be easily integrated into, and improve, a variety of courses and teaching/learning processes.
-
Ease of use by teachers and students is evaluated by considering accessibility, layout and design for effective learning, consistency in user interface and navigation, content presentation that matches learners’ prior knowledge and abilities and ease of re-use in other contexts.
Merrill’s five first principles of instruction
-
Learners are engaged in solving real-world problems.
-
Existing knowledge is activated as a foundation for new knowledge.
-
New knowledge is demonstrated to the learner.
-
New knowledge is applied by the learner.
-
New knowledge is integrated into the learner’s world.
Methodology
The initial survey
Total (N = 207) | |||
---|---|---|---|
N
| % | ||
Gender | |||
Female | 91 | 44.0 | |
Male | 116 | 56.0 | |
Role | |||
Educator | 116 | 56.0 | |
Researcher | 52 | 25.1 | |
Other | 39 | 18.8 | |
Region | |||
Africa | 20 | 9.7 | |
Asia | 53 | 25.6 | |
America | 63 | 30.5 | |
Europe | 45 | 21.8 | |
Oceania/Caribbean | 18 | 8.7 | |
Missing | 8 | 3.9 | |
Experience in using OER in their own teaching and learning | |||
Not at all experienced | 8 | 3.9 | |
A little bit experienced | 19 | 9.2 | |
Somewhat experienced | 50 | 24.2 | |
Quite a bit experienced | 52 | 25.1 | |
Very experienced | 78 | 37.7 |
-
Section 1 sought information on the reviewers’ locations, gender, principal roles and degree of experience in using OER in their own teaching and learning.
-
Section 2 invited the reviewers to rate the criticality and feasibility of 18 proposed OER selection criteria based upon MERLOT’s three dimensions and specific review questions under each dimension in terms of their own teaching and learning (using a scale from one to five, with 5 being the most useful).
-
Section 3 asked reviewers to rate the feasibility and usefulness of 10 proposed OER selection criteria based upon Merrill’s first principles of instruction and application strategies for each principle, again in terms of their own teaching and learning (using a scale from one to five, with 5 being the most useful).
-
Section 4 invited the reviewers to offer their own comments on the proposed criteria, how these might be improved, what further criteria would be useful and how else the fitness for purpose framework could be improved.
The initial survey findings
Criteria | Criticality Rating (N = 207) | Feasibility Rating (N = 207) | ||
---|---|---|---|---|
Dimension 1: Content | M | SD | M | SD |
1) The content is correct and up to date. | 4.40 | .756 | 4.00 | .873 |
2) The content includes significant concepts, models and skills appropriate for the discipline. | 4.08 | .829 | 3.77 | .844 |
3) The content is appropriate to the learners’ knowledge, skills and abilities. | 3.93 | .836 | 3.43 | .910 |
4) The content is culturally appropriate and contains no racial or gender bias. | 3.78 | .965 | 3.52 | 1.047 |
5) The content is developed through rigorous academic processes by well-qualified persons. | 3.80 | .895 | 3.48 | .944 |
6) The OER accords with copyright laws (Commons License) and is correctly cited/acknowledged. | 4.20 | .927 | 3.81 | .999 |
Dimension 2: Potential Teaching/Learning Effectiveness | M | SD | M | SD |
7) The teaching/learning objectives of the OER are clear. | 3.78 | .881 | 3.75 | .900 |
8) The OER can be readily integrated into the current curriculum and pedagogy. | 3.74 | .847 | 3.43 | .962 |
9) The OER can be easily integrated into and improve a variety of courses and teaching/learning environments. | 3.71 | .865 | 3.47 | .918 |
10) The OER’s multimedia components provide effective presentation. | 3.29 | .920 | 3.46 | .928 |
11) The OER includes interactive instructional activities. | 3.53 | 1.023 | 3.30 | .918 |
12) The OER includes test/self-assessment items that enable the learners to study independently. | 3.52 | .994 | 3.41 | .859 |
Dimension 3: Ease of Use | M | SD | M | SD |
13) The screen layout of the OER helps the learners comprehend the content and has no distracting features. | 3.80 | .902 | 3.70 | .891 |
14) The user-interface components (buttons, menus, icons, scroll bars, etc.) are arranged consistently and allow the learners to navigate the site easily. | 3.82 | .890 | 3.70 | .885 |
15) The OER facilitates flexible learning by allowing the learners to control the rate, order and nature of their learning. | 3.82 | .810 | 3.57 | .916 |
16) The OER’s navigation system enables the learners to know where they are relative to the rest of the site. | 3.64 | .799 | 3.65 | .896 |
17) The presentation methods of the OER are suited to the learners’ prior knowledge and abilities. | 3.37 | .893 | 3.24 | .891 |
18) The OER can easily be remixed, manipulated, aggregated and shared to meet other teaching/learning needs. | 3.93 | .963 | 3.31 | 1.001 |
Pedagogical applications | Fitness for purpose rating (N = 207) | |
---|---|---|
M | SD | |
1) Presenting relevant real-life tasks or problems that progress from simple to complex | 4.04 | .949 |
2) Gaining students’ attention and developing their interest in the new learning | 3.90 | .813 |
3) Helping students, recall, relate or apply prior knowledge, skills or experience as a basis for the new learning | 4.00 | .916 |
4) Presenting new content appropriate to the students’ needs and abilities | 3.90 | .807 |
5) Incorporating multimedia that extend the amount and nature of the content | 3.96 | .880 |
6) Providing supplementary/enhancement materials that extend and enrich the teaching/learning | 4.06 | .935 |
7) Providing practice/test/self-assessment items | 3.49 | 1.065 |
8) Enabling transfer of the new learning to different tasks/problems/contexts | 3.99 | .881 |
9) Promoting students’ reflection on the new learning and their own learning | 3.83 | .973 |
10) Promoting students’ discussion and debate to construct their own learning | 3.87 | 1.016 |
Criteria |
---|
Legal and Technical |
1) The OER accords with copyright laws (Commons License) and is correctly cited/acknowledged. |
2) The OER can be easily reused, revised, remixed and shared with other materials to meet particular teaching-learning needs of the course. |
3) The user-interface components (buttons, menus, icons, scroll bars, etc.) are arranged consistently and allow the learners to navigate the site easily. |
Content |
4) The content is correct and up to date. |
5) The content includes significant concepts, models and skills appropriate for the discipline. |
6) The content is appropriate to the learners’ knowledge, skills and abilities. |
7) The content is culturally appropriate and contains no racial or gender bias. |
Pedagogy |
8) The OER is used to present relevant real-life tasks or problems that progress from simple to complex. |
9) The OER can be readily integrated into the current curriculum and pedagogy. |
10) The OER can be easily integrated into and improve a variety of courses and teaching/learning environments. |
11) The OER is used to gain students’ attention and to develop their interest in the new learning. |
12) The OER is used to devise interactive ways of using OER to promote students’ engagement in the problem-solving process. |
13) The OER is used to help students recall, relate or apply prior knowledge, skills or experience as a basis for the new learning. |
14) The OER is used to present new content appropriate to the students’ needs and abilities. |
15) The OER integrates multimedia that extend the amount and nature of the content. |
16) The OER provides supplementary/enhancement materials that extend and enrich the teaching/learning. |
17) The OER facilitates flexible learning by allowing the learners to control the rate, order and nature of their learning. |
18) The OER enables transfer of the new learning to different tasks/problems/contexts. |
19) The OER promotes students’ reflection on the new learning and their own learning. |
20) The OER promotes students’ discussion and debate to construct their own learning. |
21) The OER includes interactive instructional activities. |
Designing the framework
Dimension | Fitness for purpose |
---|---|
Purposes (7 criteria) | □ Providing open, accessible and quality content for a wider community of teachers and learners. Comments …………………………………………………………….. □ Sharing best practice and helping to avoid re-inventing the wheel. Comments …………………………………………………………….. □ Helping developing countries improve and expand learning for development. Comments ………………………………………………………………… □ Offering flexible non-formal and informal knowledge and skills accumulation pathways to formal study. Comments ………………………………………………………………… □ Providing for geographically, socially or economically excluded students, non-traditional students, work-based learners, etc. Comments ………………………………………………………………… □ Improving the quality of conventional and online education by achieving greater awareness of open and inclusive educational practices and varied perspectives on fields of study. Comments ………………………………………………………………… □ Enabling collaboration between institutions, sectors, disciplines and countries. Comments ………………………………………………………………… |
Ease of Use (4 criteria) | □ The OER accords with open content licenses (e.g., Creative Commons) that have been properly referenced and applied to the resources. Comments ………………………………………………………………… □ The OER can be reused, revised and remixed with other resources or shared with students or other teachers. Comments ……………………………………………………………….. □ The screen design and navigation systems are clear and consistent. Comments ……………………………………………………………….. □ The presentation methods accord with the learner’s knowledge and abilities. Comments ………………………………………………………………… |
Content (5 criteria) | □ The goals and content are easily understood. Comments ……………………………………………………………….. □ The content is accurate and up to date. Comments ………………………………………………………………. □ The content covers educationally significant concepts and enables deep understanding. Comments ……………………………………………………………….. □ The content progresses from simple to complex. Comments ……………………………………………………………….. □ The content is appropriate to the students’ knowledge, experience, language, ethnicity, race, culture, religion age, gender or other circumstances. Comments ………………………………………………………………… |
Pedagogy (9 criteria) | □ The OER gains and maintains students’ attention and interest. Comments ………………………………………………………………… □ The OER helps the students recall, relate or apply prior knowledge, skills, experience, etc. Comments ………………………………………………………………. □ The OER provides a sound structure for knowledge and skills development. Comments ……………………………………………………………….. □ The OER provides opportunities for task analysis and solving hands-on, real-world problems. Comments ………………………………………………………………… □ The OER’s text, images, audio and video elements and hyperlinks provide diversity in learning. Comments ………………………………………………………………. □ The instructional design focuses on the key aspects of the learning and lacks distracting features. Comments ………………………………………………………………… □ The OER contains in-built feedback, support and assessment. Comments ………………………………………………………………… □ The OER enables the transfer of the new knowledge and skills to different tasks, problems or contexts. Comments ………………………………………………………………… □ The OER enables the students to consolidate their learning or construct personal meaning through reflection, discussion, demonstration of new knowledge or skills, etc. Comments ………………………………………………………………… |