FRoLLM: Framework for the Reflection on Living Learning Materials
We have created the FRoLLM to support your reflections on learning materials.
FRoLLM
Teaching and learning materials are an important core element of your everyday teaching and learning experiences. However, we rarely reflect on the materials we actually work with on a deeper level.
The FRoLLM helps you think more deeply about the teaching and learning material you use on a daily basis. It was designed to support the inclusion-sensitivity of educational materials, taking into consideration various learners' needs.
The FRoLLM has six areas of reflection. Each area comes with questions guiding your reflection. There are several examples for each guiding questions for clarification. Start with the area that speaks to you the most. When reflecting on your materials, there is no right or wrong!
Your reflections, thoughts and perspectives are what matters.
STAY IN TOUCH
TEAM
Our goal is to deepen our partnership and collectively work on the topic of inclusive digital learning materials and OER - this way, different expertises will be merged in order to reflect on inclusive Open Educational Resources. Now - working on sustainable results within an international setting needs good and thorough planning. That is why the project management is a fix part of the work packages guiding our work. That entails a lot of communication, online meetings, keeping an eye on the project roadmap and coordinating administrative work. That is how we create a framework for participatory research - which you can be a part of along the way. So stay tuned in order to collectively reflect on inclusive digitality and Open Educational Resources.
The first step is to really understand what the landscape of Open Educational Resources looks like in Czech Republic, Sweden, Luxemburg and Germany. For this, we collect and analyse OER repositories and academic literature on the topic. Both sources tell us a lot about what the handling of OER in the country looks like. We call this first overview the "mapping of OER" - just like a geographer trying to learn the topography of a landscape, we are trying to grasp what OER is all about in the partner countries. That is the more research based part of the project. Here, we also interview teachers, learners, OER providers and researches about the topic. This way, lots of perspectives on all things OER can be collected.
DigiLLM is all about inclusion-sensitive digital teaching material, particularly OER. Mapping OER helps us understand the topic more in depth. But what do we *do* with that knowledge we gather in workpackage 2? Workpackage 3 is all about creating a criteria catalogue for inclusion-sensitive OER. It will be a collection of questions you can use to evaluate learning material and OER with. It helps to estimate whether a material is inclusion-sensitive - or not. There already is a criteria catalogue from the Inclusive Teaching Material project (www.itm-europe.org - @itm.europe ) that will help you understand what a criteria catalogue looks like and how you can use it. The new digiLLM criteria catalogue for inclusion-sensitive OER will be published on our digital portal as a living document - so nothing is set in stone: with this, we respect the fluidity of Open Education. After creating the new catalogue particularly for OER (based on everything we learn in workpackage 2!), we will compile training modules introducing the criteria catalogue to teachers and learners. This way, we share the project's results with the broad public and bring it into schools - in all four partner countries!
Work Package 4 will be the heart of the DigiLLM project: the **rating function** of the digital portal we are setting up. The overall idea is to invite teachers, educators and learners (in short: YOU!) to evaluate digital learning materials and assess whether you consider learning materials inclusion-sensitive or not. The foundation for this is the criteria catalogue we are currently working on. Our aim is to invite you to be a part of the discourse around inclusion-sensitive learning materials. Asking for teachers’ and learners’ perspectives in the interviews alone (see Work package 2!) is not enough – we want to learn about your opinions throughout the entire duration of the project and beyond! The rating function of the DigiLLM-portal is designed to establish a place where we can continuously and steadily discuss inclusion-sensitivity in learning materials and share our points of view. Ratings can be given by a simple “star rating” (0-5) or a more extended commentary. In about a year, the rating function should already be online – good things take time. We will keep you posted and share the process along the way.
On this website, we introduce you to all the things we are planning to do for (and with!) you as teachers and learners. Mainly, it is all about creating a space to jointly and collaboratively discuss the quality and inclusiveness of learning and teaching material. Why? Because learning materials can contribute to creating more (or less...) inclusive learning environments. Therefore, the everday materials we are working with need to be critically assessed in the daily teaching and learning practice. But **also** in academia! There is more research needed on quality, potentials and limitations of learning materials and OER specifically. For this, we are in the process of founding a *brand new academic journal* – digital, international and peer reviewed. Working title: QuEST – Quality in Educational Material Standards. This will be the place for so-called metareviews of YOUR reviews you created on the rating function of the DigiLLM-portal (see the information on workpackage 4 for more information). Here, academics and researchers will take into consideration what your perspectives are and analyse it in a wider context. Articles based on your reviews will then be published in the journal. This way, your reviews and experiences will not only be adding to the digital discourse on the quality of learning materials – it will also be guided back into academic research (researchers, experiences or emerging: keep your eyes peeled for what's to come!) Herein, DigiLLM goes full circle. It is not mainly about carrying academic expertise into the practice – quite the contrast: we aim to center perspectives and expertise of teachers and learners at all times.
WORKPACKAGES
The material should be sensitive to various learners’ needs (different learning levels, interests, backgrounds, potentials and limitations). Hence, it should offer various ways of how learners can learn about the same subject matter on different levels of complexity.
The material should rely on different kinds of media and should be adaptable to certain needs of the individual (accessibility, cultural representation). The material should help the learner to actively shape their learning paths themselves, enabled by providing choices between different options on how to work/proceed with the learning material.
According to the learner’s needs, the material allows students to work on different learning levels. For example,
-
it provides a framing of the content (e.g. statics in bridge building) and
-
an overarching question (e.g. how can you stabilize your bridge so that it can carry cars?)
Even though all students work on the same question, the level of complexity on how to approach the question can be . For example,
-
a student can build a bridge with prefabricated components while another constructs a bridge without pre-given information. A third student might use “info cards” that provide information about static elements such as the stable triangle.
-
The material offers scaffolding to problem-solving. For example:
-
by indicating the sequential steps of the solution,
-
showing what to start with
-
giving hints
-
modelling
-
asking guiding questions
When a task requires writing an essay, graded scaffolding can be offered, for example
-
by providing an outline/structure
-
by providing guiding questions for each paragraph
-
by providing a model text to imitate
-
Key concepts presented in one form of representation (e.g., a text or a math equation) are supplemented with an alternative form of representation. For example:
-
an illustration
-
diagram
-
table
-
model
-
video
-
comic strip
-
animation
Alternatives for visual or auditory information are available. For example:
-
descriptions are provided for images and videos
-
graphic symbols are provided with text descriptions
-
relationships between elements are made explicit
-
highlighting the transition words in an essay,
-
links between ideas in a concept map etc.
-
support for vocabulary and symbols within the text is embedded
-
hyperlinks
-
footnotes
-
definitions
-
explanations
-
illustrations
-
The material can be so it is possible to make adaptations and changes. For example:
-
size of text and images
-
the colour used for emphasis
-
the volume or rate of sound
-
the speed or timing of video etc.
In addition, adaptations to the learners, their learning environment and local, temporal and political contexts are also feasible.
-
Based on a broad understanding of environment, the material should be adaptable for various kinds of learning scenarios, might they take place in groups or individually. A material should be adaptable to different physical surroundings (at home/ in the classroom/library/ hospital), but also digital environments (e.g. during distance learning scenarios).
Hence, the material should not have any particular demands, which may limit the usage to a specific location or setting.
This implies the inclusion-sensitivity of digital materials in terms of accessibility and openness.
The material can be used by one student but also provides impulses so that two or more learners can work on the same task together. This makes it necessary to provide, for example, additional selected stimuli to the material in order to encourage discussion with the partner(s). For example:
-
Comparing the partners’ ideas on action on climate change in a shared online mind map.
-
The material can be used in the classroom as well as at home, in the library or in hospital etc. Also, it is usable in different digital learning scenarios. There are different requirements for this. For example,
This requires that tasks are based on easily accessible components. For example:
-
the material may also be accessible with a poorer internet connection
-
there are alternatives for audio content if headphones are not available or the environment is noisy
it consists of easily accessible components, e.g. a task in science education might be to build a bridge out of paper in order to learn about statics.
-
The material is available on the PC or tablet as well as in a mobile phone version. This implies that the material requires formatting that makes images, text modules and interactive elements easy to display on different technical devices. The material is available in different formats and published under an open license, allowing for people to freely use, adapt, remix and share it.
The material should imply elements of qualitive as well as quantitative feedback. It should enable self-assessment formats as well as peer-feedback. Feedback can be designed as immediate learning feedback or as a more broad reflection on the learners’ progress. The provided feedback elements should support the learners in reflecting on their learning process in a valuing and affirmative way. Additionally, feedback options may appoint to next steps in a learning journey.
The material includes checklists or rubrics that guide students to evaluate their own work or the work of others and provide criteria for improvement. For example,
-
a checklist for writing a persuasive essay might include items such as “I have a clear thesis statement”, “I have provided relevant evidence to support my arguments”, “I have addressed possible counterarguments and refuted them”, and so on
-
a rubric might assign different levels of performance (such as excellent, good, satisfactory, or needs improvement) to each item, and provide descriptors for each level.
-
The material can contain examples or models that illustrate the expected outcomes or standards of performance for a given task or assignment. For example:
-
an example for a math textbook might show how to solve a complex problem using different methods or strategies, and explain the steps and reasoning involved.
-
a model for an English textbook might show a sample essay or paragraph that demonstrates the use of effective writing techniques, such as topic sentences, transitions, evidence, and analysis.
The material includes the correct solutions to the learning problems if they are convergent in nature or a suggestion of possible solutions if they are divergent in nature.
The material comprises assessment tests for students to check for themselves whether they mastered the subject matter.
-
The material should actively support the learner’s knowledge about different learning strategies. The learner should be encouraged by the material to try out different learning strategies and to integrate them into their learning process. Also, the learner should generally be supported on how to reflect on their learning process as a whole, regarding their progress, main strengths as well as weaknesses.
The material suggests different ways to approach the assignments. For example:
-
group discussions
-
learning from text
-
hands-on experiments
-
creating videos
-
story-telling
-
reciprocal questioning
-
peer teaching
The assignments/tasks are not stereotypical, i.e. it is possible to apply different cognitive operations to address the task. For example:
-
concept formation (e.g. classifying animals by their characteristics or defining abstract concepts like democracy or justice)
-
problem-solving (e.g. finding the best route to a destination or designing an experiment to test a hypothesis)
-
The assignments (sometimes) require students to explicitly describe and/or evaluate their thinking process. For example:
-
what steps they took,
-
how they thought about the problem,
-
why they adopted the procedure, or
-
give reasons why they solved the problem in a way they did.
Students are encouraged to work deliberately with mistakes, either their own or those of their classmates.
-
Within a material, learners should be considered agents of their own learning path. The material should enable the learners to express their individuality within the learning process. This is about a learner’s needs, but also a learner’s wishes, creativity and boundaries. This approach is based on inclusion as an ethical standard related to human rights.
The material encourages learners to reflect on their own learning, their strengths and capabilities. For example:
-
by addressing the learner directly and taking them seriously, by offering opportunities for self-reflection and dialogue, posing different types of tasks, enable decision making, using encouraging language.
-
by encouraging a learner to opt for another material if they do not find the material they are currently working with helpful.
A material enables learners to generate their own ideas. For example:
-
when not everything is directed at a specific learning goal or outcome,
-
a student is invited to make up more examples/scenarios for a learning objective on his/her own
-
is asked to engage with the learning objective creatively.
In order to leverage this kind of agency, the material is committed to transparency. For example:
-
by explaining its intentions and learning goals clearly
-
providing context about learning theories
-
Authors of a material communicate learning goals transparently.
The material encourages learners not to work on a task if it is categorised as unsuitable for their own learning path/needs. This gives learners control over their own learning path.
The material also allows learners to relate to their own environment. For example:
-
inviting them to reflect on examples of a topic in their environment (places, animal species, political situation, geographical conditions, etc.).
-
The material and its authors are committed to the idea of open and participatory further development of materials, e.g. by enabling automatic feedback and providing a space for comments, ratings and reviews.
The material reflects the diversity of learners in terms of their identities as well as social, family, cultural, religious and community backgrounds. This is reflected in images and texts that avoid stereotypical or even discriminatory content. Learners, no matter their background, feel seen and respected.
-
Pictures/contents mirror the diversity of people in the community. There are no stereotypical, biased or even discriminatory pictures/contents.
-
Cultural and religious diversity is being represented, but not in a stereotypical or biased manner.
-
Diverse perspectives are being represented.
Issues like racism, sexism, ableism, ageism, and homophobia are not detectable.
The material reflects the diversity of environments and lifeworlds (or it can be adapted easily).
-
This is reflected in images and texts that avoid stereotypical or even discriminatory content.
-
Aspects like family life, social life, communities, religion, sexuality etc. are not presented in normative ways.
-
Different social, political, financial backgrounds or places to live (cities, rural areas) are being represented.
-
Author(s) of a material should explain its philosophy and conceptions. This implies comments on design choices and theoretical backgrounds. Target groups should be made explicit as well as needs the material addresses. Moreover, a material’s limitations should be made transparent. Also, the material should clarify how it relates to the national school curriculum.
Reliable sources should be provided in order prevent mis- and disinformation.
Whenever a material claims to be ”inclusive”, the authors should clarify their understanding of inclusion thoroughly.
Learning and teaching philosophy/design applied in the material is explained. For example:
-
project teaching and learning,
-
on problem-solving,
-
on constructivist ideas in general, it may favour
-
storytelling,
-
case-studies
It is suggested how to use the material. For example:
-
mainly as a reference book
-
learning tasks repository
-
comprehensive guide to lessons
-
for the whole class vs. self-study
-
if it is necessary to go through the material systematically, from beginning to end, or, the contrary, it is possible to select parts of the text; the material may be preferably designed for whole-class teaching or can be used for self-study as well).
-
The material contains specific information directed to the teacher and students on its content, how it can be used and the key ideas in how it is set up. This includes a definition of inclusion, if inclusion-sensitivity is claimed. The intentions and limitations regarding inclusion and different target groups are mentioned.
The material is ordered in a transparent and coherent way. The general idea of learning and teaching, content and core concepts are presented early. A navigation or mapping function supports the understanding of the material's structure. Subject areas are displayed and ordered in logical sub-sub areas. When entering these, there is first information on the topic followed by clarifying videos or other resources. It is always possible to navigate back and to understand where in the material one is at the moment.
The material contains signals for distinguishing content with different levels of importance and novelty. For example:
-
graphical markers (signaling basic content, information to memorize)
-
polygraphic signals (different colours or fonts for certain parts of the text, etc.). At the level of expository text or instructions, this can be, for example, so-called signal words (such as: firstly, secondly..., for example ..., as in ..., as opposed to ..., as compared to ..., the result is...) or advance organizers (like Venn diagram or Fishbone chart to show cause and effect relationships, etc.).
-
The content is validated by references and refers to the latest scientific knowledge. References are both old and up-to-date.
The material encourages learners not to work on a task if it is categorised as unsuitable for their own learning path/needs. This gives learners control over their own learning path.