Adaptive Learning Systems in the World Wide Web

Gerhard Weber
Department of Psychology, University of Education Freiburg, Germany
Abstract: With the steadily growing demand for further education, the World Wide Web
is becoming a more and more popular vehicle for delivering on-line learning courses. A
challenging research goal is the development of advanced Web-based learning applications
that can offer some amount of interactivity and adaptivity in order to support learners who
their start learning with different background knowledge and skills. In existing on-line
learning systems, some types of adaptivity and adaptability require different types of user
models. This paper briefly introduces ELM-ART, an example of a substantial adaptive
learning system on the WWW. It uses several adaptive techniques and offers some degree
of adaptability. The adaptive techniques are based on two different types of user models: a
multi-layered overlay model that allows for sophisticated link annotation and individual
curriculum sequencing; and an episodic learner model that enables the system to analyze
and diagnose problem solutions and to offer individualized examples to programming
problems. The last section gives an overview of empirical results with adaptive learning
systems and discusses the problems concerned with the evaluation of complex learning
systems in real-world learning situations.
1 Introduction
The World Wide Web is becoming an increasingly popular vehicle for delivering on-line learning
courses (Khan, 1997). The benefits of Web-based education are clear: classroom independence
and platform independence. An application installed and supported in one place can be used by
learners from any place in the world. They only have to be equipped with any kind of Internetconnected
computer.
With the steadily growing demand for further education in all areas of life, intranet-based
corporate training constitutes another application area of Web-based learning systems. This is
accompanied with catchwords like "learning on the job" and "learning on demand". Many
companies are already using or planning server-based training courses delivered to an employee’s
desktop via an intranet. Principally, there is no difference between server-based learning systems
via the WWW or via intranets, although the variety of users in an intranet will not be as large as
in WWW-based learning systems. In both cases, different levels of background or prior
information will require adaptive and adaptable learning systems that are able to take into account
that existing knowledge in order to provide an individually tailored training course for the
particular learner.
The problem is that most of the existing Web-based learning systems consist of a network of
static hypertext pages. A challenging research goal is the development of advanced Web-based
learning applications which can offer some interactivity and adaptivity. Adaptation is especially
important for Web-based learning for at least two general reasons. First, most Web-based
applications are to be used by a much wider variety of users than any standalone application. A
Web application which is designed with a particular class of users in mind may not suit other
users. Second, in many cases the user is "alone" working with a Web "tutor" or "course"
(probably from home). The assistance that a colleague or a teacher typically provides in a normal
classroom situation is not available.
2 Adaptation in Web-based Learning Systems
Existing Web-based learning systems use different types of adaptation techniques (Brusilovsky,
1996). These comprise adaptive presentation and adaptive navigation support, curriculum
sequencing, intelligent analysis of students’ solutions, interactive problem solving support, and
example-based problem solving support. In the future, adaptive collaboration support, specially
designed for the context of Web-based education, may complete this list.
The goal of adaptive presentation is to adapt the content of a hypermedia page to the user's
goals, knowledge, and other information stored in the user model. In a system with adaptive
presentation, the pages are not static but adaptively generated or assembled from different pieces
for each user. For example, with several adaptive presentation techniques, expert users may
receive more detailed and deep information, while novices receive additional explanations.
The goal of curriculum sequencing (also referred to as instructional planning technology) is to
provide the student with the most suitable, individually planned sequence of knowledge units to
learn and sequence of learning tasks (examples, questions, problems, etc.) to work with. In other
words, it helps the student to find an "optimal path" through the learning material.
The goal of adaptive navigation support is to support the student in hyperspace orientation
and navigation by changing the appearance of visible links. In particular, the system can
adaptively sort, annotate, or partly hide the links of the current page to simplify the choice of the
next link. Adaptive navigation support can be considered as an extension of curriculum
sequencing technology into a hypermedia context. It shares the same goal - to help students to
find an "optimal path" through the learning material. At the same time, adaptive navigation
support is less directive than traditional sequencing: it guides students implicitly and leaves them
with the choice of the next knowledge item to be learned and next problem to be solved.
Intelligent analysis of student solutions deals with students' final answers to educational
problems (which can range from a simple question to a complex programming problem) no
matter how these answers were obtained. Unlike non-intelligent checkers which can tell no more
than whether the solution is correct, intelligent analyzers can tell exactly what is wrong or
incomplete and which missing or incorrect piece of knowledge may be responsible for the error.
Intelligent analyzers can provide the student with extensive error feedback and update the student
model.
The goal of interactive problem solving support is to provide the student with intelligent help
on each step of problem solving - from giving a hint to executing the next step for the student.
The systems which implement this technology can watch the actions of the student, understand
them, and use this understanding to provide help and to update the student model.
In an example-based problem solving context, students solve new problems taking advantage
of examples from their earlier experience. In this context, an ITS helps students by suggesting the
most relevant cases (examples explained to them or problems solved by them earlier). An
example from the domain of teaching programming is ELM-PE (Weber, 1996b). Example based
problem solving does not require extensive client-server interaction and, therefore, can be used
easily in adaptive learning systems on the Web.
All these adaptation techniques require at least a rudimentary type of user modeling. While
presentation adaptation and simple types of curriculum sequencing can be based on (typically
rough) stereotype user models, adaptive navigation support and dynamic types of curriculum
sequencing require at least overlay user models. The most sophisticated adaptation techniques
(intelligent analysis of student solutions, interactive problem solving support, and example-based
problem solving) require more advanced AI-techniques that typically are used in intelligent
tutoring systems (e.g., rule-based or case-based reasoning).
3 User Modeling and Adaptation in ELM-ART
The introductory LISP course ELM-ART (ELM Adaptive Remote Tutor) is an example of a
substantial an adaptive WWW learning system which uses different types of adaptivity and
adaptability. It consists of an electronic textbook enhanced with significant interactive features
(e.g., tests and quizzes, interactive programming support and program evaluation, and interaction
with tutors or other learners via a chat room). This section explains how adaptivity and
adaptability work in ELM-ART and describes the different types of knowledge representation
and the underlying student modeling.
ELM-ART distinguishes two different types of knowledge representation. On the one hand,
the electronic textbook with all the lessons, sections, and units is based mainly on domain
knowledge and deals with acquiring this knowledge. On the other hand, the episodic learner
model, ELM, deals with the procedural knowledge necessary to solve particular programming
problems.
3.1 The Multi-layered Overlay Model
Knowledge about units to be learned from the electronic textbook is represented in terms of a
conceptual network. Units are organized hierarchically into lessons, sections, subsections, and
terminal (unit) pages. Terminal pages can introduce new concepts, present lists of test items to be
worked at, or offer problems to be solved. Each unit is an object containing slots for the text unit
to be presented with the corresponding page and for information that can be used to relate units
and concepts to each other. Slots store information on prerequisite concepts, related concepts, and
outcomes of the unit (the concepts that the system assumes to be known if the user worked
through that unit successfully). Additionally, each unit can have a list of test items or a
programming problem to be solved by the learner.
The user model related to this declarative conceptual domain knowledge is represented as a
multi-layered overlay model. The first layer describes whether the user has already visited a page
corresponding to a concept. The second layer contains information on which exercises or test
items related to this particular concept the user has worked at and whether he or she successfully
worked at the test items up to a criterion or solved the programming problem. The third layer
describes whether a concept could be inferred as known via inference links from more advanced
concepts the user already worked at successfully. Finally, the fourth layer describes whether a
user has marked a concept as already known. Information in the different layers is updated
independently. So, information from each different sources does not override others.
The multi-layered overlay model supports both the adaptive annotation of links and individual
curriculum sequencing. Links that are shown in an overview on each page or in the table of
contents are visually annotated according to five learning state of the corresponding concept. (1)
A concept is annotated as ‘already learned’ if enough exercises or test items belonging to that
concept or the programming problem have been solved successfully. (2) The concept is annotated
as ‘inferred’ where the concept is not ‘already learned’ and it was inferred as learned from other
concepts (third layer). (3) The concept is annotated as ‘stated as known by the user’ in case the
user marked this concept as already known and there is no information that the concept is
‘already learned’ or ‘inferred’. (4) A concept is annotated as ‘ready and suggested to be visited’
where it is not assigned to one of the first three learning states and all prerequisites to this concept
are assigned to one of the first three learning states. (5) A concept is annotated as ‘not ready to be
visited’ if none of the other four learning states hold. Link annotation is used as a hint only. That
is, a learner can visit each page even if it is not recommended to be visited.
Individual curriculum sequencing in ELM-ART means that the system’s suggestion of the
next page to visit is computed dynamically according to the general learning goal and the learning
state of the concepts as described above. The next suggested page will belong to the concept that
is not assigned to one of the first three learning states and that is the next one ready to be learned.
3.2 The Episodic Learner Model
The system’s knowledge consists of both common LISP domain knowledge (described above)
and episodic knowledge about a particular learner. Both types of knowledge are highly
interrelated. That is, on the one hand, the system is able to consider individual, episodic
information for diagnosing code and for explaining errors in addition to using the common
domain knowledge. On the other hand, when explaining individual errors and examples from the
learner’s individual learning history, the system can combine episodic information with
information from the domain knowledge
The representation of the domain knowledge used in episodic modeling consists of a
heterarchy of concepts and rules (Weber, 1996a). Concepts comprise knowledge about the
programming language LISP (concrete LISP procedures as well as superordinate semantic
concepts) and schemata of common algorithmic and problem solving knowledge (e.g., recursion
schemata). These concept frames contain information about plan transformations leading to
semantically equivalent solutions and about rules describing different ways to solve the goal
stated by this concept. Additionally, there are bug rules describing errors observed by other
students or buggy derivations of LISP concepts which, e.g., may result from confusion between
semantically similar concepts.
The individual learner model consists of a collection of episodes that are descriptions of how
problems have been solved by a particular student. These descriptions are explanation structures
(in the sense of explanation-based generalization, Mitchell, Keller, & Kedar-Cabelli, 1986) of
how a programming task has been solved by the student. That is, stored episodes contain all the
information about which concepts and rules were needed to produce the program code the
students offered as solutions to programming tasks. Episodes are not stored as a whole. They are
distributed into snippets (Kolodner, 1993) with each snippet describing a concept and a rule that
was used to solve a plan or subplan of the programming task. These snippets are stored as
episodic instances with respect to the concepts of the domain knowledge. In this way, the
individual episodic learner model is interrelated with the common domain knowledge.
To construct the learner model, the code produced by a learner is analyzed in terms of the
domain knowledge on the one hand and a task description on the other hand. This cognitive
diagnosis results in a derivation tree of concepts and rules the learner might have used to solve
the problem. These concepts and rules are instantiations of units from the knowledge base. The
episodic learner model is made up of these instantiations. In ELM, only examples from the course
materials are pre-analyzed and the resulting explanation structures are stored in the individual
case-based learner model. Elements from the explanation structures are stored with respect to
their corresponding concepts from the domain knowledge base, so cases are distributed in terms
of instances of concepts. These individual cases—or parts of them—are used in ELM-ART for
two different adaptation purposes. First, episodic instances can be used during further analyses as
shortcuts if the actual code and plan match corresponding patterns in episodic instances. The
ELM model and the diagnosis of program code is described in more detail in (Weber, 1996a).
Second, cases are used by the analogical component to show similar examples and problems for
reminding purposes (Weber, 1996b).
3.3 Adaptable Features in ELM-ART
Some users like to be able to adapt a system to their own needs. In particular, this is true for more
experienced users and those with previous experience and knowledge in the learning domain.
Perhaps the most important feature that ELM-ART offers a user is the possibility to inspect and
to edit his or her own user model as already mentioned above. In this way, the user can cooperate
with the learning system in refining the learner model. That is why we can call the multi-layered
user model a collaborative (Brusilovsky, 1996) or cooperative (Kay, 1995) learner model.
We think that in the WWW context and especially in a lifelong education context,
collaborative student modeling becomes very important. Students accessing a WWW course may
range from complete novices in the subject being taught to students who missed just a few
aspects of the course. A Web-based learning system should not assume, as many on-site learning
systems do, that all students have no knowledge of the subject. Instead, it has to provide an
interface for advanced students to communicate their starting knowledge. Therefore, ELM-ART
not only offers a cooperative student model but also offers pre-tests for each section. Students can
show in the pre-tests how much they already know and the system will infer, from the
successfully solved test items, which concepts the learner already knows. Then, the system will
annotate these concepts as already known and will guide the learner to those pages that have to be
learned or re-learned by the student.
Other adaptable features of ELM-ART comprise the possibility to determine the color of link
annotation, to switch off or on link annotation and guiding, to suppress warnings, and to read the
texts alone,without the exercises being presented.
4 Evaluation of Adaptive Learning Systems
Over the last two years, more and more results of empirical investigations have been reported.
Most of them show only moderate (or even no) effects of single adaptation techniques. E.g., the
empirical study on the effects of link annotation and individual guiding with ELM-ART (Weber
& Specht, 1997) showed only some hint of how these adaptive techniques may influence the
learning process. Subjects who had no previous experience with any programming language tried
to learn longer with ELM-ART when they were guided by the system using a NEXT button.
These results can be easily interpreted when one looks at the navigation behavior of the complete
beginners more closely. All but one of the beginners had no experience in using a WWW
browser. That is, these subjects profited from being guided directly by the system when using the
NEXT button. Without such a button, they had to navigate through the course materials on their
own. Learning to navigate through hypertext in addition to learning the programming language
may have been too difficult. So individual adaptive guidance by the system is especially helpful
for the complete beginners. Most subjects who were also familiar with at least one other
programming language were familiar with Web browsers. They were more pleased with the link
annotation and stayed with the learning system longer when links were annotated adaptively.
Crucial to a learning situation is how successful and how fast learners complete the course. As
ELM-ART directly stems from the on-site learning environment ELM-PE (Weber and
Möllenberg, 1995), comparing results of learning with ELM-ART to previous results from
learning with ELM-PE will give an idea of how well one can learn with a web-based learning
system. In ELM-PE, students used the system to solve programming tasks in parallel to the
traditional classroom course. ELM-PE offered automatic diagnosis of problem solutions and the
individual presentation of example solutions based on the episodic learner model. In ELM-ART,
presentation of the texts and all explanations were given by the system in addition to all adaptive
features mentioned above. Results from the final programming tasks (three tasks on recursive
programming) show that students learning with ELM-ART were more often successful in solving
the third, most difficult programming problem. Interestingly, this effect not only holds for
learners with previous programming knowledge but also for the very beginners. This may be
interpreted as a hint that adaptive techniques in combination with interactive feedback and
knowledge-based problem solving support may result in a more successful learning situation.
The main problem with most studies investigating the effects of adaptive link annotation and
individual curriculum sequencing not showing up clear effects may stem from the very specific
learning situations. Most of these studies were done with learners that were introduced to a totally
new domain. That is, the best learning strategy of the learners was to follow the pages of the
course from the beginning to the end like reading a book from the first to the last page. This was
the same sequence of pages that link annotation and guiding suggested.
In the beginning it was argued that adaptive techniques like link annotation and guiding will
be most effective in further education and especially in re-learning situations. What we need are
empirical investigation of learning situations where learners with totally different background
knowledge of the learning domain are compared. A first example of such an investigation was
done by Specht (1997) with his adaptive statistics tutor AST. In a re-learning situation, students
first had to answer an introductory questionnaire. In the Experimental Group, students were only
guided to pages where they had shown gaps in the introductory questionnaire while students in
the Control Group had to follow their own path through the course. The results show that students
from the Experimental Group much faster worked through the re-learning situation and answered
the questions more successfully in the final questionnaire. I expect that further investigations in
similar learning situations will support these preliminary results.
References
Brusilovsky, P. (1996). Methods and techniques of adaptive hypermedia. User Modeling and User-
Adapted Interaction, 6, 87-129.
Kay, J. (1995). The UM toolkit for cooperative user models. User Models and User Adapted Interaction, 4,
149-196.
Khan, B. H. (1997). Web-based instruction. Englewood Cliffs, NJ: Educational Technology Publications.
Mitchell, T. M., Keller, R. M., & Kedar-Cabelli, S. T. (1986). Explanation-based generalization: a unifying
view. Machine Learning, 1, 47-80.
Kolodner, J. L. (1993). Case-based reasoning. San Mateo, CA: Morgan Kaufmann.
Specht, M. (1997). Adaptive Methoden in computerbasierten Lehr/Lernumgebungen. Dissertation:
Universit‰t Trier.
Weber, G. (1996a). Episodic learner modeling. Cognitive Science, 20, 195-236.
Weber, G. (1996b). Individual selection of examples in an intelligent programming environment. Journal
of Artificial Intelligence in Education, 7, 3-31.
Weber, G. & Möllenberg, A. (1995). ELM programming environment: A tutoring system for LISP
beginners. In K. F. Wender, F. Schmalhofer, & H.-D. Böcker (Eds.), Cognition and computer
programming (pp. 373-408). Norwood, NJ: Ablex Publishing Corporation.