This copy of the report is meant for paper publication, therefore may make better reading as a printed document.

1 Introduction to MetaFace Research

2 Objectives

A user - the client - can interact with the 3D animated face, in most cases using the keyboard to type natural language. Interactions form the basis for MetaFace finding information and displaying it in the web browser. The Virtual Face is cast in a supportive and helpful role and is able to respond to the user with animation and speech (using the MPEG-4 standard). Interaction can also occur inside the web browser via Java, HTML and any other technologies enabled by the browser. Interaction data is stored by the system and used to improve service as a user's behaviour and preferences are learnt.

The server side of the system operates by taking user interactions, form the client via the network, and attempting to retrieve the relevant data. Raw data that normally makes up a web site is described using XML (eXtensible Markup Language) and MPEG-7, the resulting metadata then stored in a database for quick and powerful searching. A multi-agent sub-system is used on the server, and is simply a collection of agents that interpret user interactions in their own specific ways, then search the database(s) for information. As an example one agent might know how to look for audio, another for HTML documents. Once the information that a user requires has been formulated, the corresponding raw data is retrieved and the personality determines how the Virtual Face will act and what it will communicate. The personality determines a response based on its environment (i.e. the relevance of data, the time taken, the size of downloads etc.). All data is then sent to the client in an efficient way. The server appears transparent; the client interacts with the Virtual Face instead of the complexity of the server.

The following exhibit the need for extensive research, and are to be used in the design of MetaFace:

3 Background

3.1 Metaphors and the World Wide Web

Metaphors are used in computer systems in order to bring real life closer to a computer abstraction. To date, the most predominant metaphor used in computer systems is the desktop metaphor, but now as we enter the 21st century, the web and its legacy of ad-hoc metaphors are becoming more and more a part of our lives.

The user's relationship with media technology is an ongoing research area. Researchers have discovered that humans employ a wide range of techniques in order to bring a media experience closer to real life. Reeves & Nass (1996) draw the conclusion that humans perceive "media = real life" when interacting with media, and that it is actually less mental work to sustain this belief than to interact any differently. Reeves and Nass also assert that a reason for treating media this way could be the fact that humans have not evolved with computers and media, instead humans and computers have been thrust together.

There are many ways in which users perceive the Internet, and Bruce (1999) has conducted research showing that these perceptions can be classified into several metaphorical groups. As this and other research shows (Palmquist, 1996), the web doesn't have a single metaphor that is consistent between all users, and yet metaphors can be considered powerful tools when using unfamiliar and or complex entities.

By retrieving and analysing online literature for metaphors, Palmquist (1996) has found that metaphors pertaining to the anthropomorphic group occur very often. Large proportions of people see the Internet as an entity relating to human life yet there exists no explicit metaphor implementation to capitalize upon this.

Bruce (1999) states that people using the perhaps-unfamiliar Internet are identifying with concepts that are common place or easily assimilated with existing cognitive structures. Marcus (1993) further articulates that there is a need for collections of metaphors in systems to convey the data and/or functional capabilities of a system.

There are various cognitive problems with the existing desktop metaphor, as outlined by (Krueger, 1999). The reason that the virtual desktop has been so successful to date, despite the limitations and mismatches in its implementation, is because of the metaphor's likeness to our perception of the world. The web, however, has no premeditated metaphor to enjoy the success that the virtual desktop has captured.

The Internet is very complex by nature. A recent attempt at graphing the Internet (Brodner et al., 2000) has likened the web to a complicated organism exhibiting a morphological structure. To generalize the results, it was found that only 30% of web sites make up the well connected inner core of the Internet, 20% of links lead towards this core whilst nearly 50% literally go nowhere. A buzzword commonly used today is "information overload". With all of the information available on the Internet, and the fact that it is interconnected so haphazardly, people feel overwhelmed by the immensity of information and the effort required to search through it to find what they want. Users have no one to turn to, or help them; this is where MetaFace will apply its efforts.

There is a need for re-thinking the way in which we associate with the Internet and the metaphors involved. In this association perhaps the use of an anthropomorphic metaphor, so prevalent in existing literature, could reduce the difficulty of interaction. The complexity of the human social structure is second to none, but humans, in their interaction, have special ways of dealing with this day-to-day complexity. This natural way of coping with complexity can be used to combat the involvedness of the online world.

3.2 Human/Computer Interaction and the World Wide Web

3.2.1 Internet Delay

MetaFace proposes to use a personality sub-system to alter users' attitudes towards delay. This will be achieved through having an awareness of its immediate environment (relevance of data, effective network transfer rates, size of downloads, etc.) and tailoring service according to the environment. The service will include incremental loading, quantitative feedback, user interaction and explanations for delay as well as emotive responses.

3.2.2 Finding Information on the Web

1. Failure to index all web pages. As of June 2nd 2000, no search engine has indexed more than half of the 1 billion pages estimated to be on the web (Sullivan, 2000b).
2. Inability to update the search engine index regularly (Cheong, 1996, p. 107). With a rapidly growing Internet, documents are added, deleted and updated on a daily basis, but due to the size of the Internet it is not possible to update an index that frequently.
3. Too many false hits due to searching large indexes that have vast information domains (Morris, 2000). For example, a soccer fanatic who queries "football" will get results including American football (Sullivan, 2000c).
4. Lack of variation that users have in their searches. An index is implemented in a fixed format meaning that a user can only search information according to that format. For example, search engines can't tell the difference between content and a description of content (Morris, 2000).
5. A lack of client side utilization. Currently computing devices connected to the Internet do little more than fill out forms and swap them back and forth with web servers until a job is completed (Bosak & Bray, 1999).
6. Inability of search engines to utilize the many different media types. AltaVista (AltaVista Company, 2000) is one of the few search engines to cater for multiple categories of media, it can search for images, video and MP3 audio, but even it can't index and search all media types, nor search the media in great detail.

Success rates for search engines are dropping as the Internet continues to grow and limitations become more obvious. In 1999 the success rate for a user finding what they were looking for was 77%, this is down from 1997 when the success rate was 86.6% (Sullivan, 2000a). There has been a steady drop over these three years suggesting a downward trend, but preliminary results in early 2000 suggest a sharp rise as new proprietary search engine technology comes to the fore (Sullivan, 2000a). Keeping in mind the limitations of search engines discussed previously, this increase in the success rate has little evidence of being permanent. It is foreseeable that centralized indexes will not be able to uphold these success rates with the rapid expansion of the Internet.

3.2.2.1 Vortals and Portals

Portals are similar to vortals, but instead of being in a fixed information domain, they provide an interface to information based on the individual. An example of a portal is project Raven by Lotus (Lotus, 2000), which is a corporate tool, utilizing catalogs and search engines in a web site. What a person sees on the site depends on their employment details and gives access to information that is most relevant to them. Two portals designed for the average Internet user are Telstra.com (Telstra, 2000) and My Yahoo! (Yahoo! Inc, 2000).

Portals and vortals address the first, second and third limitations of search engines discussed previously, but there still exists the problem of finding these sites to begin with. They are a scalable and decentralized solution, and soon we may see portals and vortals explicitly designed to index the location of other portals and vortals. MetaFace will use the idea of vortals and portals when constructing an index for a web site, thus gain the benefits.

3.2.2.2 Indexing with XML and MPEG-7

By using XML, data can be described in both its intrinsic and general structure. By storing the meta-data in this format it is possible to conduct searching in many different ways, and thus is a solution to the fourth limitation of search engines.

XML also makes it possible to distribute some processing load from servers to client machines (Khare & Rifkin, 1997), saving time and bandwidth (the fifth limitation of search engines). Bosak & Bray (1999) give an example of client side processing, which uses retrieval of XML data about airplane timetables. Along with the XML data, a small Java program is also downloaded from the Internet, which allows the XML data to be viewed and searched in many different ways. Instead of getting new data from the server every time the user has a query, all processing is done by the client. Multiply this by a few million Web users and the efficiency becomes very dramatic.

The goals of MPEG-7 are to describe multimedia content, manage data flexibility and globalize data resources (Nack & Lindsay, 1999). This could have a dramatic impact on the functionality of future indexing research, as the standard allows indexing of multiple types of media (a solution to the sixth limitation of current search engines). MPEG-7 proposes to achieve content description through the use of a DDL (Data Description Language), as described by Hunter (1999), which utilizes the ongoing work of XML.

An application that shares similarities to MetaFace is the ongoing DICEMAN project (DICEMAN, 2000). DICEMAN will attempt to develop an end-to-end chain of technologies for indexing, storage, search and trading of digital audio-visual content for professional end users (Schrooten, 1999). The difference is that DICEMAN targets professional end users for the purpose of trading media, and MetaFace targets the average user and provides a metaphor and anthropomorphic interface for the retrieval of media and information. MetaFace also provides an easy to use and intuitive interface, and will be among the first wave of applications to use the MPEG-7 and XML standards.

3.3 The Use of Anthropomorphic Interfaces

3.3.1 Domain Knowledge

One problem with domain knowledge is the resources (time, money, people etc.) required in construction. CHAT, an NLP (Natural Language Processing) database requires about 3 person-months for the construction of domain knowledge for a single area (Whalen, 1996). The aim of the FAQBot was to improve upon this time by taking FAQ's (frequently asked questions documents), readily available on the Internet, and converting them into a form that a computer program could use. There was a saving in time, but the performance of the system was reliant on the sometimes-poor quality of the FAQ's (Beard, 1999). Ananova (Ananova Ltd, 2000a), a virtual news caster, attempts to search the Internet, retrieve and broadcast news stories automatically, but there is a need for content specialists to monitor and maintain the retrieval (Ananova Ltd, 2000b).

Systems that utilize domain knowledge need an automated way of constructing it in order to save resources. There is also a need for a framework so that more than one domain can be catered for. MetaFace seeks to address both of these areas, especially since constructing XML domain knowledge is a very intensive process.

3.3.2 Personality

Being emotional whilst making decisions was previously thought of as irrational, but now research is questioning whether emotions are a way of making quick decisions in complex environments (Ventura, Custódio & Pinto-Ferreira, 1999). Most research into modeling emotional agents revolves around the use of internal emotional states that can be activated and influenced. These types of architectures use combinations of events, goals, and motivational forces in activating the different states, and some even employ learning algorithms to change the method of activation over time. Almost all research into these types of architectures has been theory driven in order to better understand the modeling of emotion, there has been little attempt for applications driven research to empower users.

El-Nasr et al. (1999) have conducted theory driven research into the use of an emotional architecture in conjunction with facial animation to convey emotional states. Their work to date has been successful, suggesting their principles could be used in more complex situations and end-user applications such as MetaFace.

The action expression problem is a well-known difficulty with emotional architectures - it occurs when emotions with diverging effects are activated in an oscillating matter (El-Nasr, Loerger & Yen, 1999). For example, an experience might be both joyful and distressful, the agent oscillating between the two emotional states. Petta (1999) among others has solved the action expression for a particular situation through the use of transitional emotional states, whilst other researchers have used dithering and blending effects.

One of the superficial personality techniques used in anthropomorphic interfaces is the back-story, which is a Hollywood term for the off-camera background information used to flesh out a character. The August system (Gustafson, Lindberg & Lundeburg, 1999), makes use of a back-story by having an agent that looks like August Strindberg, the 19th century actor. One of the reasons for modeling an actual person was to quickly give the agent some personality, Strindberg was famous for statements about politics, women, reviewers etc. (Lundeburg & Beskow, 1999). Ananova (Ananova Ltd, 2000a), a virtual news anchor, also makes use of a back-story, complete with photos of her construction, personal pages and a forum to interact with users (Berkowitz, 2000). Other superficial techniques, used by many projects, are the use of physical appearance and also scripted behaviour as in Plantec (1999). While this may convey some degree of personality to the user, often, scripted behaviour is too predictable (see section "3.3.3 Pedagogical Aspects") and appearances are not dynamic. Lifelike personality is something that must be programmed as well as be perceived by the user. A lifelike personality should also be persistent.

Two applications using anthropomorphic interfaces and trying to model lifelike personality are the PSA (Personal Service Assistant), an ongoing research project (Arafa, Charlton & Mamdani, 1999), and Ananova. When the PSA is implemented, it will rely on its immediate environment and interaction factors (resources, time, delay, importance etc.) when generating personality and emotion. Ananova generates personality and emotion based on the content of a news story, however a current limitation is the need for a team of human helpers to provide clues about the content of a news story (Ananova Ltd, 2000c).

MetaFace will construct a personality that is aware of its environment not only to automatically generate personality and emotion, but also to alter service (for example, not downloading a large video over a slow connection unless the user explicitly requests it). MetaFace will also provide a framework for site managers to alter the personality of the Virtual Face, according to their own needs. Neither application explicitly tries to create a persistent personality, which is another aspect that MetaFace will address.

3.3.3 Pedagogical Aspects

The most important aspect that pedagogical agents try to address is user learning. There is some research to suggest that these agents have potential and assist the learning process (Lester, et al., 1997). Other research shows less of a learning impact and more of an influence upon users' perceptions of complexity (van Mulken, André & Müller, 1998), (Lester, Zettlemoyer & Grégoire, 1999).

Motivation is a key ingredient in learning (Johnson, Rickel & Lester, 2000), and research has shown that pedagogical agents can achieve user motivation (Lester, Zettlemoyer & Grégoire, 1999), (Lester, et al., 1997). Motivation is important because it allows users to maintain their attention span for much longer as well as generate an interest in the material being presented.

Believability of pedagogical agents plays an important role in motivation (Lester & Stone, 1997). Johnson, Rickel & Lester (2000) assert that there are four rules that users must address when constructing a pedagogical agent. Firstly, the agent must show that it is always alive, and secondly, it must control its visual impact so as not to distract the user. The pedagogical agent must exhibit complex behaviour so as not to be easily predictable and boring, and lastly, it must display natural and unobtrusive behaviour. Bates (1994) shows that there are parallels between animated films and animated agents with respect to how they must sustain their believability.

MetaFace will sustain believability in order to gain motivation, a factor in consumer goodwill, as well as reduce users' perceptions of complexity through the use of an assistant/instructional role.

3.3.4 Personalized Interface Aspects

The three ways in which an interface agent may learn about a user are machine learning, end-user and knowledge-based approaches. In the beginning, as with real life, an assistant is not familiar with the habits and preferences of the person they are assisting. By watching how the person goes about tasks, being instructed by the person and learning from other assistants, the assistant agent can gradually become more and more useful, this is the concept of machine learning. When the user has to explicitly program the agent to account for their individual habits and preferences this is the end-user approach. The knowledge-based approach is where a third party constructs an information base for the agent to use. (Maes, 1994)

The problem with the knowledge-based approach is the resources that must go into construction, and that user habits and preferences may differ from those allowed by the designers. A dilemma with the end-user approach is that typically the users that can program the agent have less use for an interface agent, and those who can't program would get the most out of an interface agent (Maes, 1994). It is because of these limitations and problems that machine learning is the best approach for interface agents.

There is an ongoing debate about the merits of direct manipulation vs. indirect management. Maes is an advocate for indirect management, whereby agents are empowered to understand a user's goals, and users do not have to worry about initiating every step to achieve these goals (Maes, Petrie & Wiggins, 1997). Schneiderman as cited in (Dean, 2000), believes that attention needs to be directed at empowering users in order to achieve remarkable things. Shneiderman (1997) is an advocate for direct manipulation (using strict construction guidelines) and lists many benefits of this approach. Both sides of the argument have merit, and to benefit users, indirect management and direct manipulation should be combined to empower both agent and user.

Trust is an important aspect in a system that attempts to automate functions previously carried out by a human counterpart. Singh (2000) shows that there are many aspects to trust, but three maxims can be used to describe what it means to be trustworthy. Firstly, interface agents must be ethical and adhere to a high standard. Secondly, interface agents must look out for the interest of the user and collaborators. Lastly, failing all else, the interface agent must be rational. As an example of how important trustworthiness is, imagine that you were using an agent in something "…more serious than your choice in singers? What if agents were shopping for your medicine and your kid's education?" (Lanier, 1996).

MetaFace will use machine learning to capture relevant user characteristics and tailor its service accordingly. For example, a user may always check the latest news on web sites, so MetaFace may make this the first item it loads when visiting a new site. A combination of direct manipulation and indirect management will be used in the MetaFace system. The combination will be achieved through MetaFace either making suggestions, that can be directly manipulated by the user, or automating a process. When deciding whether to make a suggestion or automate a process, MetaFace will use heuristics that include a measure of trust as well as the environmental variables already discussed (see section "3.3.2 Personality Aspects"). For example, if a user asks for information about the Sydney Olympics, MetaFace may choose to automate the download of some small pictures, but it will confer with the user before downloading a large video.

3.4 Summary

The area of time delay was shown to be very important in the way that users perceive quality. In order to address time delay some potential solutions were to use iterative feedback and page loading, quantitative data about downloads, and explain the reasons for delay to the user. These time delay techniques could be achieved through the use of a personality with an awareness of the immediate environment.

To construct a better technology for finding information, vortals, portals, XML and MPEG-7 can be combined to reduce the six limitations that plague search engines. The solution is to have data for a domain residing at a specific site, use the XML standard to index the data, and use the MPEG-7 standard when deciding how the data should be described.

Domain knowledge, in anthropomorphic interfaces, was shown to use vast resources in construction, so there is a need for automation and a framework for the task.

Many emotional frameworks have been researched in a theory driven way, but as of yet none have been applied to applications that benefit users. There was a demonstrated need for addressing the action expression problem when designing an emotional architecture. Shown also were the superficial techniques that some projects use to give an illusion of personality, and how they were lacking. Also discussed were some complex attempts at personality that used the environment and content when determining emotional response.

The use of pedagogical agents was shown to aid learning, reduce the perception of complexity, and enhance motivation. To achieve these benefits it was revealed that these animated agents must try to sustain believability.

Of the three learning methods used in personalized interfaces, machine learning was concluded to be the best approach. It was also discovered that these agents must employ a mixture of indirect management and direct manipulation as well as address the need to be trustworthy.

4 Significance

The MetaFace system's metaphor will benefit users by providing an interface to Internet data that is intuitive to use, motivational, reduces perceptions of complexity, and thus makes it easy to search and navigate data. Site managers will have access to a framework that enables this and is extensible enough for application to many different domains.

The significance of the MetaFace system, its metaphor, and its framework, stems from the changing online climate. Internet users are growing tired of complex and hard to navigate web sites and, site managers concerned with human/computer-interaction issues, need frameworks that can be easily tailored to their site to address these issues. As consumers take more and more notice of a company's online presence the interface will come under scrutiny and there will need to be better ways to generate significant consumer goodwill. MetaFace is concerned with users' perceptions of a company when interacting with a web site, and the ways in which to affect these perceptions in a positive way (i.e. generating consumer goodwill).

A press release by Jupiter Communications Incorporated predicts that in 2005, US online consumers will spend in excess of $632 billion through off-line channels, based on web interaction (Jupiter Communications Inc, 2000). The long-term significance of MetaFace will be cultivated from the need for businesses of the future to establish an online presence (even if they are not trying to sell products online). Facial animation, personality, speech and ease of use are all combined to inspire consumer goodwill as well as providing easy access to information about merchandise and services. A news release by Zona Research states that an estimated $4.3 Billion in E-commerce is at risk each year because of frustrating time delay (Wong, 1999). Time delay frustration is one of the human/computer-interaction issues that the MetaFace system is attempting to alleviate.

5 Research Methodology

5.1 Hypotheses

1. It is possible to integrate existing technologies, outlined in the objectives, to implement the MetaFace system including its Virtual Face Metaphor and framework.
   1. The metaphor will adhere to design principles present in current literature.
   2. The framework will allow extensions to metaphor, and be applicable to many user tasks and information domains.
2. A MetaFace-based web site will be capable of :
   1. Making searching and navigation of a web site easier for users.
   2. Reducing the users' perceptions of complexity and information overload.
   3. Generating significant and worthwhile consumer goodwill.

5.2 Assumptions and Delimitations

When collating data from user case studies, the population sample will consist of people that have used the Internet before and vary in degree of skill.

The MetaFace system will provide access and avenues to information on a web server, but the system is not responsible for the correctness or quality of data and information. For the purpose of testing and evaluating, the system will be applied to a few specific information domains in which Curtin has expertise. By limiting the system to these domains, quality and correctness should be assured for the testing and evaluation phase.

The MetaFace system will be designed in a system independent manner, with a case study implementation on the Linux system. Attention will be paid to later porting to different platforms.

The MetaFace system will be designed and evaluated based on the delimitation of the hardware and network access existing at Curtin University of Technology's School of Computing. The MetaFace system will be evaluated for use outside of Curtin University as a comment for any future work.

5.3 Types of Investigation

5.3.1 Design and Demonstration / Action Research

When designing a software interface there must be particular attention paid to the human/computer-interaction. The cognitive design approach has guidelines for designing and implementing systems with mental models and metaphors, two concepts that are quintessential to the MetaFace system. The anthropomorphic design approach has strategies for designing and implementing systems that users perceive as human-like, as is the goal of MetaFace. These design approaches are to be used incrementally through out the software development life cycle.

5.3.2 Comparative

5.3.3 Evaluative

5.3.4 Meta-Analysis

6 Ethical Issues

7 Facilities and Resources

8 Data Storage

References