Abstract

This thesis describes an investigation of World Wide Web telerobotics. The notion of a web-controlled telerobot emerged from a study of the failure of robotics to achieve the expectations of the early pioneers in robotics. In the early 1980's roboticists were predicting that robots would become ubiquitous in the near future. This, however, has not eventuated and it is suggested that this failure is due mainly to a lack of robotic intelligence. The solution to the problem, presented in this thesis, is that humans should provide the missing intelligence and apply it to simple reduced instruction set robots using teleoperation and the World Wide Web.

To investigate this proposal three web telerobots were built, all of which could be used to manipulate toy blocks. Their operation and development is described including photographic imaging, robot control and issues of reliability. Techniques for recording and analysing operator behaviour are also discussed. The interface development process is described, and the result of operator reaction to the various interface elements trialed is documented.

Prior to this research nothing was known about how people would interact with a web telerobot. However, the large numbers of people who operated the telerobots have enabled insight to be gained into operator behaviour. Some impressive structures built by operators are illustrated and operator comments are categorised and discussed. This is followed by an analysis of telerobot traffic and the implications of this traffic for provision of teleoperated web services. It is suggested that revenue will flow disproportionately to the most popular services. Referrals to the telerobot are analysed and compared with Zipf's law. It is shown that most people who take the trouble to register to operate a telerobot will want to operate it more than once but that people will rarely wait more than three minutes to control a web telerobot on any particular occasion.

Also it is shown in this thesis that the number of requests in a session can be fitted by a Weibull distribution. The Weibull distribution allows the requests per session for each of the telerobots to be characterised by two parameters one of which defines the shape of the curve and the second defines the scale. The shape parameter is similar for the three telerobots built. This similarity can be used to reduce a comparison of operator preference to the scale parameter which describes how the distribution of requests per session for one telerobot can be rescaled to match another. Assuming that operators make more requests to a telerobot which they prefer, the scale parameter can then be used to compare telerobots for operator preference. Finally, suggestions are made for further research in the area of web telerobotics.

Acknowledgments

I am very grateful to the following people:

Firstly, I would like to thank my family who have had to share the trials and tribulations this research has required without the benefit of the satisfaction gained when things went well. I would like to say thankyou to my academic supervisor, James Trevelyan for his assistance throughout this period. Thankyou to Judith Rochecouste for editing my work, Barney Dalton a colleague, Matthew Carter, Shalini Cooray, Stephen LePage, Dan Macey, Mark Mordini, Peter Murphy, Troy Phillips, Gintaras Radzavinas, Bradley Saracik and Paul Woldan, who all contributed to the project at various times and to the many undergraduate students who contributed also.

Thanks also goes to the staff in the Department of Mechanical and Materials Engineering and in particular Ross Raabe and Carmel Trigwell, your support and assistance was greatly appreciated.

As I look back over the period spent on my research, there have been some really good moments and also some challenging and frustrating times. Thankyou once again to everyone who assisted me and if I have forgotten to mention anyone please accept my apologies.

Introduction

The web telerobot at the University of Western Australia first came on-line in September 1994. The idea of web control was conceived some months earlier as a technique by which human intelligence could be used for robot control. At the time of project commencement there were no physical devices controlled through a web interface and therefore no teleoperated devices available for operation by large numbers of people. The idea of using the Web to control a physical device was new and totally unproven. However, it was immediately apparent that people were interested in the concept, as the telerobot, which was then less sophisticated than it has since become, attracted a substantial number of operators.

Since then web telerobotics and web control of other physical devices has attracted widespread interest to the extent that there are now specialist sections of conferences dedicated to web telerobotics, for example, the Robots on the Web Workshop at the IEEE International Conference on Intelligent Robots and Systems 1998. In addition, there is at least one US company [ ] exploiting the technology commercially, a web interface will be used in NASA's 2001 rover mission to Mars (Backes 1998) and the first two groups to develop web telerobots have them exhibited in museums. A telerobot from University of Western Australia is on show at the Carnegie Science Center in Pittsburgh, USA and another was lent to the Australian National Science and Technology Centre. There is also the Telegarden exhibition at the Ars Electronica Centre Austria developed at the University of Southern California, Berkelely.

At the time of the first web telerobots, little was known about how to implement a web controlled device and more importantly, how people would interact with such a device. Much of this research was directed at investigating these issues.

This research commenced with the question, why has robotics failed to achieve the ambitions and expectations of the early robotics pioneers? The predictions of early robotics researchers are reviewed in section 2.2. One early expectation was that robots would dominate manufacturing enabling reduced labour costs. Robots were expected to be flexible with the same machine being suitable for a wide range of tasks. The popular perception included machines, perhaps with human-like form, that could operate with a considerable degree of autonomy and be quickly and easily assigned to new tasks. Simons (1980) predicted " that people alive today will come to have social and industrial intercourse with artificial machines at least as intelligent and capable as ourselves". Marsh (1982) suggested that the impact of robotics would be as significant as the industrial revolution. Many authors speculated on the social implications of this revolution particularly with regard to industrial relations and the impact of an environment where machines would provide most of society's needs and humans would be left to lead a life of leisure.

An examination of what has occurred in the intervening period follows in section 2.3. Statistics indicate that, worldwide, the growth in robotics has been substantial, but much lower than was expected in the early years of robotics. Growth is also variable it has been much greater in some countries, notably Japan, and in some industries, notably the automotive industry. The numbers of robots employed in industry continues to increase, growing by 6% worldwide in 1995 (Robotics and Europe 1997). Furthermore, the unit cost of robots continues to decline with the average unit cost worldwide decreasing by nearly one third between 1991 and 1995 and the range of tasks that robots perform continues to expand. Robotics has been a successful technology and it seems likely that, over time, the use of robots in manufacturing industries worldwide will approach that of industries and nations where robot use is highest. This will mean continued expansion in robotics and automation technology.

Many robotics practitioners believe the major opportunity for growth is in service robotics rather than industrial robotics and this is discussed in section 2.4. Joseph Engelberger, 'the father of robotics', is a leading proponent of this view (Engelberger 1989). Since Engelberger expressed his vision, many researchers have developed robotic solutions for the applications which he proposed. Some of the applications Engelberger and others propose as suitable for service robots and subsequent developments are discussed in section 2.4. These include the Autonomous Land Vehicle project, fast food service, farming, automotive fuel dispensing, commercial cleaning, the surgeon's assistant, and the nursing assistant. Engelberger has himself developed a robotic nursing assistant for hospitals, which is marketed as HelpMate. This enables an interesting comparison to be made between the vision and the product marketed in section 2.4.10. The product marketed was less capable than that envisaged. In section 2.4.11 it is concluded that while service applications offer opportunities for robotics, they often require more complex solutions than industrial robotics and this limits the suitability of robotics for service applications.

Despite the successes achieved in the field of robotics, it is widely felt in the robotics research community that robotics has not progressed as anticipated. This issue is explored in section 2.5. Durrant-Whyte articulated this view in the plenary session of the conference Field and Service Robotics 97. "In the past fifteen years remarkably little progress has been achieved by the robotics research community" (Durrant-Whyte 1997) and "too often we end up developing systems in which the original theory, techniques and technology are often too fragile, too expensive and inappropriate for an industrially-hard application." Certainly, the age of leisure that robots would bring about has not eventuated.

Applying robotic technology has often proved more difficult than was anticipated. As with the Autonomous Land Vehicle Project, frequently there is a large engineering effort required to close the gap between a system which works most of the time and one which always works. Furthermore, adopting robotics technology can require changes in the way things are done. An example cited is automotive fuel filling. There was an erroneous assumption that, because computing technology was so easily able to do things which humans found difficult, it could also do the things humans found trivial. This has not been the case, in fact, the everyday things which are intuitive to a human are complex to reproduce. For example to move about and do useful things actually requires a highly developed understanding of the world. It requires an appropriate response for every combination of circumstances a robot will encounter. Therefore, there needs to be a great deal more research success to realise the ambition of a robot able to learn and build up an understanding of its environment so that it can be given high level commands and self correct in much the same way as a human worker.

A major impetus for the development of robotics has been the desire to replace human labour with cheaper machine labour. The premise is that human labour is expensive and machines are cheap. In section 2.6 it is shown that the cost of labour is geographically determined and it is argued that an alternative to replacing human labour is to remove the geographic location of the human labour force as a determinant of labour cost. Section 2.7 argues that this can be achieved through teleoperation. Teleoperation is the control of a machine remotely. It has been applied to robotics and found application in hazardous environments such as undersea and space due to the impracticality of human presence. Telerobotics also removes the requirement to provide machines with significant intelligence and this offers the opportunity to build simple robots. They can be commanded to perform simple tasks, which in isolation are not useful. A human can interpret the robot's environment and then combine these simple tasks into useful functions. This could be termed a reduced instruction set robot. This strategy reduces the subset of circumstances to which the robot must be able to respond to a manageable number. An engineer can then develop a solution for possible combinations of circumstances that can occur. Solutions need not be comprehensive. There is not even a requirement for the robot to understand whether it has carried out the instruction successfully. A reduced instruction set approach would be to use a simple algorithm, accept that it will frequently fail and rely on the human giving the instruction to determine the appropriate action when it does.

Teleoperation has been the subject of considerable research with most systems being developed for hazardous environments. Prior to the commencement of this research all systems developed and studied had been intended for a small number of operators and usually had only one purpose-built operator station, which would have required extensive investment and development. However, with the advent of the World Wide Web there is an opportunity to control teleoperable robots and other devices through web browsers. As millions of people use web browsers, there are millions of operator stations available already at no additional cost.

The existence of multitudinous operator stations creates opportunities that have not existed before. Applications previously not suitable for teleoperation now become feasible, such as sharing of expensive equipment among many users eg. machine tools, teaching applications, entertainment etc. Questions arise as to what features can be offered within the constraints of a web browser, what features affect operator satisfaction and what are the demographic characteristics of people using the technology. Web teleoperation also provides new opportunities for analysing the behaviour of large numbers of people teleoperating a robot.

Having concluded that web telerobotics offers a way forward for robotics the rest of the thesis then describes the implementation of web telerobots and addresses a number of questions thought likely to be important for the successful use of the technology. There is a considerable history of telerobotics used for hazardous environments, so telerobotic control schemes are examined in section 3.1 including manual closed loop control, shared control, supervisory control and autonomous control. These machine control schemes are classified in increasing order of sophistication. Operator input devices for telerobotics are reviewed in section 3.2. Many of the workstation types, for example kinematically identical master and slave manipulators, are not suited to web telerobotics.

The spread of the Internet is discussed briefly in section 3.3.1 as its phenomenal growth and ubiquity has made it an attractive communications medium for telerobotics. Prior to the Web being used as a platform for teleoperation, some telerobotic projects used the Internet for communication. These are reviewed in section 3.3.2 including a distributed telerobotics laboratory shared by four universities and NASA's Jet Propulsion Laboratory. A further project reviewed in section 3.3.2 is the Channel Link Telerobotics experiment conducted by Buehlmeier (1995). Since the first web telerobots in 1994, many telerobots and teleoperable devices have been built and some of these are discussed in section 3.3.3 with an emphasis on the earlier installations.

The web telerobots built for this project and some of the issues involved are discussed in chapter 4. There were three separate robots connected to the Web. Initially a 6 axis ASEA IRb6/L2-6 was used. An ABB1400 robot later replaced this telerobot and another 5 axis IRb6/L2 robot was brought online at Carnegie Science Center in Pittsburgh, USA. The concept of a telerobot controllable through the World Wide Web, accessible to a large user base and not requiring software to be installed or special equipment at the operator's end has remained the same throughout the project. However, the implementation has been constantly modified. The telerobot software has been operated under three different operating systems, Windows 3.11, Windows 95 and Windows NT. A variety of frame grabbers and imaging techniques were trialed and the software constantly revised. The Web makes it possible to build an application from services provided by geographically dispersed computers. This was done with one version of the interface where a computer in Germany was used to generate a model of the robot, which was displayed on the operator's browser. This technique is also described in chapter 4 Section 4.1 describes how imaging for telerobot operators is implemented and section 4.2 describes how the telerobots are controlled. Reliability which is difficult to achieve with web telerobots is discussed in section 4.3. The software controlling the telerobots is described in section 4.4. Records available for analysis of operator behaviour are discussed in section 4.7 including the facility added to the software to record every request submitted by operators and the state of the robot after carrying out each request. In addition, a facility for accumulating operator demographic information and providing a method of tracking operators over multiple sessions is described. This allowed a comparison to be made with another commonly used technique for tracking operators over multiple sessions in section 5.8.

The approach adopted to develop the interface and the reasoning for it are described in section 4.8 including the cognitive frameworks for human computer interaction and possible interaction styles including command line, form fill and direct manipulation. Heuristic analysis of interface designs is also discussed in this section. The task-centred design process adopted for interface development is explained in section 4.9 and the methods adopted for representing three dimensional space and the six degrees of freedom of the robot to an operator through the medium of a two dimensional screen are presented in section 4.10. Larger and higher quality camera images are easier to interpret but require more time to transmit, therefore, operator image quality preferences are documented in section 4.11. Various techniques for specifying telerobot commands have been trialed including relative moves, absolute moves and point then click. Operator preference for these alternatives is presented in section 4.12.

The HTML interface is generated from templates with template variables, such as current position, replaced by the actual values during template processing. This method is described in section 4.5. Operators were provided with a facility to submit their own interface templates. In addition, an interface design competition was conducted to encourage submission of interface designs and ideas were sought from students. Some of these ideas are documented in section 4.13.

Chapter 5 investigates how people interact with a web telerobot. Some telerobot operators displayed considerable ingenuity and built impressive structures. A few are illustrated in section 5.1. Operators' approach to exploring the interface is discussed with reference to the interface exploration techniques of Rieman in section 5.2. A large number of comments have been submitted by operators and these are reviewed in section 5.3. Traffic statistics are provided in section 5.4 and the demographics of telerobot operators is documented in section 5.7 and compared with internet demographics. Further issues investigated in chapter 5 are:-

• How long will people wait to operate a telerobot?
• Traffic from referring sites and whether it matches a Zipf distribution as suggested by Nielsen (1997b).
• How often do operators return?
• Time taken for operators to make a request to the telerobot and the change in time taken as session length increases.
• Possible revenue sources for web telerobots such as sponsorship through banner advertising.

Chapter 6 proposes a new method for comparing telerobots for operator preference. Existing methods for comparing telerobots are discussed in section 6.1. The new method is based on comparing the number of requests operators make per session. Sessions may involve gaining control of the telerobot and few or even no further requests, or a session might last up to five or more hours and contain many requests. The distribution of number of requests in a session is found to fit a Weibull distribution. The properties of a Weibull distribution and a model that produces this distribution is described in section 6.2. Weibull distributions are most often used in reliability engineering to characterise machine failure (Walpole and Myers 1972:133) however some social phenomena which have been found to fit a Weibull distribution are examined in section 6.3. In section 6.4 it is shown that a Weibull distribution fits the data set better than any other special cases of the generalised gamma distribution. The Weibull curve is fitted to the number of requests in a session data, for three different telerobots and interface designs in section 6.5. When comparing telerobots in section 6.6 it can be seen that the number of requests in a session are quite different. However, one of the two parameters of the Weibull curve is similar for the telerobots. This similarity can be used to reduce the comparison between telerobots to a single parameter. The data sets can be directly compared to produce a single value that measures operator preference for one telerobot compared to the other. The quality of the comparison is affected by the size of the data sets and the degree to which they satisfy specified conditions. A method for measuring the quality of the comparison based on the correlation coefficient is also given in section 6.6.

Chapter 7 concludes the thesis with a review, summary of the results obtained and suggestions for further research.