How people interact with a web telerobot


How people interact with a web telerobot

Prior to this research, there was no information available on how people interact with a web telerobot or other web device and there has never been, as far as I am aware, a telerobot with a large user base. This chapter seeks to provide information on how humans interacted with the telerobots built during the project. The emphasis in this chapter has been to pose questions that will provide general information which will be useful to those aiming to provide a web-based service or developing techniques for increasing the effectiveness of web-based teleoperation. It is anticipated that many of the behaviours investigated with data obtained from a web telerobot will be similar for other web devices.

A large user base and high levels of activity generate data that can be used to answer questions not previously considered. The particular questions addressed included:

  • What will people do with a telerobot?
  • What are people's attitudes to teleoperation?
  • How much interest is there in the technology and could a telerobot generate income?
  • How do people discover the telerobot and does Zipf's law apply to referring sites?
  • Can banner advertising fund a web telerobot?
  • Who is interested in the technology?
  • How many times do people want to try it?
  • How can teleoperable devices be shared and how long will people wait to control a telerobot being used by someone else?
  • How quickly will people issue instructions to a telerobot, how does this change as session length increases and what does this change tell us.
  • How variable is the time taken to issue instructions and how does the variability change as session length increases.

Some of these questions require tracking the actions of individuals using the methods described in section 4.6 and section 5.8 also includes an investigation of the commonly used method of tracking people by tracking internet addresses.

What telerobot operators build

Most telerobot operators make insufficient requests to build anything substantial (see Chapter 6 for session length analysis) but a few have built some impressive structures. Structures are recorded if they happen to impress someone working near the telerobot when they are built but often they are constructed during the night and knocked down by the next operator so remain unobserved. The most impressive structure seen was that built by Bjorn Lehnardt, shown in the left hand image of Figure 61.

Bjorn Lehnardt demonstrates precise control with teleoperation. Building on inverted triangular blocks with a single hand shows considerable ingenuity.
Figure 61

His structure demonstrates just what can be done with web telerobotics. Placing the inverted triangular blocks would be difficult for a human who was restricted to the use of one hand and the strategy he adopted was not immediately obvious. The right hand image of Figure 61 shows the structure before completion and reveals how the inverted triangular blocks were supported prior to placing the upper blocks.

Another impressive structure, shown in Figure 62, was the cantilevered bridge where John Willoughby endeavoured to span the maximum possible distance. 

Cantilevered bridge built by John Willoughby This demonstrates that it is possible to create complex structures using the telerobot.
Figure 62

The cantilevered bridge is finely balanced and required precise placement of blocks to avoid collapse. The 'Acropolis' and 'Fence' by unknown operators and 'Wall with Pipe' by Ryan Betting are further examples of successful teleoperation and are illustrated in Figure 63.

Some more two dimensional constructions. From left to right: 'Acropolis' and ’Fence’ by unknown operators and the 'Wall with Pipe' by Ryan Betting.
Figure 63

Most constructions are two dimensional including those illustrated above. Some of the much rarer non-planar structures are 'Square Castle' by an unknown operator, 'Round House' by Rick Sylander and 'Triangle' by Matt Murdock shown in Figure 64.

Some three dimensional structures. From left right: 'Square Castle' by unknown, 'Round House' by Rick Sylander and 'Triangle' by Matt Murdock.
Figure 64

These structures indicate that some operators find operating a telerobot sufficiently satisfying to be prepared to devote considerable effort to mastering the task. They also show precise control, ingenuity, and creativity. The originality shown demonstrates a principal advantage of teleoperation over automatic control for it is difficult to imagine these structures or their originality being reproduced by any form of machine intelligence.

Exploration by operators

There are a number of issues important to successfully completing tasks with telerobotics, one being the ease of interaction. Many researchers have suggested that mastering the software should be intrinsically motivating, that features should be revealed incrementally, and that the system should be at least minimally useful with no formal training Rieman (1994:8).

What is a user’s motivation when first coming across the telerobot? An obvious motivation is to explore the interface ie to try out each option and discover its effect. This is called “task free exploration”. Rieman (1994:8) studied the activities of 14 computer users for a total of 243 hours, operating a variety of interfaces, and found “no evidence of time spent by any subject in task-free exploratory learning of an interface” (Rieman 1994:43). The studied users give two reasons for this. The first being, that it was not a productive use of time and the second being, that it was not interesting. It was also shown that, for other than very simple systems, task-free exploration of an entire interface was likely to exhaust the user long before the user exhausts the possible combination of states and actions afforded by the system.

Task free exploration does not seem to be the approach taken by telerobot operators. Very few operators comprehensively explored the interface, the vast majority making insufficient requests for this to be possible. As discussed in section 4.12, few users tried moving the telerobot by clicking on the wire frame and there was very little exploration of image quality controls. From a sample of 30,800 requests discussed in section 4.11 less than 0.05% tried several of the image quality options. It was expected that data could be acquired on operators image quality requirements. What in fact occurred was that operatorss generally decide the defaults are acceptable and do not investigate the trade-off. These results confirm that opersators overwhelmingly did not adopt task free exploration.

An alternative strategy to task free exploration is to perform “task oriented exploration” where a user explores an interface in pursuit of a goal. In the case of the telerobot this may be, to build a block construction. Goals can be provided to the user as would typically occur in a working environment. Such goals are called "strongly posted goals". Alternatively a user may set their own goals, which are called “weakly posted goals”. As operating the telerobot is purely recreational any goals must be regarded as weakly posted. Rieman (1994:88) has predicted that weakly posted goals are an effective strategy for exploration. Such goals would be stated and pursued, but if success was not achieved within a reasonable amount of time and effort, they would be abandoned. He studied 22 subjects, each assigned the task of drawing and formatting a graph. Thirteen of the twenty sub-goals required to complete the task were dropped by some of the subjects after efforts to achieve them failed. He found there was a wide variation between subjects in the time they spent pursuing a goal unsuccessfully before giving up and pursuing a new goal. These times varied from 2 to 11 minutes. Rieman concluded there is no rational way to decide how long a goal should be pursued before it is dropped.

The first goal, which all operators must achieve, is to gain control of the telerobot. Once in control, the goals an operator is pursuing are unknown but the next goal, probably pursued by most operators is to move the telerobot. The likely goal after this is to manipulate blocks.

An examination of the requests made to the IRb6/L2-6 telerobot in Perth from 16 March 1995 to 25 September 1995 showed that 39% of the users who requested control of the telerobot did not make any requests to it on that occasion. This indicates a lack of enthusiasm by many operators for exploration of the interface at all. The goal of moving the telerobot can often be achieved with the first request after having gained control. 67% of users having gained control made only a single request suggesting their most ambitious goal was to move the telerobot. Many would have failed with this request as they may have requested the telerobot to go to a point which it can not reach or the telerobot was disabled, having been crashed in a previous request and requiring the reset function to be selected. These results suggest goals are weakly posted for most operators and quickly abandoned with a web telerobot. On the other hand there were some operators who pursued the ambitious goal of building complex structures. This number was small however, with only 2.5% making more than 10 telerobot requests in a session.

The first indication of the telerobot having crashed is its failure to move. An operator can then try the reset function or click on the hyperlink “Didn't move? Look in the log”. This brings a page containing information on the reason for the robot's failure to move. The reset function was not selected nearly as often as it should have been which (see section 4.12), provides further evidence of the low commitment to the goal of moving the telerobot.

As suggested by Rieman it appears that telerobot operators use task oriented exploration to understand the telerobot and as for Rieman's experiment, a widely varying level of persistence in the face of failure is encountered. However, the level of persistence with the telerobot is generally shorter than the 2 to 11 minutes found by Rieman. This suggests that where operators are not provided with strongly posted goals, web telerobots and other web devices must aim for interfaces that can allow the operator to immediately get some results.

Operator feedback

Operator comments were collected by inviting operators to submit their comments to a page available for all visitors to read. These comments were acquired with a HTML form and appended to a web page of previous comments using the software Polyform (Bracewell 1998). The recording of comments began on 20 May 1995. By 31 August 1998 there were 1,157 comments recorded, even though a few months of recording was lost due to hard disk failures. Only a small fraction of people choose to submit a comment (well under one percent of operators) and comments are likely to be biased towards those who are particularly interested in the telerobot. Some people, particularly those seeking a response, preferred to email one of the addresses which appeared on the web site rather than filling in the comment form. The 1,157 comments were categorised according to topic and the number addressing each topic counted. Those which addressed more than one topic were counted in each category they addressed so that the total of all categories is larger than the total number of comments recorded ie, 1,735. The results are presented in Table 13.

Category 1991 1995
Congratulations/Amazing 886 77%
Suggestion 278 24%
Complaint 116 10%
Reporting Fault 134 12%
Busy Telerobot 13 1%
Query 102 9%
Other Detailed Comments 187 16%
Garbled 19 2%
Total 1735 150%
Comments people have made regarding the telerobot categorised by subject from 20 May 1995 until 31 August 1998. Some people addressed more than one subject so the total of all categories is larger than the 1,157 comments recorded in total.
Table 13

By far the largest number of comments were expressions of congratulations or amazement that the person was causing physical changes at a point far removed from their current location. This is the source of novelty that many people find initially appealing. A few of the more effusive comments were:-

  • Man this thing is incredible!!! After 5 minutes I felt trembles 10 minutes later perspiration dripped from me oooops ... 3 hours later and I've got no chance of finishing that assignment!!!!! But the funny thing is I don't care, moving these bricks, dropping them on the floor, picking them up, woooooooooo I feel like a two year old all over again!!! GREAT STUFF! - Randall Fletcher (Fletch) 20 March 1995.
  • I thought I was finished playing with blocks! Well, I was wrong. Where else can you build a castle half-way around the world? Peter Krevat, M. D. Atlanta, GA, USA. - Peter Krevat, M. D. pkrevat@emory.edu 18 March 1995.
  • Brilliant! One of the most fascinating exhibits i have come across in my virtual travels...both in actuality and in implications for the future of interconnectivity! - Graham E Melley gemelley@ccinet.ab.ca 24 April 1995.
  • Awesome glimpse of the future - here and now ! I never dreamed I'd do something like this in my lifetime (I'm 53 yrs old). I now believe in the realities of remote telemetry and control - like out in space. It's one thing reading about it but something else to actually do it yourself, from your own home PC. Good one ! - Chris Warring cwarrin@ibm.net 27 March 1995.

Many operators offered suggestions as to how the telerobot could be improved but only a small number of these were perceived to be useful. Frequently suggestions included things which we had already considered and had decided not to implement for various reasons. One of the most common suggestions was live video. It is likely that this will be added at some point since it is an area of web technology that is improving rapidly. However, live video is difficult to implement in a way that does not swamp the available bandwidth. This technology in fact was in place for a time, using a method whereby an animated GIF file was continually forwarded to the browser using Web Video software from the University of Ulm (Wolf et al. 1998). This software worked well as the frame rate could be controlled and it could be set to only update when more than a set percentage of the image varied. The software requires a Microsoft Video for Windows compatible camera or frame grabber, but the one we had was not reliable so the live video was abandoned.

Some people were not happy about various aspects of the telerobot operation and left complaints. A common complaint was that the blocks had been swept from the table by other operators and a few people complained about not being able to access the telerobot, as it was always busy. Quite a few had queries. These were frequently already addressed in help pages on the site, which these operators had apparently not read. Other comments drew comparisons between the telerobot and other systems and concepts or addressed extraneous issues, for example the weather. A few people left garbled messages. The full list of comments can be seen at http://telerobot.mech.uwa.edu.au/robot/comindex.htm. Although this feedback provided data on people's reactions to the telerobot it was not used to provide conclusive evidence for evaluation of interface changes or other purposes.

Telerobot traffic and Zip's law

The large installed base of web browsers makes teleoperation suitable for many new application areas such as entertainment and sharing of expensive equipment where revenue is likely to be proportional to traffic. This makes traffic volumes, what affects traffic volume and the sources of traffic important issues. This section first examines traffic volume on the Perth IRb6/L2-6 telerobot early in its life. This is followed by analysis of traffic volume on the ABB1400 telerobot in 1998 and the implications for generating income. Traffic volume is measured by requests made to the ABB1400 telerobot as well as an alternative measure of the frequency of operator sessions. The frequency of operator sessions is compared with the frequency of sessions on the Carnegie IRb6/L2 telerobot and the implications of the difference for web service providers considered in the context of Zipf's law as it applies to web site traffic.

Records were not kept prior to 16 March 1995 but from then until 10 July 1995, there were 310,000 hits recorded on the telerobot web server. A hit is recorded for every element in a web page and can not easily be related to page views or requests to the telerobot. This statistic is commonly used to measure web site usage but the method is crude and not used for any further analysis in this thesis. These 310,000 hits equated to 72,000 requests to the Perth IRb6/L2-6 telerobot from 11,000 internet addresses including requests where only the observer page was returned. Although the web server received requests from 15,000 internet addresses, not all visitors attempted to operate the telerobot. The figures quoted understate traffic, as many records were lost. This is because the web server software only closed the log file correctly when the server was properly shut down so that whenever the computer failed no records since the web server was last started were saved. From mid May 1995, an upgrade of the web server made it possible to save the log file periodically to another file and the number of records lost was minimised by saving the log file hourly. As well, records were lost due to computer viruses and hard disk failures. The fact that the web server is accessed at least every few minutes was used to assess record completeness and telerobot availability. The intervals where the records show that the server has not been accessed for more than two hours have been summed to determine the percentage of time for which either the records have been lost or the telerobot was not available. The results are shown in Table 14.

Period Percentage Of Time For
Which No Data Exists
March 1995 (from 16th)     83
April 1995 75
May 1995 25
June 1995 18

Percentage of the time for which the telerobot was unavailable or operator data was lost in the early period due to system crashes.
Table 14

Table 14 shows that the traffic in the early period was significantly under recorded but the 72,000 requests recorded was high enough to prove there was a lot of interest in the concept of web teleoperation.

In 1998 a web site monitoring service, Webside Story (1998c) measured usage of the ABB1400 telerobot. The 'Cookie' method, described in section 4.6.4, was used to measure unique visitors. Webside Story counts page requests and reloads [ ] which, for a telerobot are the same thing, so that reloads and requests are summed to produce the requests to the telerobot. The requests to the ABB1400 telerobot and the numbers of unique visitors in the first eight months of 1998 are shown in Figure 65. 

Unique visitors and requests to the telerobot in 1998 as measured by Webside Story (1998a).
Figure 65

Figure 65 shows a high level of use by a large number of people which generates large sample sizes for analysis of operator behaviour. If it were possible to charge even a small fee per request this could generate significant income. For instance, a figure of 5 cents per page view would generate an income of $1620 per month, based on the first 8 months of 1998. The extent to which even a small fee would affect traffic remains untested and although in 1998 micro payment systems exist, they are not widely used.

Another way of measuring telerobot usage is by the frequency of sessions. This measure was made by counting the number of telerobot sessions with a request count of zero. The number of sessions on the IRb6/L2 Carnegie telerobot from 19 February 1997 to 25 June 1997 and the ABB1400 telerobot from 16 April to 25 June 1997 are shown in Figure 66.

Comparison between the Perth ABB1400 telerobot and the Carnegie IRb6/L2 telerobot from 19 February 1997 to 25 June 1997 [ ].
Figure 66

In the period, 16 April to 25 June 1997, the ABB1400 telerobot had just over twice as many sessions as the IRb6/L2 telerobot which raises the question of why this occurred. One possible explanation of the much higher traffic on the ABB1400 telerobot is that this telerobot was offline for shorter periods of time than the Carnegie IRb6/L2 telerobot. To test whether the time offline was the reason sessions per week differed between telerobots, the figures were recalculated assuming that sessions would have been at the same rate for the time the telerobot was offline. The rate used is the average rate for that week. This yields the result in Figure 67.

Sessions per week. Comparison between the Perth ABB1400 and Carnegie IRb6/L2 telerobots adjusting for time off line.
Figure 67

The gap in Figure 67 exists because it was produced at a later date than Figure 66 and some of the source data for the ABB1400 telerobot was lost when the hard disk became corrupted in the intervening period. Figure 67 shows that there was still much higher traffic on the ABB1400 telerobot after adjusting for time offline so that the higher frequency of sessions on the ABB1400 telerobot is not explained by it being offline less than the Carnegie IRb6/L2 telerobot.

It is possible that the ABB1400 telerobot was used more frequently than the IRb6/L2 telerobot because people preferred to operate it. An investigation of the number of requests issued by operators to each of the telerobots in a session (described in chapter 6) shows that the number of requests per session is slightly higher for the ABB1400 telerobot than for the IRb6/L2 Carnegie telerobot. The slightly larger number of requests per session with the ABB1400 telerobot suggests a slight preference for it compared to the Carnegie IRb6/L2 telerobot but the preference is small and the difference in frequency of use is large.

Another possible explanation of the much higher traffic on the ABB1400 telerobot than the Carnegie IRb6/L2 telerobot is Zipf's law. Zipf's (1965) law was originally applied to natural language. If words are ranked according to popularity and plotted against frequency of use then the number of times each word is used is inversely proportional to its rank. This plots as a straight line on a graph with logarithmic scales on both axes. That is, a Zipf distribution means the probability of selecting the i'th item from a ranked list is proportional to 1/i. Zipf’s law has also been applied successfully to other examples of popularity such as usage of books in a library. Zipf's law applies where there is perfect competition and Carlos et al (1995) have been able to show, from a sample of over half a million web page requests that Zipf's law applies to web page usage. Additionally, Glassman (1996) has found Zipf's law to apply to web page usage and Nielsen (1997c) has found Zipf’s law to apply to document requests at Sun’s web site but with the tail affected by the limited number of pages available. According to Zipf's law a more popular web site will receive disproportionately higher traffic than a less popular one. This is consistent with the comparison of the ABB1400 telerobot and the Carnegie IRb6/L2 telerobot where the more popular ABB1400 telerobot received a disproportionately higher frequency of use.

When revenue is proportional to traffic, the disproportionately higher traffic received by a more popular web site has profound implications for a service provided on the Web. When Zipf's law applies it means that the most popular site will receive twice the revenue of the next most popular and three times the revenue of the third most popular site. The difference in the quality of the service provided by the first and third most popular site may be very small and the cost of providing the service similar, but the revenue generated would be three times greater. This is different from most other forms of commerce where factors other than popularity also affect the number of customers attracted. For example, few people will travel to the other side of a city to buy a newspaper from the most popular newsagent in town and ignore the newsagents they pass along the way.

Referrals

Having established the importance of popularity for a web service that generates revenue proportional to traffic, the question arises as to how people find a web service. Most people arrive at a web site by following links from search engines or from other web sites. Nielsen (1997b) suggests traffic from referring sites also follows Zipf's law. Nielsen measured the frequency from which traffic was referred to useit.com from other websites in a 3-month period in 1997, and these results are shown in Figure 68.


Distribution of traffic referred to useit.com from other websites in a 3-month period during 1997 showing best-fit Zipf curve (Nielsen 1997b).
Figure 68

From the results shown above Nielsen concluded that "Even though the data is not a perfect match with the Zipf curve, it does seem to be the case that the referrals are reasonably close to the Zipf curve (Nielsen 1997b)."

To test whether Zipf's law applied to telerobot referrals, the frequency at which traffic was referred to the ABB1400 telerobot and the Carnegie IRb6/L2 telerobot from other web sites was measured. The Carnegie IRb6/L2 telerobot provides an example where Zipf's law does not apply. In this case 91% of the traffic was referred from the Perth ABB1400 telerobot server. This was established from a data set of 887 referrals for the period 13 July 1997 to 27 Aug 1997. A larger data set was not available due to the home page for the Carnegie IRb6/L2 telerobot being moved to a server for which log data was not accessible.

Data for 67,751 referrals to the ABB1400 telerobot was accumulated and plotted with the best fit Zipf curve and logarithmic scales on both graph axes in Figure 69.


Distribution of the source of referrals to the telerobot from other web sites and a comparison with the Zipf distribution. The Zipf distribution does not match the measured referrals but does approximate the referral data reasonably well.
Figure 69

The Zipf curve was fitted by choosing a value K in Equation 8 to minimise the chi squared statistic when applying the chi squared goodness of fit test between the observed and expected data.


Where: -

R    = Rank order of referring site.
eR = Expected Number of referrers for the Rth Ranked referring site.
K    = Expected number of referrers from the highest ranked site.

The analysis requires referrers to be ranked, however, it was not always clear how referrers should be categorised. Some referring pages were moved and it was unclear whether they should be regarded as the same referrer. For instance, for a period after the telerobot featured on the Net Talk Live radio program, the telerobot was linked from http://www.nettalklive.com/hotsites/jovan/ from which 577 referrals originated making this the 22nd ranked referrer. Later this link disappeared and a link was added on http://www.nettalklive.com/hotsites/jovan/archive.html which generated 209 referrals. Moreover, some sites link to the telerobot from more than one page and it was unclear whether they should be categorised separately or as a single referrer. Some sites are mirrored and it was also unclear whether they should be categorised separately. For instance, the number one referring page was http://www.yahoo.com/Computers_and_Internet/Internet/Entertainment/Interesting_Devices_Connected_to_the_Net/Robots/ having generated 6699 referrals in the data set, and the 15th ranked referrer was http://search.yahoo.com/bin/search with 720 referrals and Yahoo occured again as the 20th ranked referrer from the page http://www.yahoo.com/Computers_and_Internet/Internet/Interesting_Devices_Connected_to_the_Net/Robots/ with 590 referrals and there were a further 58 Yahoo pages that generated referrals to the telerobot including the UK yahoo page http://www.yahoo.co.uk/Computers_and_Internet/Internet/Entertainment/Interesting_Devices_Connected_to_the_Net/Robots/ and the Australian Yahoo page http://www.yahoo.com.au/Regional/Countries/Australia/Computers_and_Internet/Internet/Entertainment/Interesting_Devices_Connected_to_the_Net/Robots/. For the analysis in Figure 69 each referrer was regarded as different if the referring address was different from another in any way.

As seen in Figure 73, the Perth telerobot did show an approximate Zipf distribution of referrers but the discrepancy between the sample and the Zipf distribution is not due to sampling error. This is shown by the chi squared statistic of 5707 with 774 degrees of freedom which indicates that the probability of the difference being due to random sampling error is almost zero. Apart from the problem of classifying referrers, it was also observed that referral numbers changed over time. The telerobot was used by some school groups and was mentioned in the press from time to time. This causes short-term changes in the number of referrals from a particular address. This effect was strongest when the telerobot featured in radio shows Net Talk Live (1995) in the United States and Safari (Heldal 1998) in Norway. The largest number of requests to the ABB1400 telerobot recorded in a day (1782) occurred on the day the telerobot was featured on Net Talk Live. Referring sites also gain and fade in popularity over time which will affect the number of referrals they generate.

The uncertainty of categorising referrers and the change in referrer numbers over time suggests that Zipf's law does not provide an adequate explanation for referral data however, it does seem to provide a useful approximation.

Income from advertising

A web telerobot service aimed at providing entertainment could be funded by advertising revenue and people with whom I have discussed the telerobot project commonly suggest this. Web advertising is commonly implemented as banner advertising. The banner is an image whose content is chosen by the advertiser, included in a web page that provides content of interest. The banner is served from an advertiser's web server and the rest of the page is served from the content providers web server. Clicking on the image will transfer a person to the advertiser's web site. One large banner advertiser pays sites US$0.0075 per impression and this is a fairly typical figure (Welch 1997). For the first eight months of 1998 (see Figure 65) the ABB1400 telerobot delivered an average of 32,439 operator and observer pages per month which would generate an income of US$243. Because observers do not stay for long, most of the traffic is created by telerobot operators. But there can be only one operator at a time and the mean time between requests during a session for the ABB1400 telerobot was measured at 56 seconds. If the telerobot was always in use, the maximum impressions provided to operators would be 47,800 (with perhaps 3,500 impressions provided to observers) giving a total of 51,300 impressions. This would generate an income of US$385 per month, far from enough to cover cost of running the telerobot. Some web telerobots, where the operator does not significantly alter the environment, can allow multiple operators which would increase the maximum possible income generated, but probably never to the point of paying for the site. Banner advertising does not appear capable of funding web telerobots or many other specialist services which prompts the question as to why people see and use many web sites funded by banner advertising. It is because most people use the same web sites some of the time in accordance Zipf's law described in section 5.4. This means the top few web sites can generate large revenues from banner advertising. One example is Yahoo which in 1997 was reported as providing 11 billion page views per year with the number growing by 400 percent per year (Nielsen 1997c).

Demographics of telerobot operators

It was apparent when demonstrating the telerobot to visitors on university open days that there was a diverse reaction to web telerobotics. Some visitors regarded the technology and its implications as fascinating while others saw it as pointless and these opinions seemed to frequently be related to age and sex. This is important for a service seeking to offer entertainment and is likely to influence the enthusiasm for other services such as the sharing of expensive equipment which may be more attractive to people who already have an interest in the technology. To test the relationship between demographics and interest in web teleoperation, demographic information was obtained from the user registration process described in section 4.7. Registered operators only controlled the telerobot for 8% of sessions or 15% of the time, therefore the demographic information is not complete and could be biased because some demographic groups may have been less willing to register. Registrations averaged 23 per week over a period of 34 weeks providing a database of 794 registered operators as of October 1997. Ninety five percent of the registered operators were male which is much higher than for internet usage generally, which in 1997 has been established at 68.7 percent of male users (Read et al. 1998). This indicates a much greater interest in web teleoperation among males.

The age profile of registered operators is compared with the overall internet users age profile (Read et al. 1998) in Figure 70.


Age profile of registered telerobot operators, showing that telerobot operators tend to be younger than the internet population generally (Read et al. 1998).
Figure 70

Results plotted in Figure 70 show that telerobot operators are much more skewed towards youth than the general internet population. The greatest interest in teleoperating a robot comes from people in their late teens. Therefore, a web teleoperation service aimed at entertainment is likely to appeal mainly to young males and it may well be easier to interest this group in other web teleoperation services for example sharing expensive equipment.

Return visits by telerobot operators

Another aspect of operator behaviour to be measured in this study is the number of times that operators return. There is no value to operators in manipulating blocks other than curiosity and entertainment, but measuring the number of times that operators return provides an indication of whether the novelty of manipulating blocks is sufficiently interesting to want to try it again. It also provides a measure of the experience of operators when examining their behaviour.

Initially, repeat visits was measured by counting repeat visits from the same internet address. This method of tracking individuals is commonly used by researchers, for example Saucy and Mondada, (1998) as it is easy to do. However, as discussed in section 4.6.1 there are errors introduced by dynamic internet addresses and proxy servers and, therefore, the initial measurements are followed by an investigation of the accuracy of tracking individuals by internet address over multiple sessions. Finally this section provides information on repeat visits by operators using the log-on name of registered operators to track individuals.

When tracking individuals by internet address a repeat visit was defined as a request to the telerobot from the same address more than 15 minutes after a previous request. The distribution of return visits is shown in Figure 71.

Return visits to the telerobot. Compiled by tracking internet addresses. A return visit is defined as a request to the telerobot more than 15 minutes after a previous request.
Figure 71

The results in Figure 71 show that 54% of operators had only a single session on the telerobot, while 24% had three sessions or more. Saucy and Mondada reported (1998) that the mobile web telerobot KhepOnTheWeb (Longchamp 1998) had a slightly higher proportion of one -time users, that is 67%, of operators had only a single session. Saucy and Mondada used internet addresses to identify visitors but repeat visits were defined as a request to the telerobot from the same address more than 10 minutes after a previous request and both data sets include requests from observers as well as operators.

The level of error introduced by tracking individuals by internet address was investigated by counting the number of internet addresses used by registered operators. This data is difficult to summarise, as the maximum number of internet addresses that can exist for a registered operator is limited to the number of sessions by that operator, which is frequently small. In addition, the total number of internet addresses that any particular person can access is limited, so the number of sessions per internet address should increase as the session count increases. Table 15 provides data collected between 31 January 1997 and 7 July 1997 on the relationship between number of sessions (row 1) and internet addresses (row 2). The number of operators with each combination of sessions and internet addresses is shown in row 3.

Number of Sessions per registered Operator 2 2 3 3 3 4 4 4 4 5 5 5 5 5 6 6 6 6 6 7 7 7  
Number of internet addresses 1 2 1 2 3 1 2 3 4 1 2 3 4 5 1 3 4 5 6 1 3 6  
Number of Occurrences 29 21 12 9 9 4 8 6 3 1 6 2 2 4 2 2 1 2 2 1 1 1  


Number of Sessions per registered Operator 8 8 8 8 9 9 10 10 11 12 12 12 13 13 14 14 15 15 17 20 23 53 58
Number of internet addresses 1 2 3 7 6 8 5 6 5 3 6 10 3 6 6 7 2 7 2 7 2 28 26
Number of Occurrences 2 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
A comparison of the number of sessions by registered operators to the number of internet addresses used by each over a period of 157 days.
Table 15

The table shows, for example, that there were 29 registered operators who had two sessions from the one internet address and that there were 21 registered operators who accessed the telerobot from two different internet addresses. These 50 (ie 29+21) operators had only two sessions on the telerobot in total. At the other end of the table is shown one registered operator who had 58 sessions with the telerobot from 26 different internet addresses. From the data in Table 15 it can be seen that if internet addresses are used to identify individuals, 21 out of 50 individuals or 42% would not have been recognised as having had two sessions and that the individual who had 58 sessions would be identified as having had only 26 sessions with the telerobot. For every number of sessions per registered operator shown in Table 15 there was a poor correlation between people and internet addresses.

The error introduced by proxy servers has the opposite effect. That is, different individuals are identified as the same person when tracked by internet address. No test was conducted for the error introduced by proxy servers but the data in Table 15 already shows that tracking people by internet address is inaccurate over periods greater than a session. The level of inaccuracy depends on the number of sessions but is always large. This is an important finding, which was unexpected and indicates tracking people by internet address, though commonly done, is unsuitable for counting repeat visits.

Having concluded that tracking individuals over multiple sessions is inadequate, sessions by registered operators were counted to determine frequency of return visits. From 31 January 1997 to 15 April 1998, 1,295 different registered operators used the telerobot. The distribution is shown in Figure 72.

Return visits to the telerobot. Compiled by counting the number of sessions for a registered operator.
Figure 72

Forty-four percent of operators had only a single session, and 29% had three or more sessions. These percentages are remarkably similar to those quoted above and obtained by measuring return visits by internet address. The distributions can be compared by normalising the data and plotting it on the same graph, as shown in Figure 73.

The distribution of return visits by registered operators and return visits as measured by internet address is similar but this is coincidental.
Figure 73

While the distribution of return visits, as shown in Figure 73, is similar whether measured by counting registered operator sessions or internet addresses, this must be regarded as coincidental due to the poor correlation between internet addresses and registered operators shown in Table 15.

It can be concluded from Figure 72 however, that most people who take the time to register are sufficiently interested in operating the telerobot to return, 22% of those that register will return five times or more and 10% will return 10 times or more and this does not include return visits where the person was unable to gain control of the telerobot.

Waiting to take control

When a web telerobot or other web device is made available to one operator at a time, any other person who wants to operate it must wait. This raises the question of how long people will wait to gain access to a web telerobot which is important for determining strategies for sharing web devices and for calculating the optimum number of devices required to satisfy a market for a teleoperated web service.

How long people will wait to gain access to a web telerobot was measured by tracking internet addresses as described in section 4.6.1, but as established in section 5.8 this method is unsuitable for tracking people over the long term and is therefore unsuitable to determine if a person who does not get access to the telerobot returns a considerable time later. It is possible, however, to measure how long a person will wait on any particular occasion. Occasions are defined as requests to access the telerobot from the same internet address at intervals of no longer than five hours from a previous request. A request made after an interval of five hours constitutes a separate occasion. Choosing the maximum period to regard as a single occasion was a question of judgement as the longer the period, the greater the error introduced by people trying again from another internet address. This measurement was made for all people who tried to gain control of the ABB1400 telerobot in the period 1 April 1997 to 2 September 1997 and gave up before they succeeded. The sample size is 4661 sessions. Figure 74 shows the distribution of the time that people waited to gain control of the telerobot before giving up.


Wait time to gain control of the telerobot. Few people are prepared to wait more than 3 minutes for the telerobot to become available before giving up.
Figure 74

Three quarters of the population gave up waiting after just three minutes and they did not return for at least five hours. This suggests that someone seeking to offer a teleoperable device as a web-based service will lose considerable custom if they do not provide enough devices to satisfy immediate demand.

An alternative to providing a device for each operator is to allow multiple simultaneous operators. In these cases requests are queued and carried out quickly enough to satisfy a limited number of operators. Multiple operators are common with pan and tilt cameras and were allowed with the Telegarden (Goldberg and Santarromana 1995), but this is only suitable when one operator can not alter the environment to the point that it interferes with another operator.

Time taken to make a request to the telerobot

One important question is how quickly can operators learn to control the telerobot? This is of interest because web services generally need to gain people's interest instantly and because most telerobot operators are novices. Once the learning period has been measured, there is potential for examining ways of reducing it or for examining the effect of more sophisticated control on the learning period.

The number of requests required to become accustomed to the interface can be inferred from the change in time per request as more requests are made during a session. The time per request is the sum of:-

  • Operator decision making time;
  • Communication time; and
  • Request performance time.

The communication time and request performance time are not related to the request number in a session so that, in a large sample, the mean and variability of the request performance time and the communication time will be constant for all requests. Therefore, any change in the time per request, as the number of requests increase, is due to a change in operator decision making time. To further reduce the effect of variability in the request performance time, the sample was restricted to the period before operators were able to specify only a single move in a request. The data sample used for this analysis was large covering 6,082 operator sessions and 50,624 requests by a telerobot operator from 31 January 1997 to 6 June 1997. In accordance with the general pattern of telerobot usage, shorter sessions dominate, which means that larger samples are available for shorter sessions. In some cases adjacent request numbers were combined to create bins representing a range of request numbers. The sample size falling into each bin can be seen in Figure 75.

The sample size for each bin in number of sessions and total requests for that bin. Longer sessions require fewer sessions to produce the same total number of requests in a bin than shorter sessions.
Figure 75

As can be seen from Figure 75 the sample sizes for longer sessions were increased by combining adjacent request numbers to produce a value that represented a range of requests. Longer sessions require fewer sessions to produce the same total number of requests in a bin than shorter sessions.

The mean time per request and the sample standard deviation were plotted against the request number in a session as shown in Figure 76.

Time taken per request to the telerobot and variability in time taken per request decreases as the number of requests in a session increases.
Figure 76

The mean time per request is shown to drop sharply in the first few requests. This is likely to be due to operators becoming accustomed to the telerobot as they make more requests. After eight or so requests the mean time per request drops only marginally, suggesting that an operator will have developed sufficient understanding to interpret and respond to the interface in the minimum time. The standard deviation of the time per request also reduces as the number of requests increases, following a similar pattern to the mean time per request providing further evidence that operators understand the telerobot after about the eighth request.

Comments