chap 7

Designing the User Interface : Strategies for
Effective Human-Computer Interaction
Sixth Edition

Chapter 7

Direct Manipulation and
Immersive Environments

Copyright © 2017 Pearson Education, Inc. All Rights Reserved

Copyright © 2017 Pearson Education, Inc. All Rights Reserved

Direct Manipulation and Immersive
Environments

Topics

1. Introduction

2. What is Direct Manipulation?

3. Some examples of Direct Manipulation

4. 2D and 3D Interfaces

5. Teleoperation and Presence

6. Augmented and Virtual Reality

Copyright © 2017 Pearson Education, Inc. All Rights Reserved

Introduction

• Positive feelings associated with good user

interfaces

– Mastery of the interface
– Competence in performing tasks
– Ease in learning the system originally and in

assimilating advanced features

– Confidence in the capacity to retain mastery over time
– Enjoyment in using the system
– Eagerness to show the system off to novices
– Desire to explore more powerful aspects of the

system

Copyright © 2017 Pearson Education, Inc. All Rights Reserved

Principles of Direct Manipulation

1. Continuous representations of the objects and actions

of interest with meaningful visual metaphors.

2. Physical actions or presses of labeled buttons, instead

of complex syntax.

3. Rapid, incremental, reversible actions whose effects on

the objects of interest are visible immediately.

Copyright © 2017 Pearson Education, Inc. All Rights Reserved

Attributes of Direct Manipulation (1 of 4)

•Novices can learn basic functionality quickly, usually through a demonstration by
a more experienced user

•Experts can work rapidly to carry out a wide range of tasks, even defining new
functions and features

– Knowledgeable intermittent users can retain operational concepts

– Error messages are rarely needed

•Users

– Immediately see whether their actions are furthering their goals, and, if the

actions are counterproductive, they can simply change the direction of their
activity

– Experience less anxiety because the interface is comprehensible and

because actions can be reversed easily

– Gain a sense of confidence and mastery because they are the initiators of

action, they feel in control, and they can predict the interface’s responses

Copyright © 2017 Pearson Education, Inc. All Rights Reserved

Attributes of Direct Manipulation (2 of 4)

• One way of trying to understand and categorize the direct

manipulation metaphor is by looking at the translational
distance between users and the representation of the
metaphor, which will be referred to as strength

– Strength can be perceived along a continuum from

weak to immersed

– This can be further described as the level of

indirectness between the user’s physical actions and
the actions in the virtual space

Copyright © 2017 Pearson Education, Inc. All Rights Reserved

Attributes of Direct Manipulation Systems
(3 of 4)

• Multi-touch allows new actions to be assigned to various

combinations of finger touches

– The 2-finger actions like zoom in/out are intuitive, but

others must be learned and take longer to discover

– This accounts for why a young child can easily learn

to tap, change screens, and touch on a tablet (the
intuitive actions), but does not have the skills to re-
arrange the icons on the screen (the learned actions)

Copyright © 2017 Pearson Education, Inc. All Rights Reserved

Attributes of Direct Manipulation Systems
(4 of 4)

• Examples of Translational Distances

– Weak - early video game controllers
– Medium - touch screens, multi-touch
– Strong - data glove, gesturing, manipulating tangible

objects

– Immersive - virtual reality, oculus rift

https://www.oculus.com/en-us/press-kit-hardware/

Copyright © 2017 Pearson Education, Inc. All Rights Reserved

Direct Manipulation Systems (example)
(1 of 5)

• Three users working concurrently on a large tabletop

touch device.

– They can use their hands and fingers to manipulate

the objects on the device

– Note the use of the different hand gestures

Copyright © 2017 Pearson Education, Inc. All Rights Reserved

Direct Manipulation Systems (example)
(2 of 5)

• A tangible user interface for molecular biology, developed in

Art Olson’s Laboratory at The Scripps Research Institute

– Utilizes auto-fabricated molecular models tracked with the

Augmented Reality Toolkit (from the University of
Washington Human Interface Technology Lab)

– The video camera on the laptop captures the molecule’s

position and orientation, enabling the molecular modeling
software to display information such as the
attractive/repulsive forces surrounding the molecule

Copyright © 2017 Pearson Education, Inc. All Rights Reserved

Discussion of Direct Manipulation

Problems with direct manipulation

• Spatial or visual representations can be too spread out

• High-level flowcharts and database-schema can become

confusing

• Designs may force valuable information off of the screen

• Users must learn the graphical representations

• The visual representation may be misleading

• Typing commands with the keyboard may be faster

Copyright © 2017 Pearson Education, Inc. All Rights Reserved

Direct Manipulation Systems (example)
(3 of 5)

• GPS solutions

– This is a screenshot from Google StreetView of the inside of the

University Center at Nova Southeastern University in Florida

– On the bottom is a scrollable image of other views on campus. In

the left hand corner is a more conventional static map showing
the physical street location of the campus

– Users can move the “person” to a different location on campus

and the views will change accordingly

Copyright © 2017 Pearson Education, Inc. All Rights Reserved

Direct Manipulation Systems (another
example) (4 of 5)

• Video games

– Woman playing World of Warcraft, using both her

keyboard and mouse, while hearing sounds of the
game via her headset

Copyright © 2017 Pearson Education, Inc. All Rights Reserved

Direct Manipulation Systems (another
example) (5 of 5)

• 3D Printing

– This shows astronaut (Bruce Wilmore) onboard the International

Space Station with the ratchet wrench that was created with
Made in Space’s 3D printer

– This device was designed, qualified, tested, and printed in space

in less than one week

Copyright © 2017 Pearson Education, Inc. All Rights Reserved

DISCUSSION

• An airline company is designing a new on-line reservation

system. They want to add some direct-manipulation
features. For example, they would like customers to click
a map to specify the departure cities and the
destinations, and to click on the calendar to indicate their
schedules. From your point of view, list one benefits and
one problems of the new idea compared with their old
system, which required the customer to do the job by
typing text.

Copyright © 2017 Pearson Education, Inc. All Rights Reserved

2D and 3D Interfaces

• “Pure” 3D interfaces have strong utility in some contexts,

for example:

– Medical
– Product design
– Scientific visualization

• In some situations, 2D may actually be preferable to

simplify interactions

• The power of 3D interfaces lies in applying them in the

appropriate domain or context where the added
dimension provides more understanding and improves
task outcomes

Copyright © 2017 Pearson Education, Inc. All Rights Reserved

3D Interfaces (1 of 4)

• By using a medical simulation inserted into a large scale

visualization (using CAVE technology), physicians were
able to find a solution, that would not have been possible
without doing the actual surgery

Copyright © 2017 Pearson Education, Inc. All Rights Reserved

3D Interfaces (2 of 4)

Features for effective 3D

– Use occlusion, shadows, perspective, and other 3D techniques carefully

– Minimize the number of navigation steps for users to accomplish their

tasks

– Keep text readable

– Avoid unnecessary visual clutter, distraction, contrast shifts, and

reflections

– Simplify user movement

– Prevent errors

– Simplify object movement

– Organize groups of items in aligned structures to allow rapid visual

search

– Enable users to construct visual groups to support spatial recall

Copyright © 2017 Pearson Education, Inc. All Rights Reserved

3D Interfaces (3 of 4)

Guidelines for inclusion of enhanced 3D features:

– Provide overviews so users can see the big picture
– Allow teleoperation
– Offer X-ray vision so users can see into or beyond

objects

– Provide history keeping
– Permit rich user actions on objects
– Enable remote collaboration
– Give users control over explanatory text and let users

select for details on demand

– Offer tools to select, mark, and measure

Copyright © 2017 Pearson Education, Inc. All Rights Reserved

3D Interfaces (4 of 4)

Guidelines for inclusion of enhanced 3D features
(continued):

– Implement dynamic queries to rapidly filter out

unneeded items

– Support semantic zooming and movement
– Enable landmarks to show themselves even at a

distance

– Allow multiple coordinated views
– Develop novel 3D icons to represent concepts that

are more recognizable and memorable

Copyright © 2017 Pearson Education, Inc. All Rights Reserved

Teleoperation and Presence

• Two “parents”:

– direct manipulation in personal computers
– process control in complex environments

• Physical operation is remote

• Complicating factors in the architecture of remote

environments:

– Time delays

▪ transmission delays
▪ operation delays

– Incomplete feedback
– Feedback from multiple sources
– Unanticipated interferences

Copyright © 2017 Pearson Education, Inc. All Rights Reserved

Teleoperation and Presence (example) (1 of 2)

• Nurse demonstrates using the Telehealth cart otoscope to

conduct a real-time tympanic (ear) membrane exam

• On screen is Physician Assistant who, from a remote location,

can see and evaluate the patient and provide an appropriate
plan of care

Copyright © 2017 Pearson Education, Inc. All Rights Reserved

Teleoperation and Presence (example) (2 of 2)

• Immerse Board allows two users to be co-located and

work on the same shared screen

Copyright © 2017 Pearson Education, Inc. All Rights Reserved

Augmented and Virtual Reality

• Virtual reality breaks the physical limitations of space and

allow users to act as though they were somewhere else

• Augmented reality shows the real world with an overlay of

additional overlay

• Situational awareness shows information about the real

world that surrounds you by tracking your movements in
a computer model

• Augmented reality is an important variant

– Enables users to see the real world with an overlay of

additional interaction.

Copyright © 2017 Pearson Education, Inc. All Rights Reserved

Augmented and Virtual Reality (example)
(1 of 4)

• Homeowner can fix plumbing problem, with guidance and direction

from plumber

• Plumber (actually Dad) and plumbing problem are not in the same

location, but by use of augmented reality, both people can clearly see
the problem and give directions as to how to resolve it

Copyright © 2017 Pearson Education, Inc. All Rights Reserved

Augmented and Virtual Reality (another
example) (2 of 4)

• Using augmented reality overlays, various points of interest can be

shown on a mobile phone

• Icons represent the type of place (food, shopping, etc.) and distances

from the current location

• Links are provided to user reviews

Copyright © 2017 Pearson Education, Inc. All Rights Reserved

Augmented and Virtual Reality (another
example) (3 of 4)

• Customers can use their personal mobile devices to pull

up objects from the IKEA Catalog and see how the
various items would look in their own house

Copyright © 2017 Pearson Education, Inc. All Rights Reserved

Augmented and Virtual Reality (another
example) (4 of 4)

• Image of a virtual meditative world that users can use for engaging in

meditation activities

– The virtual world has sounds
– They change with each chakra (stage) of the meditation process
– This is an application of positive computing

Copyright © 2017 Pearson Education, Inc. All Rights Reserved

Copyright