Definition of terms

The following gives the definitions of terms used in this guideline.

  • Information design:

It refers to the design about what information to convey and in what form in order for it to be easily understandable for the user.

  • Task:

It refers to the entire work that needs to be performed. It consists of one or more subtasks.

  • Subtask:

It refers to the units of work that constitute a task. In general, the subtasks described in the paper-based manual can be further broken down into smaller subtasks.

  • Initial state:

It refers to the state before starting a task. Performing a subtask allows one to advance to the next state.

  • Target (goal) state:

It refers to the state after completion of a task.

  • User:

It refers to the individual who performs a task using the system.

  • Object (target object):

It refers to the parts and tools needed to perform a subtask. (When a block needs to be assembled in a task, the object is the “block.”)

  • 3D position and rotation:

The 3D position and rotation refer to the positional information expressed using the three parameters (x, y, z) and the 3D rotation information such as the gradient expressed using the three parameters, respectively.

  • Tracking DOF(degree of freedom):

This refers to the detectable information on the 3D position and rotation of the user or object. This guideline expresses the tracking DOF using the following four types: “3D position and rotation,” “3D position only,” “3D rotation only,” and “N/A,” depending on the detectable information.

Example of 3D position tracking of a block

Example of 3D rotation tracking of a block

  • Global (world) coordinates:

It refers to the mechanism that determines the position and rotation information of a user or an object in the space used by the system. It consists of a reference origin and coordinate axes.

  • Information presentation method:

It represents the method of expression used in presenting the required information in real space using images, videos, and 3D models.

 


Information design steps (flow) in the system development

The important part part of the “Information presentation method”  in the information design of the AR system can be determined using the “Subtask type” and “Degrees of freedom available in the tracking technology.” Accordingly, in this guideline, the formulation of the design table summarizing the information design of the AR system for supporting work consists broadly of three stages as follows:

1. Decomposition the task into subtasks

2. Decision the tracking DoF for each subtask

3. Decision the presentation method  for each subtask

In the information design of the work support system, the contents of the tasks were checked and Decomposed into subtasks based on the type of subtasks set in this guideline. For each detailed subtask, the available tracking DOF were investigated considering the device. Lastly, for each subtask, the information presentation method was selected based on the tracking DOF detected, completing the information design for the entire task. The following describes each of the stages used in this guideline and the system design after the design table was created:

1. Decomposition the task into subtasks

Tasks vary from simple to complex. In this guideline, the subtasks were grouped to contain a certain level of complexity to  be able to handle all possible tasks. Accordingly, six types of subtasks were provided and the Task Decomposition was done using these (“subtask types” can be checked using list of subtasks). Sequentially, the task was checked and then broken down into subtasks.

Checking the task

Observe the task that needs to be supported. Check the procedure starting from the initial state of the task (initial state Si) up to end state (target/goal state Sg). For work involving the assembly of the block shown below, understand each work procedure (step) involved in the Si and Sg, respectively, where the block was not yet placed on the foundation.


・Decomposition the task into subtasks

Based on the procedure each task, decompose the task into subtasks. This was performed by categorizing the subtasks into the six types of work listed in the subtask table. The following is an example of decomposition the block assembly task into n subtasks (A1–An denote the subtasks).

A1Pick up the target block (for installation) (blue 2 × 4 rectangular block) (Selection)
A2Block (selected in A1) is placed at the installation location on the foundation (Installation)

An-1Pick up the block (red 2 × 4 rectangular block) to be placed (Selection)
AnBlock (selected in An-1) is placed at the installation location on the blue 2 × 2 block and yellow 2 × 4 block (Installation)

The Decomposition into subtasks


2. Decision the tracking DoF

Investigate the available DOF for user tracking using the HMD for the entire task. Additionally, investigate the available object tracking DOF for each of the subtasks in the decomposition. As for the tracking DOF, in addition to technologies enabling the collection of position and rotation information, such as RGB-D camera and markers and optical motion capture, technologies using embedded gyro sensors were available where only rotation information could only be collected. This guideline uses user tracking and two types of object tracking, defining the 3D position and rotation for the user and object in terms of the global coordinate system set within the space used, as shown in the following figure.

 

Types of tracking

・User tracking: Shall be able to track 3D position and rotation of the HMD using global coordinate system

Example) HoloLens functions, etc.

Global coordinate system-based object tracking: Shall be able to sequentially track the 3D position and  rotation of the object (part or tool used in the subtask) using the global coordinate system.

Example) When calculating the position and rotation of the receiver affixed to the object in terms of the transmitter placed inside the environment, magnetic sensors (such as Flock of Birds) are used. However, the detection and recognition of an object using the HMD camera is done by attaching the reference markers to the object or preparing a 3D model of the object.

HMD-based object tracking: shall be able to sequentially track the 3D position and rotation of the object with respect to the HMD

Example) The detection and recognition of an object using the HMD camera is done by attaching the reference markers to the object or preparing a 3D model of the object.

For the entire task, investigate the available degrees of freedom for user tracking. Moreover, for each of the subtask, investigate the types and the degrees of freedom of the object tracking that is applicable to the parts and tools used.

Decision the tracking DOF


3. Decision the presentation method

Based on the detailed subtasks and decided tracking degrees of freedom(DOF), detect the optimal information presentation method for each of the subtasks. Use the information presentation method selection tool in selecting the type of information presentation method.

Information presentation method selection tool

This is a tool for displaying the optimal information presentation method for each of the subtasks based on the input of the “subtask type” and “tracking type.” After the input, if more than one information presentation method is displayed, select a suitable option considering the content of the support. Consequently, the design table summarizing the “subtask order,” “subtask name,” “subtask type,” and “information presentation method” can be created.

Information presentation method selection tool


Development after creating the design table
 
In producing this guideline, we developed various AR systems. We summarized the concepts obtained in the development of the systems. After the design is completed, we believe that better systems can be implemented by referring to this information.