Following is the summary of the knowhow related to the system development:

Index

  1. Using the HMD with a narrow viewing angle
  2. Displaying support information on the display (screen)
  3. Work using both hands
  4. Using video or animation
  5. Performing command operation
  6. Subtask: handling placement of an object

    1.Using the HMD with a narrow viewing angle

The CG viewing area of many HMDs is designed to be narrower than that of the viewing angle of the user. Accordingly, it would be good to check this during system trial runs because there is a possibility that the viewing range of the HMD is narrower than that of the CG despite the display screen (content) being developed within the CG display range. 

Example: Development screen for selecting wider blue block (2 × 4) and narrower blue block (2 × 2)

Example: Image of an HMD display screen for selecting the wider blue block (2 × 4) and narrower blue block (2 × 2) using the real system(The red frame shows the possible range of CG display)

As shown above, there are objects that are not visible when seen through the HMD, despite the assumption that that the entire CG object will fit within the user’s field of view during development. The following gaze guidance methods can be used when the objects that need to be within the field of view are outside the possible range of the CG in the HMD:

・Using arrows to show the location of the object.

・Using concentrated linework to show the location of the object.

・Using a small map showing the location of object.

・Displaying text to show the change of gaze.


2.Displaying support information on the display (screen)

There are three formats displaying the support information such as the CG containing the text and object.

① HMD-fixed coordinate system: format where the support information is displayed in a way that it is fixed to the HMD.

Example: “①HMD” text is displayed on the screen at the same location even when the user’s head position or posture changes.

② Object-fixed coordinate system: format where the support information is displayed in a way that it is fixed to the object.

Example: “②Object” text display the change in position and posture in congruence with the change in position and posture of the block.

③ Fixed to global coordinate system: format where the support information is displayed in a way that it is fixed to the global coordinate system

Example: “③World” text is displayed at the same location in real space, irrespective of the changes in the position and posture of the user’s head or that of the object

Issues that need to be considered (addressed) in using format ①

As shown in the figure below, there is a large difference in the depth distances of the part and support information as measured from the eye when using the HMD equipped with two displays for each eye. The user cannot focus both eyes on two or more depth positions at once due to the convergence angle cause by the movement of both eyes. Thus, it is necessary to adjust and compare the focus alternately; however, because the focus of the eyes cannot be adjusted instantaneously, most of the support information may look like two images.

To handle such a problem, previous research has reported that by changing the display for one eye, the focus adjustment of both eyes can be avoided, resulting in an easier viewing of the part and support information. 

Moreover, the missing field of view of one eye can be unchanged since it is supplemented by the field of view complementing the capability of both eyes.

Example: Figure showing the convergence angles of both eyes when an object is placed in front of the eye and the support information is placed in between the object and eye

Displaying images in format ② and ③

It is better to display the images in actual size, if there are several similar-looking parts around.

Normally, if the user searches among several similar-looking parts for the part shown in the image, the search will be based on a comparison with the appearance information shown in the image. In such a case, using the image showing the actual size of the part helps in the comparison based on the size and appearance information, making the search easier.

Example: Select the connector shown in the image (left figure: displays the image, right figure: displays the image after adjusting for the actual size)

3.Work using both hands

When both hands are required in performing work, it is better to use the voice-activated operation.

The following describes the minimum required commands in a voice-activated operation

・Proceed to the next step

・Go back to the previous step

・Go back to the first step

・Open the menu

Example: Voice command-based operation of the “Next”, “Back”, and “Reset” options

It is better if the available commands are always shown in the display when the voice-activated operation is used. (for details of the command operation, please refer to using command operation)


4. Using video or animation

It is better to arrange the dedicated operation commands when showing work procedure using videos or animations. (for details of the command operation, please refer to using command operation)

The following describes the minimum required commands for operation:

・Pause

・Playback

・Playback from start

Example: Voice-activated video operation for work involving the installation of a screw on the PC chassis

Moreover, the following introduces the measures about the displayed videos and animations.

  • It is better to provide a playback loop function to enable the worker to verify the video or animation as many times as necessary.
  • It is better to provide a trailer by going to the image of the final state and pausing for a certain period of time after the video in order for it to be easily conceptualized.
  • It is better to provide  zoomed-in  locations that the user needs to pay attention when a (information on) detailed work is presented using a video or animation.


5. Performing command operation

It is better to constantly display the command list, either at a fixed position in the HMD-fixed coordinate system or at the global coordinate system-based fixed position, when the command operation is performed using a voice or gesture.

Note that the display needs to be changed by altering the color of the text, to lighter or paler colors, for the commands that are not available. Alternatively, the display may not show such commands at all.

Moreover, even for commands that are available, one may always show only those that are used more often. The commands that are not used often can be listed in the hidden menu.

Measures such as changing the color of the applicable command in the listed commands are necessary to provide feedback to the user after the operation using the command is completed.


6. Subtask: handling placement of an object

For “object placement,” the following methods can be used when aiming for improving work efficiency using the 3D CG animation. 

・Clearly show the final placement position of the object using “wireframe”.

Example: Placing the yellow block (2 × 4) at the designated position on the foundation.

・If the placement method is simple, show only the 3D model at the final position of the object without using the animation

Example: Placing the green block (2 × 6) on the yellow block (2 × 4)

Moreover, for the “object placement,” when expressing object position and posture using text, it is sometimes easier to understand if the position and posture are expressed relative to a nearby object instead of expressing the absolute position and posture of the target object.

Example: When placing the blue block (2 × 2) five spaces (stud-spacing) from the left edge and nine spaces (stud-spacing) from the lower edge of the green foundation, it is easier to understand if the placement position is shown relative to the nearby blue block (2 × 4).