vrtsplash

User Guide

www.eyecreateworlds.com

 

Table of Contents:

 

1.      Installation and Setup

1.1.   General Information

1.2.   Known Issues

1.3.   System Requirements

1.4.   Installation

1.5.   Leap Motion Setup and Best Practices

2.      Interface/Control explanation and diagrams

2.1.   Leap Motion Control

2.2.   Gamepad

2.3.   Keyboard

2.4.   Oculus Remote

2.5.   Oculus Touch

2.6.    Leap+Touch

3.      Scene and Mode details

3.1.   Direct Video Mode

3.2.   AR Void Mode

3.3.   Mnemonic Mode

3.4.   Mnemonic Browser

3.5.   AR Mnemonic Mode

3.6.   Leap Passthrough Mode

4.      360 Video Playback

5.      Resources and Links

 

 

 

 

Installation and Setup:

 

General Information:

VRTheatre is a Mixed Reality media player application powered by VLC Icon.tiny VLC. It allows the user to watch videos in a variety of simulated VR environments. VRT currently supports the Oculus DK2 and Leap Motion Controller as well as Gamepad or Keyboard and mouse for interface/control.

 

Features:

-Playback engine powered by VLC. Supports most formats supported by VLC.

-Direct video device display support (allows things like webcams, capture cards, 360 Cameras etc. to be displayed)

-360 Video Playback

-Multiple immersive video watching locations (asteroid, island, void, etc)

-AR modes (Leap Motion Required)

-Resizable screens

-Dynamically generatable/movable/resizable display mode (Mnemonic mode, Leap Required)

-Multiple supported control types: Leap Motion (Recommended), Oculus Touch, Gamepad, Oculus Remote, Keyboard + Mouse.

-Web browsing with HTML5 support (Mnemonic Mode only atm and very basic functionality, will improve over time)

-Supports both Xbox 360 (xinput) and Xbox One Gamepads

-Supports Oculus Remote for file navigation/selection

-Supports Leap motion for motion control

-Supports basic Oculus Touch control (full support coming soon)
-Dynamically generatable/movable/resizable display mode (Mnemonic mode)
-Multiple supported control types: Leap Motion (Recommended), Gamepad, Oculus Touch (Basic), Oculus Remote, Keyboard and Mouse or any combo of the four

 

Known Issues:

-Oculus Touch interaction with GUI buttons on tablet or other control panels is disabled as it is not working reliably atm. Full Support is coming.

-Office Scene has a lot of real time lighting and geometry which makes it a more CPU/GPU intensive scene then the others, optimization needs to be done.

-Navigation panel for file selection with Leap sucks. A better solution is coming. We recommend using one of the other supported hardware devices when trying to scroll/select files to play.

-The Mnemonic mode requires Leap Motion and is heavily a beta/WIP. Behavior of the displays in the Mnemonic modes are sometimes unpredictable. The displays become stuck together if you are in actively pinching one display and collide with another. If displays are stuck together, try moving one without pinching it.

-MNemonic Web browser is WIP, incredibly basic, and has lots of issues. It will improve over time.

-Text entry for web browser relies on your physical keyboard. Better solution is in the works

-Video Playback sound will play from default Windows audio source only. Make sure to choose Rift Headphones as your Default Audio Source if you want the video sound to play from there.

 

System Requirements:

-Windows 7/8/8.1/10 (64bit)

-Oculus Rift (DK2 or CV1)

-Oculus Platform 1.11 or higher

Optional: Gamepad, Oculus Remote, Oculus Touch, or Leap Motion (Note: Several modes require Leap Motion for operation, see the Scene and Mode details section for more information.)

 

Leap Motion Support Requirements:

-Leap Motion Orion (v3.0) or higher

-Leap Motion in head mounted position

 

 

Installation:

Extract zip file to any location on your system and run VRTheatre.exe.

 

If you wish to change the quality settings press and hold your right Control key and launch the executable. You will then be presented with the following configuration dialog:

Select your desired Quality setting, higher quality provides a better graphic fidelity but requires more system resources. The Screen resolution setting determines the resolution of the output to the PC�s main display of the Prewarped image (for spectators). It is usually best to set this as high as your main display will handle as it does not seem to affect the VR performance. Select Monitor option, �Windowed� option, and the Input settings tab should all be left at default. After you have set your desired settings, press Play and the application will launch.

 

 

 

Leap Motion Setup and Best Practices:

 

 

Hold your hands at eye level about 1 foot away fingers slightly spread.

http://vvvv.org/sites/default/files/imagecache/large/images/Person%20oculus%20leap.jpghttps://s-media-cache-ak0.pinimg.com/236x/45/43/77/45437716a2cdb3a247c0c5ca12f659e9.jpg

 

 

Leap Motion detects your hand best when holding your hand spread, fingers spread apart. Press buttons by holding your hand in this pose and using your index finger slightly bent inwards to press buttons:

 

 

 

If you have trouble getting leap motion to detect your hand try turning your hands towards your face palms facing you, with your fingers spread.

 

 

 

The video below provides excellent information on the proper setup and use of Leap motion for use in VR:

How to Optimize Your VR Tracking with Leap Motion

https://www.youtube.com/watch?v=lJ4twi6jaSk

 

 

 

Interface/Control:

Below is information about the various ways in which you can control VRTheatre. Keep in mind you can use a combination of all three if you desire.

 

Leap Motion Control:

For Leap Motion Control, look down at the back of your left hand. You will notice a black button. This is the Leap Motion control panel activator. ​To open the Leap Motion control panel, place your right index finger on the back of your left hand on the activator button and hold for 1-2 secs. **Note: It is best to hold your right index finger out first before getting close to your left hand, otherwise Leap Motion may lose tracking. This will spawn the control panel tablet in your view. Press and hold the activator button again to close the panel. You will notice the button will remain green while the tablet is activated.You can also close the panel by pressing and holding the circular button on the bottom center of the panel.

 

 

Once the tablet has been spawned you will be presented with a variety of buttons. Below are explanations of each button�s function:

 

 

Leap Motion Control interface:

Play/Pause video

 

 

Stop video

 

 

 

Open file browser

 

 

 

Skips video ahead/back 10 seconds

Dynamic Display mode (Mnemonic Mode) allows the dynamic generation and movement of displays. This mode allows the user to create multiple displays and position them where ever they like.

 

 

Displays the scene selection menu allowing the user to change their current environment.

 

 

 

360 Video Mode Allows for playback of various spherical/hemispherical 360 video formats.

 

 

Change to Direct Video mode (Webcam, Capture Card support)

 

Mute/Unmute audio

 

Volume Up/Down

 

 

 

Gamepad Control:

Note: At any time, you can press the Dpad, or B button on the gamepad to display the on screen file selection menu.

 

 

 

Keyboard Control:

Note: At any time, you can press the arrow keys, enter key or backspace key to display on the on screen file selection menu.

 

 

 

Oculus Remote Control:

Note: At any time, you can press the navigation buttons, select or back key to display on the on screen file selection menu.

 

 

 

Oculus Touch:

 

**User movement is performed with the Analog sticks on the Oculus Touch Controllers

 

 

If using the Oculus Touch without Leap Motion, the users hand representations will mimic that of human hands. Please note that most GUI elements are currently not working with Oculus Touch only control.

GUI elements for motion control are currently coded for use with Leap Motion. Full Oculus Touch support is coming.

 

 

 

 

 

 

Oculus Touch Menu selection:

To select items in the on-screen Menu, Hold both triggers with your index fingers on each Touch controllers. The thumb stick on the right hand will then allow you select items on the menu. Press A or B on the Right Touch controller to select items/go back respectively

 

**Note: Oculus Touch does not currently work with most UI elements. Oculus touch support is very limited at present and is used mainly as a supplement to the Leap Motion.

 

 

 

Leap+Touch:

When using Leap Motion with Oculus Touch, the Touch controllers will appear as they do in real life and the Leap Motion will take over for 5 finger hand tracking for displaying the users hands combined with the Oculus Touch controllers tracked by the Constellation system. If the Leap Motion is disconnected, and only the Oculus Touch controller is used the representations will change to mimic human hands instead of controllers.

 

 

 

 

If using the Oculus Touch without Leap Motion, the users hand representations will mimic that of human hands. Please note that most GUI elements are currently not working with Oculus Touch only control.

GUI elements for motion control are currently coded for use with Leap Motion. Full Oculus Touch support is coming.

 

 

 

 

Menu selection can be done by holding down both triggers with your index fingers and using Up/Down on the Right Touch controller to navigate the Menu. A and B buttons on the Right Touch controller make menu selections or go back respectively.

 

**Please note that the tracking systems are not 1:1 and the hands will not always line up perfectly with the placement of the Oculus Touch controllers.

 

 

 

 

 

Scene and Mode details:

In addition to the environmental scenes, VRTheatre features a variety of additional viewing modes. Details about each mode are as follows:

 

 

Direct Video Mode:

This mode allows the user to display Direct video devices such as web cams or capture cards on the virtual screens. This allows Game consoles, cable/sat boxes etc. to be connected and displayed on your virtual screens.

**Leap Motion Controller will appear in your direct video devices list. Selecting this will result in loss of hand tracking. Do not select leap Motion Hand controller. If you do, disconnect and reconnect your leap motion and you should regain tracking.

***Some Capture cards may exhibit instability when changing scenes.

 

 

 

Void Mode:

Void Mode features a large screen in a black expanse instead of a detailed environment. You can switch to an AR version of this mode by clicking the AR button in the top left. Most of the same controls are available as with the other scenes. Click here for button explanations.(Note: Leap Motion is required to operate UI elements however menu selection can still be performed without it)

 

 

AR Void Mode:

AR Void mode features a large screen in an augmented reality view of your real world environment provided by Leap Motion�s passthrough capability. (Note: Leap Motion is required for this mode.)

 

 

Mnemonic Mode:

This mode allows the user to dynamically generate and then move video displays. In addition to VLC playback, this mode also supports Direct Video devices such as webcams, video capture cards etc.

Click here for Video footage of the Mnemonic Mode

 

 

Generating and manipulating displays:

To generate a new display, press the Create display button. A display will be created in front of you. You will then be able to move the display by pinching to grab. It can be grabbed anywhere but it is recommended to pinch and grab in the center of the display for the best control. The display will follow your hand position while you continue to hold the pinch pose.

http://www.virtualrealityreviewer.com/wp-content/uploads/2014/08/leapmotion-hand-tracking.jpg

 

You can also resize the display by pinching and grabbing and pinching each side of the display and pulling it apart as if you were stretching it.

 

Once you have generated and placed the display(s) in the desired position, press the Open Configuration panel button. The Configuration buttons will then be displayed for each display. Press the Open Configuration panel button again to hide the panel buttons and unlock the display positions.

 

You can then select a video for playback by pressing the File section button or select a Direct Video device by pressing the Source button.

 

 

Below are explanations of the various control and configuration panel icons:

Mnemonic mode control panel icons:

 

Creates a new blank display

 

 

 

 

 

Changes the environment from nighttime to daytime.

 

 

 

Clear all generated displays and reset scene

 

 

 

�� Change to/from AR Mnemonic mode

 

 

 

 

Exit Mnemonic mode

 

 

 

 

Below are explanations for the display configuration panel icons:

Mnemonic mode display configuration panel icons:

 

Direct Video Source cycles through Direct Video devices

 

 

 

 

 

Opens the file selection menu

 

 

 

 

 

Play/Pause video

 

 

 

 

 

�� Displays the web browser interface

 

 

 

Mute/Unmute audio

 

 

 

 

 

Changes the transparency level of the display

 

 

 

 

 

Destroy display

 

 

 

 

 

 

Mnemonic Web Browser:

**Please note the web browser is a work in progress and will improve over time

 

The Mnemonic browser supports most websites including HTML5. Video sites such as Youtube will work, however there are problems with full screening the video.

To navigate, simply look at the web browser display, you will notice a blue cross following your gaze. To click a link, press thehttp://t10.deviantart.net/M1FW6PTzeypRfq7-ztiqVaUblp8=/fit-in/150x150/filters:no_upscale():origin()/pre08/1710/th/pre/f/2014/239/4/b/xbox_360_button__a_by_thedevingreat-d7wv4i5.pngbutton your gamepad, or the Left Control button on the keyboard.

For text entry, simply �click� in the text box by using the gaze+gamepad/keyboard and then type with your actual physical keyboard. We understand this is not ideal and we are working on a better solution.

**Note while gazing at the browser window some keyboard commands may be unavailable because they could be confused for text entry.

 

 

Mnemonic Browser specific icons:

 

Browser reload current page

 

 

 

Browser navigation back to home page

 

 

 

 

Browser navigation back

 

 

 

Browser navigation forward

 

 

 

Stop loading current page

 

 

 

Changes the transparency level of the display

 

 

 

 

 

 

Destroy display

 

 

 

 

 

 

�Click� button for motion control interface

 

 

 

 

 

 

AR Mnemonic Mode:

As with the Mnemonic Mode, AR Mnemonic Mode allows the user to dynamically generate and then move video displays. Instead of a rendered environment, the user is presented with an augmented reality view of your real world environment provided by Leap Motion�s passthrough capability. See Mnemonic Mode section above for button descriptions. (Note: Leap Motion is required for this mode.) Click here for Video footage of the Mnemonic AR Mode

 

AR Mnemonic mode control panel icons:

 

Create a new virtual display.

 

 

 

 

Resets the scene and removes all virtual displays.

 

Show/hide configuration panel for each virtual display.

 

 

 

 

Exit AR Mnemonic mode and return to previous scene.

 

 

 

 

 

Leap Passthrough:

This mode provides a view of your real provided by Leap Motion�s passthrough capability. It can be accessed at any time by pressing P on the keyboard or the left thumb stick on the gamepad. Press again or click the X icon on the top right to return to previous mode. (Note: Leap Motion is required for this mode.)

 

 

 

 

360 Video Playback:

VRTheatre supports both hemispherical and spherical 360 video from live streams or local video sources from most formats supported by VLC.

**Note: The control tablet can be deployed/used in the 360 Modes the same way it is used in regular scenes.

 

 

 

To enter 360 Video Playback mode, select �360 Video Mode� from the on-screen menu or press the 360 Video button on the motion control tablet.

 

 

 

You will be given options for the type of 360 viewing screen format you wish to use. These control the placement and orientation of the 360 viewing screen.

 

 

 

Resources and Links:

Below are links to additional information and resources.

 

http://www.unixstickers.com/image/cache/data/stickers/vlc/vlc_bumper.sh-600x600.pngVideoLan.org � Home of VLC.

 

Eye Create Worlds

 

http://www.getacoder.com/blog/wp-content/uploads/2014/03/leap_motion.jpgLeap Motion

 

Oculus