Unreal SDK

Welcome to the Weart Unreal SDK documentation.

Support Unreal 4.x+ and 5.x+

The SDK allows to connect to the Weart middleware and perform various actions with the TouchDIVER devices:

• Start and Stop the middleware operations
• Calibrate the device
• Receive tracking data from the devices
• Send haptic effects to the devices

Architecture

Unreal SDK Architecture

Set-up project and import plugin

The minimum setup to use the weart SDK consists of:

• A PC with the Middleware installed
• A TouchDIVER device
• An Unreal project using the SDK plugin

Create a new project by selecting Virtual Reality.

Copy the whole plugin folder either in $UE_LOCATION$/Engine/Plugins for global use or in $PROJECT_FOLDER$/Plugins to use in a specific project. Enable it inside the UE editor in Edit/Plugins.

Enable the plugin if it was placed globally

In the World Settings change the Default Pawn Class to WeArtPlayerBP

If the VR Preview is inactive make sure to install the plugin that corresponds to your headset.

Guide Components

Components

WeArtController

It is actually created automatically as soon as you enable the plugin. But if you want access to data members etc, just create a Blueprint Class (Add->Blueprints->Blueprint Class) which inherits from WeArtController.

WeArtTouchableObject

Component responsible for the description of the haptic effect to be applied in the event of a collision with the HapticObject actor.

Properties:

• Temperature – Temperature value implemented on the target thimble or thimbles (from 0.0 to 1.0) [0.5 is environment temp – 0.0 really cold – 1.0 really hot]
• Force – Force value applied on the target thimble (s) (from 0 to 1) [0.0 no force – 1.0 max force]
• Texture – Type of texture rendered on the thimble or target thimbles (from 0 to N)
• Volume Texture: Configure the intensity of texture rendering (from 0 to 100)
• Graspable: Enable the ability to grasp the object with virtual hands

The WeArtTouchableObject component has a field called ForcedVelocity. if it is enabled, when the hand enters the touchable object, the texture feeling will run at maximum speed. It will not take in consideration the movement of the hand.

GameObject Requirements:

• Collider (Inherited)
  • Collision 🡪 Generate Overlap Events = True

WeArtHapticObject

Component responsible for the haptic actuation of the individual digital devices belonging to the TouchDiver device.

Properties:

• Hand Side Flag – Which device it belongs to (RIGHT | LEFT)
• Actuation Point Flag – Target of the thimble or the thimbles on which to apply implementation (THUMB | MIDDLE | INDEX) [Multi selection]
• Control: - (Set by TouchableObject during interaction or manually by developer)
  • Temperature – Temperature value implemented on the target thimble or thimbles (from 0.0 to 1.0) [0.5 is environment temp – 0.0 really cold – 1.0 really hot]
  • Force – Force value applied on the target thimble (s) (from 0 to 1) [0.0 no force – 1.0 max force]
  • Texture – Type of texture rendered on the thimble or target thimbles (from 0 to N)

GameObject Requirements:

• Collision
  • Generate Overlap Events = True

How to create Haptic/Touchable objects:

By adding actor components to existing actors. Either insert any actor in the map, and equip it with a Haptic/Touchable component and a collision, or create a blueprint actor class which you can then reuse anytime you want. In the details panel, you should have access to all the modifiable variables.

Game needs to be running and Haptic/Touchable objects need to overlap for anything to be visible. All the actors containingn Haptic/Touchable components need to have "Generate Overlap Events" tickbox enabled. Currently, only overlap callbacks are supported. UE has a Hit callback as well, but not anything similar to Hit End.

WeArtThimbleTrackingObject

Component responsible for tracking of thimble's movements for quantifying its closed state and animating virtual hands. Properties:

• Hand Side Flag – Which device it belongs to (RIGHT | LEFT)
• Actuation Point Flag – Target of the thimble or the thimbles on which to apply implementation (THUMB | MIDDLE | INDEX) [Multi selection]

Hands Bluprints

These are the two blueprints for the hands:

Hand Grasp Events

In order to enable the events to be called, drag this blueprint into the scene.

The blueprint contains the following events inside it. Each event returns the hand side that grabbed the object and the object that was grabbed as a WeArtTouchableObject.

Flag Show Hand Rendered During Grasp

The two hands blueprints contain the component called WeArtHandContoller. This component has a flag called IsHandGrabInvisible. If the flag is active, during the grasping, the hand rendering will stop.

Hand Offset

The two hands blueprints contain the component called WeArtDeviceTrackingObject.

This component has a flag called UseOffsetPreset. If it is enabled, the hand will use the offset present in the field OffsetPreset.

These are the presets available:

WeArtDeviceTracking

Component responsible for the tracking of the wrist on which the TouchDiver device is placed Properties:

• Update method – Mode in which the position of the object is updated on the basis of the source tracking
• Tracking Source – Transform reference of motion controller source (ex. VR Controller or Vive Trackers, etc. etc.)
• Position Offset – Trasform position offset respect from the tracking source
• Rotation Offset – Trasform rotation offset respect from the tracking source
• Offset presets – You can choose a preset offset parameters of specific platform controller like Oculus or Vive or in any case a custom configuration

API

Start/Stop Client

Once connected to the middleware, it's still not possible to receive tracking data and send haptic commands to the devices. In order to do so, it's important to start the middleware with the proper command.

From WeArtController GameInstance get Subsystem

UGameInstance* gameInstance = GetOuter()->GetWorld()->GetGameInstance();
UWeArtController* weArtController = gameInstance->GetSubsystem<UWeArtController>();

weArtController->PauseController();     //Start Middleware communication
weArtController->UnpauseController();   //Stop Middleware communication

Start/Stop Calibration and events

From WeArtController GameInstance get Subsystem

UGameInstance* gameInstance = GetOuter()->GetWorld()->GetGameInstance();
UWeArtController* weArtController = gameInstance->GetSubsystem<UWeArtController>();

weArtController->StartCalibration();    //Start Middleware calibration
weArtController->StopCalibration();     //Stop Middleware calibration

Blueprint:

In order to enable the events to be called, drag this blueprint into the scene.

Inside this blueprint there are four events that are fired when the calibration starts, finishes, it succeeds or fails. They all return the side of the hand that called the event.

There is another event that fires when the calibration stops and it does not contain a hand side.

WeArtTouchEffect

The SDK contains a basic WeArtTouchEffect class to apply effects to the haptic device. The TouchEffect class contains the effects without any processing. For different use cases (e.g. values not directly set, but computed from other parameters), create a different effect class by implementing the WeArtEffect interface.

Create Custom effect

Instantiate new effect:

WeArtTouchEffect* effect new WeArtTouchEffect(); 

Update effect

Create and activate actuations

FWeArtForce force;
force.active = true;    //active actuation
force.value = valueForce;   //set value

Set actuation to effect:

effect->Set(temperature, force, texture);

Add effect

Apply to your HapticObject (finger/thimble) effect:

hapticObject->AddEffect(effect);

Remove effect

To remove effect and restore actuation, get the same instance of effect and call “Remove” for the same HapticObject:

hapticObject->RemoveEffect(effect)

Tracking

After starting the middleware and performing the device calibration, it's possible to receive tracking data related to the TouchDIVER thimbles.

The player pawn contains two UWeArtHandController. One for each hand. Every UWeArtHandController contains three UWeArtThimbleTrackingObject. One for index, middle and thumb. From each UWeArtThimbleTrackingObject we can get the closure and abduction values. The values are from 0 to 1, 0 representing no closure or abduction and the 1 representing the maximum value.

Getting Closure and Abduction:

UWeArtThimbleTrackingObject* thumbTrackingObject;
thumbTrackingObject->GetClosure();
thumbTrackingObject->GetAbduction();

Here is a blueprint representation of getting the values, in this image the values are used for the animation of the hands.

Adding new custom texture type

WeArtCommon.h contains an enum called TextureType. We can add a new member here and assign its number. Then inside the code and the editor we can set this member to WeArtTouchableObjects and inside Texture classes.

After compiling the code, the member will be available in the editor:

Plugin configuration

Go to Project Settings, then under the Plugins category you will find the WeArt settings. Here you can edit if you want to start the middleware and the calibration at the beginning of the experience.

Example

Load Unreal project with plugin WEART SDK already integrated ready for testing "VRTemplateMap", VR character with virtual Hands and two different TouchableObject for testing the haptics feedback with the TouchDIVER and HMD

Link to Unreal sample project

• Unreal Engine 4 Sample Project
  
• Unreal Engine 5 Sample Project

Migration and Update from previous SDK

To update your application to the latest SDK, download and extract the Unreal SDK archive, then copy the source/header files in the same place as the older SDK version.

The new version includes additional files, so it's necessary to add them to the project in order to avoid linking errors.

This can be done on Visual Studio by right-clicking on the solution, then clicking on Add -> Existing Item and selecting all the SDK files. On other systems (e.g. cmake) the procedure might be different.

The SDK is retro-compatible with older versions, so there's no need to update the application code. To see the new feature and fixes added to each version, refer to the Changelog section.

Changelog

• Expose public property clousure thimble
• Add calibration procedure start/stop and listener
• Add hand grasping events
• Fix Blueprint behaviours Hand