Unity SDK

Welcome to the Weart Unity SDK documentation.

Support Unity 2019.4+, 2020.x+, 2021.x+ and 2022.x+

The SDK allows to connect to the Weart middleware and perform various actions with the TouchDIVER devices:

• Start and Stop the middleware operations
• Calibrate the device
• Receive tracking data from the devices
• Send haptic effects to the devices

Architecture

Unity SDK Architecture

Set-up project and import plugin

The minimum setup to use the weart SDK consists of:

• A PC with the Middleware installed
• A TouchDIVER device
• An Unity project using the SDK package

Create a new project or open an existing one.

Go to "Window" and then to "Package Manager".

Then press on the "+" and select "Add package from disk".

Find the location of the SDK, select package.json and press "Open".

From here it is possible to import samples to the project.

Guide Components

Components

WeArtController

Responsible for communication with the TouchDiver device through dialogue with Middleware Path: Packages🡪 WEART SDK 🡪 Runtime 🡪 Prefabs 🡪 WeArtController.

Inspector Properties:

• Client Port:
  • Socket TCP Port for dialogue with Middleware (don’t change)
• Start Automatically:
  • When this flag is active, the communication between Unity client and Middleware start automatically during Editor play mode and when Unity application execute is launched
• Start Calibration Automatically:
  • When this flag is active, the middleware calibration begins when the scene starts playing
• Debug Messages:
  • Enable or Disable WEART SDK Logs in console
• Start middleware:
  • Manually start or stop the touchdiver connection inside the middleware
• Start Calibration:
  • Manually start the calibration inside the middleware

WeArtTrackingCalibration contains Unity events that correspond to the middleware calibration events

WeArtTouchableObject

Component responsible for the description of the haptic effect to be applied in the event of a collision with the HapticObject actor Properties:

• Temperature – Temperature value implemented on the target thimble or thimbles (from 0.0 to 1.0) [0.5 is environment temp – 0.0 really cold – 1.0 really hot]
• Force – Force value applied on the target thimble (s) (from 0 to 1) [0.0 no force – 1.0 max force]
• Texture – Type of texture rendered on the thimble or target thimbles (from 0 to N)
• Volume Texture: Configure the intensity of texture rendering (from 0 to 100)
• Graspable: Enable the ability to grasp the object with virtual hands

GameObject Requirements:

• Rigibody
• Collider

The component WeArtTouchableObject has a flag called Forced velocity, if it is enabled, when the hand enters the touchable object, the texture feeling will run at maximum speed. It will not take in consideration the movement of the hand.

WeArtHapticObject

Component responsible for the haptic actuation of the individual digital devices belonging to the TouchDiver device Properties:

• Hand Side Flag – Which device it belongs to (RIGHT | LEFT)
• Actuation Point Flag – Target of the thimble or the thimbles on which to apply implementation (THUMB | MIDDLE | INDEX) [Multi selection]
• Control: - (Set by TouchableObject during interaction or manually by developer)
  • Temperature – Temperature value implemented on the target thimble or thimbles (from 0.0 to 1.0) [0.5 is environment temp – 0.0 really cold – 1.0 really hot]
  • Force – Force value applied on the target thimble (s) (from 0 to 1) [0.0 no force – 1.0 max force]
  • Texture – Type of texture rendered on the thimble or target thimbles (from 0 to N)

GameObject Requirements:

• Collider
  • Is Trigger = True

How to create Haptic/Touchable objects:

By adding components to existing game objects. Either insert any prefab in the map, and equip it with a Haptic/Touchable component and a collision, or create a prefab which you can then reuse anytime you want. In the inspector panel, you should have access to all the modifiable variables.

WeArtThimbleTrackingObject

Component responsible for tracking of thimble' movements for quantifying its closed state and animating virtual hands Properties:

• Hand Side Flag – Which device it belongs to (RIGHT | LEFT)
• Actuation Point Flag – Target of the thimble or the thimbles on which to apply implementation (THUMB | MIDDLE | INDEX) [Multi selection]

Hands Prefabs

Represents virtual hands in your experience linked to TouchDiver and able to update finger tracking for activation hand animation and haptic for thimbles. In base of the project configuration, the hands can move with your motion controller. Path: Packages🡪WEART SDK 🡪 Runtime 🡪 Prefabs 🡪 WEARTLeftHand | WEARTRightHand

Hand Grasp Events

Each WeArtHandController has two events. One fires when an object is grasped and the other when it is released.

Create two functions that will be subscribed to the events

void OnRightGrasp(HandSide handSide, GameObject gameObject)
  {
 
  }
 
void OnRightRelease(HandSide handSide, GameObject gameObject)
  {
 
  }

And then subscribe them to the hand controller;

rightHandController.OnGraspingEvent += OnRightGrasp;
rightHandController.OnReleaseEvent += OnRightRelease;

Flag Show Hand Rendered During Grasp

The hand prefabs contain a script called WeArtHandController.

This component has a field called ShowHandOnGrasp. If this is enabled, the hand will be visible during the grasping of objects. If it is not, the hand will be invisible during the grasping.

Hand Offset

These hand prefabs contain a script called WeArtDeviceTrackingObject.

This component has offset presets that can be changed based on the touch diver setup.

WeArtDeviceTracking

Component responsible for the tracking of the wrist on which the TouchDiver device is placed Properties:

• Tracking Source – Transform reference of motion controller source (ex. VR Controller or Vive Trackers, etc. etc.)
• Update method – Mode in which the position of the object is updated on the basis of the source tracking
• Position Offset – Trasform position offset respect from the tracking source
• Rotation Offset – Trasform rotation offset respect from the tracking source
• Offset presets – You can choose a preset offset parameters of specific platform controller like Oculus or Vive or in any case a custom configuration

Here are the offset presets available:

API

Start/Stop Client

Manage Start and Stop Middleware communication Import in your C# class this namespace:

using WeArt.Components;

From WeArtController GameObject get Instance

WeArtController.Instance.Client.Start(); //Start Middleware communication
WeArtController.Instance.Client.Stop(); //Stop Middleware communication

Start/Stop Calibration and events

From WeArtController GameObject get Instance

WeArtController.Instance.StartCalibration();
WeArtController.Instance.StopCalibration();

Tracking

After starting the middleware and performing the device calibration, it's possible to receive tracking data related to the TouchDIVER thimbles.

There are two hand prefabs in the scene. Every hand prefab has a WeArtHandController. Every WeArtHandController has three WeArtThimbleTrackingObject references. We can get the closure and abduction from these components. The values are from 0 to 1, 0 representing no closure or abduction and the 1 representing the maximum value.

Getting Closure and Abduction:

WeArtThimbleTrackingObject _thumbThimbleTracking;
float closure = _thumbThimbleTracking.Closure.Value;
float abduction = _thumbThimbleTracking.Abduction.Value;

WeArtTouchEffect

The SDK contains a basic WeArtTouchEffect class to apply effects to the haptic device. The TouchEffect class contains the effects without any processing. For different use cases (e.g. values not directly set, but computed from other parameters), create a different effect class by implementing the WeArtEffect interface.

Create Custom effect

Instantiate new effect:

WeArtTouchEffect effect = new WeArtTouchEffect();

Update effect

Create and activate actuations

Temperature temperature = Temperature.Default;
temperature.Active = true; //you have to active 
temperature.Value = 0.6f; //hot

Set actuation to effect:

effect.Set(Temperature, Force, Texture);

Add effect

Apply to your HapticObject (finger/thimble) effect:

hapticObject.AddEffect(effect);

Remove effect

To remove effect and restore actuation, get the same instance of effect and call “Remove” for the same HapticObject:

hapticObject.RemoveEffect(effect);

Adding new custom texture type

In WeArtCommon.cs we can find enum called TextureType We can add a new texture like this:

public enum TextureType : int
    {
        ClickNormal = 0, ClickSoft = 1, DoubleClick = 2,
        AluminiumFineMeshSlow = 3, AluminiumFineMeshFast = 4,
        PlasticMeshSlow = 5, ProfiledAluminiumMeshMedium = 6, ProfiledAluminiumMeshFast = 7,
        RhombAluminiumMeshMedium = 8, 
        TextileMeshMedium = 9,
        CrushedRock = 10,
        VenetianGranite = 11,
        SilverOak = 12,
        LaminatedWood = 13,
        ProfiledRubberSlow = 14,
        VelcroHooks = 15,
        VelcroLoops = 16,
        PlasticFoil = 17,
        Leather = 18,
        Cotton = 19,
        Aluminium = 20,
        DoubleSidedTape = 21,
        NewExampleTexture= 50
    }

And we can access it by index:

Texture texture = Texture.Default;
texture.TextureType = (WeArt.Core.TextureType)50;
texture.Active = true;

Velocity

We can change the velocity of the texture of an effect by changing its value [0.0 to 0.1] 0.5 is recommended for the normal speed of the texture. This will give the effect a constant speed for the texture without moving the hand

Texture texture = Texture.Default;
texture.TextureType = (WeArt.Core.TextureType)11;
texture.Active = true;
texture.Velocity = 0.5f;
 
touchEffect.Set(temperature, force, texture, new WeArtTouchEffect.WeArtImpactInfo());

Tools

Add Weart Startup Components

This tool will add the controller and the hands prefabs

Current object to TouchableObject

This tool will transform the selected game object into a touchable object and it will apply the necessary components.

Finder

This tool will open a window that will show the touchable objects present in the scene

Debugger

This tool will open a window that will show the haptic objects in the scene and its parameters

Example

You could import a sample scene to understand SDK components and simplify your test using ready scenes with any components you need.

Simple Temperature

This sample contains a very basic scene with one Touchable Cube and one finger Haptic Object for testing actuation

Haptics Hands

This sample contains a ready-to-use prefab for a virtual hand having haptic controls

Sample Demo

This sample contains a ready-to-use scene which shows the capabilities of the sdk. It has three scenes:

PlaygroundDemo

This scene contains four material plates that have different textures, four objects on the table that can be grabbed and have different temperatures, an objects that responds to a gesture system, a group of slider objects that we can interact with and two objects that create a continuous temperature and texture effect

TemperatureDemo

ClosureDemo

The scene contains a visual representation of the closure values of the ThimbleTrackingObjects

Migration and Update from previous SDK

To update your application to the latest SDK, download and extract the Unity SDK archive, then copy the files in the same place as the older SDK version.

The new version includes additional files, so it's necessary to add them to the project in order to avoid linking errors.

This can be done on Visual Studio by right-clicking on the solution, then clicking on Add -> Existing Item and selecting all the SDK files. On other systems (e.g. cmake) the procedure might be different.

The SDK is retro-compatible with older versions, so there's no need to update the application code. To see the new feature and fixes added to each version, refer to the Changelog section.

After the update of the SDK the WeArtController and WEARTRightHand will be broken in the scene. Replace them with the new prefabs from the sdk.

Changelog

• Added sample scenes
• Editing at run-time textures
• Add calibration procedure start/stop and listener
• Add hand grasping events
• Add new default tracking message and values for closure and abduction