Technology Under Consideration for ISO/IEC 23090-32

ISO/IEC 23090-31 [1] defines a scalable haptics coding framework supporting up to 65536 channels per perception through a 16-bit channel identifier (channel_id).

ISO/IEC 23090-32 [2] specifies the carriage of haptics data in ISOBMFF. However, in its current form, it constrains the channel identifier to 8 bits in several structures. This contribution proposes an alignment of the channel identifier bit-width. == Problem Statement In ISO/IEC 23090-31, the haptic channel identifier is defined as a 16-bit unsigned integer (16 uimsbf) in both metadata and data syntax.

In ISO/IEC 23090-32, the channel identifier is limited to 8 bits in multiple structures, including: • HapticChannelDescriptionHeaderBox • HapticBandConfigurationBox • MIHSBandGroupBox

This discrepancy leads to the following issues:

As a result, interoperability between encoders, decoders, reference software, and delivery systems cannot be guaranteed.

ISO/IEC 23090-32 [2] specifies the carriage of this haptics data in the ISO Base Media File Format (ISOBMFF), including both single-track and multi-track encapsulation modes. In the multi-track mode, a MIHS track may reference one or more MIHS band tracks that carry the haptic data samples associated with the experience.

Clause 5.2 of ISO/IEC 23090-32 [2] currently specifies the following requirements for referenced MIHS band tracks:

The sample entries of each referenced MIHS band track shall satisfy the following conditions: a) It shall contain all the necessary metadata in its configuration box needed for decoding the samples of this band track without relying on the MIHS track. b) Its configuration box and haptic experience boxes shall not contradict the information in the same boxes of the MIHS track that is referencing this MIHS band track. c) Its configuration box and haptic experience boxes shall not contain any information about the samples in other referenced MIHS band tracks […​] However, these requirements introduce ambiguity and potential inconsistency with the overall architecture defined in the specification.

In particular: • The specification defines that, in multi-track mode, the MIHS track may reference MIHS band tracks that carry the samples of the experience, while the MIHS track provides global configuration and metadata. • The MIHSConfigurationBox and associated description boxes in the MIHS track reflect initialization units and provide decoder configuration for the haptic experience.

In this context, requirement (a) implies that each MIHS band track shall be independently decodable without relying on the MIHS track. This contradicts the intended separation of responsibilities between: the MIHS track, which provides global experience-level metadata, and the MIHS band tracks, which carry band-specific data and associated metadata.

As a result: • Requirement (a) may incorrectly suggest full independence of MIHS band tracks • Requirements (b) and (c) do not clearly define the expected scope of metadata in band tracks versus the MIHS track • The current wording may lead to inconsistent implementations regarding metadata duplication and dependency handling

The specification ISO/IEC 23090-32 [2] defines that: • A MIHS track may either contain all samples (single-track mode) or reference MIHS band tracks that contain the samples (multi-track mode) • Metadata describing the haptic experience, perceptions, channels, and bands originates from the MIHS stream and is reflected in configuration and description boxes

In multi-track mode: • The MIHS track acts as the entry point and global description of the experience • MIHS band tracks provide band-level data and associated metadata • Track referencing and grouping mechanisms are used to associate band tracks with channels and perceptions

Therefore, requiring that each MIHS band track be decodable “without relying on the MIHS track” is not aligned with: • the referencing model • the distribution of metadata across tracks • the intended decoding workflow combining MIHS track and band tracks

It is proposed to revise the requirements on referenced MIHS band tracks to clarify the dependency model and avoid ambiguity.The sub-clause for which the changes are proposed is 5.2 MIHS streams and tracks ISO/IEC 23090-32 [2]

The following conditions apply to ensure consistency between the MIHS track and the referenced MIHS band tracks.

The current wording of the requirements for referenced MIHS band tracks introduces ambiguity regarding metadata ownership and decoding dependencies. The proposed changes clarify that MIHS band tracks contain band-specific metadata while relying on the MIHS track for global information, ensuring consistency with the overall design of ISO/IEC 23090-32

Haptics as a media type is an emerging modality that enhances the realism of the overall user experience. It can play a subtle but important role in shaping the user’s experience. It acts as a third sensory channel alongside sight and sound. Haptics introduces tactile signals making the entire user experience engaging and immersive. A haptic movie trailer [1] was released on Apple TV app on iPhone that played synchronized vibrations that matched the scene in the movie trailer.

Another key role is in accessibility. For users with visual or auditory impairments, haptic media data can act as primary signals guiding navigation, confirming actions, or alerting users to changes. For such users, haptic media becomes an alternative way of experiencing the content.

The key to achieving a rich experience with haptic media along with audio-visual is the synchronization with audio-visual media. With improper synchronization, the experience can feel poor and uncomfortable to enjoy the content as it is with audio-visual synchronization.

One common way Haptic format is described is using human-readable textual form. In MPEG, HJIF format [2] has been described which is developing using JSON providing haptic information. HJIF describes properties which when provided to a HJIF decoder synthesizes a haptic signal accordingly.

Currently, ISO/IEC 23090-32 [4] defines the encapsulation of binary MIHS unit data in ISOBMFF. There is a gap in the specification on how haptic content described in textual format can be encapsulation in ISOBMFF container.

The Haptics JSON Interchange Format (HJIF), defined in ISO/IEC 23090-31 [2], provides a structured and human-readable way to represent haptic media. It organizes haptic content using a hierarchical model. At the top level, an HJIF document defines an experience, which represents a complete haptic scene and captures the overall structure of the content. Within an experience, haptic data is grouped into one or more perceptions, each corresponding to a distinct type or modality of haptic sensation.

Each perception is further divided into channels, which represent logical streams of haptic information. A channel may map to a specific target, such as a particular body location or a group of actuators. Channels are then subdivided into bands, which capture different perceptual components of the haptic signal.

Within each band, the content is described using effects, which are the fundamental time-based units of haptic media. Effects are defined by temporal properties such as start time and duration, along with perceptual parameters like intensity, frequency, magnitude, or other modality-specific attributes. They may also include parametric descriptions, such as envelopes or modulation curves, enabling the haptic signal to evolve dynamically over time.

The timing information for a haptic signal is described with band and effects in HJIF. The ‘position’ property indicates the time offset of the effect in a band. Within each effect, keyframes are included. A keyframe has a property ‘relative_position’ which indicates the relative time of the keyframe with respect to the parent effect.