MPEG is currently exploring new opportunities for standards serving the needs of the media industry. This is a list of ongoing explorations:
Explorations
7 - Immersive Video: Processing and coding of immersive video
Motivations: Computational imaging technology offers users ways for immersive experiences with six degrees-of-freedom in limited volume free navigation, providing more freedom of user movement than in 3DoF+. Eventually, full-6DoF will be achieved (any translation and rotation in space), synthesizing virtual viewpoints from multiple, fixed cameras set up in various arrangements (planar arrangement, cameras in an arc, 360 divergent, etc).
Objectives: To provide normative improvements on compression of 6DoF content on top of the state of the art anchor. The improvements are evaluated simultaneously on decoded views and synthesized views.
7 - Immersive Video: Compression of dense representation of light fields
Motivations: Recently, camera technology has evolved and new capturing devices are emerging. Such capturing devices can simultaneously acquire dense spatial and angular light information. Having such information we can extract dense multiviews, perform refocusing and estimate depth information. Such cameras are expected to be replaced with current cameras to acquire 3-D real-world visual data.
Objectives: New capturing devices can capture light field in one shot, having both spatial and angular light information. Due to higher dimension of such data, i.e. 4-D, the size of capture data is not only larger but also different from traditional camera images. Therefore, a new and compatible compression for such formats would be essential so that new services can be provided.
7 - Immersive Video: Quality of Immversive Visual Media
Motivations:
Objectives:
28 - In advance signalling of MPEG containers content: In-advance signalling of MPEG containers content
Motivations: MPEG defines several container formats, in particular ISOBMFF and MPEG-2 TS. Files conformant to these formats may contain multiple media streams, each of which may conform to different media formats, with different profiles and levels. There are several file consumption scenarios under which the full content of the file is not available to a player but under which the player has nevertheless to take a decision to retrieve the file or not. These scenarios include progressive file download, adaptive streaming, etc. In such scenarios, the player needs to have sufficient information to determine if it has or not the capabilities of playing the entire content or only a part of the container content, and when multiple container files are provided, to enable a player to choose the most appropriate file(s) to process. The practice to send information about the container content, together with URL(s) to the content and prior to its retrieval, is called hereafter \"in-advance signaling\".
Objectives: To investigate what action should MPEG take
32 - Data Compression: Data Compression
Motivations: Digitalisation in all areas of society creates a data stórage problem
Objectives: Define application specific data coding algorithms for areas like UAVs, automotive, geographic information, biotechnology, industry 4.0, etc.
33 - MPEG-21 Based Smart Contracts: MPEG-21 Based Smart Contracts
Motivations: MPEG-21 provides an extensive set of standard functionalities for commerce of digital media. However, there is no standard interface with transaction systems.
Objectives: To develop standard interfaces to convert CEL contracts to code that can be executed as a smart contract on the Virtual Machine of a Blockchain
34 - Video Coding for Machines: Video Coding for Machines
Motivations: Traditional video coding standards are optimized for human consumption of video. Since today most video is analyzed by machines and human consumption is mostly related to verification of machine analysis results - if at all - a new standard might enable lower bitrates without lowering the performance of machine analysis.
Objectives: The MPEG activity on Video Coding for Machines (VCM) aims to standardize a bitstream format generated by compressing both a video stream and previously extracted features. The bitstream should enable multiple machine vision tasks
35 - 5G Opportunities: 5G Opportunities
Motivations: With the advent of the 5G network, the promised latencies and data rates might enable new applications.
Objectives: Investigate standardization activities related to 5G networks. Identify areas relevant to MPEG that require standards given the new network features of 5G and the applications enabled by them.
36 - Neural Network-based Video Compression: Neural network-based video compression
Motivations: Neural network technology opens new potential in video compression
Objectives: Exploring technology in both hybrid (replacing elements or augmenting conventional video coding architecture) and end-to-end approaches
40 - Haptics Support: Haptics Support
Motivations:
Objectives:
41 - Enhanced compression beyond VVC capability: Enhanced Compression beyond VVC Capability
Motivations: It is known that tools exist which could further increase the compression performance beyond what is possible using existing VVC profiles.
Objectives: The purpose of this exploration is assessing the potential of tools that could further increase the compression performance beyond existing VVC profiles.
42 - Future Capabilities for MPEG-I: Future Capabilities for MPEG-I
Motivations:
Objectives:
43 - Dynamic Mesh Coding: Dynamic Mesh Coding
Motivations:
Objectives:
44 - AI-based coding for graphics: AI-based coding for graphics
Motivations:
Objectives: explore the AI technologis for coding graphics
45 - Gaussian splat coding: Implicit Neural Visual Representation
Motivations: Explore potential advantages of implicit neural visual representation of 2D and 3D video to identify use cases beyond streaming, e.g. super-resolution, spatio-temporal scalability and view interpolation. Compactness of the neural network and architectural designs that provide genericity are interesting challenges to address at this stage of the exploration. Neural network compression with quantization and entropy coding will be addressed in later meeting cycles.
Objectives: Identify advantages of implicit neural visual representation of 2D and 3D video and explore the chance of potential new standard(s).
46 - Audio Coding for Machines: Audio Coding for Machines
Motivations:
Objectives:
47 - Metadata Definition and Carriage for Split Rendering: Exploration on Metadata Definition and Carriage for Split Rendering
Motivations:
Objectives:
48 - Indicating AI generated/altered content using the MPEG Systems technologies: indicating AI generated/altered content using the MPEG Systems technologies
Motivations:
Objectives:
49 - Quality metrics for 2D video: Dataset of compressed video for study of quality metrics
Motivations: Build a dataset for study and development of quality metrics for 2D video
Objectives: Assess the performance of quality metrics for 2D video
49 - Quality metrics for 2D video: Naming conventions for raw video data
Motivations: Provide guidelines for naming raw video files submitted to SC 29 working groups
Objectives: - easy naming convention - applicability across WGs
50 - Interactive testing for volumetric visual media: ITVVM
Motivations: Allow for active testing of interactive visual media
Objectives: Assess the possibility of enhancing quality assessment to incorporate interactivity
51 - MPEG multilayer video coding: Performance of multilayer video coding
Motivations: Give guidance on use of MPEG multilayer video coding standards
Objectives: Demonstrate performance of MPEG multilayer video coding standards for example use cases