Introduction to ISO/IEC JTC 1/SC 29 Coding of audio, picture, multimedia and hypermedia information: The world of JPEG and MPEG

Gary J. Sullivan (SC 29 chair)

April 2026

Scope of ISO/IEC JTC1/SC 29

Standardization in the field of

• Efficient coding of digital representations of images, audio
  and moving pictures, including
  • Conventional (natural, computer-generated and immersive) images,
    moving pictures and audio
  • Invisible light and other sensory (such as medical and satellite) images
  • Static and dynamic graphic objects
• Efficient coding of other digital information, including
  • Multimedia, environment and user related metadata
  • Sensor and actuator information related to audiovisual information
  • Other digital data in agreement with the relevant committee, such as
    genomics
• Digital information support, including
  • Synchronization, presentation, storage and transport of single or
    combinations of media
  • Media security and privacy management
  • Quality of Experience evaluation and system performance metrics

Highlights

• About 75 standards under development, more than 600
  standards published, 31 P-members, 17 O-members, >1000
  experts, ~500 attending WGs quarterly
• Reorganized in 2020: MPEG (WG 11) became MPEG WGs 2-8
  and AGs 2, 3, 5
• Most convenor terms and chair term renewed for 2024-2026,
  one advisory group retirement replacement appointed for 2025
• Highly-prominent standards recognized by ten Emmy awards
  (four within the last 5 years, ISO Base Media File Format,
  DASH, Open Font Format, and CMAF)
• Recent standards for Versatile Video Coding, JPEG XL Image
  Coding, JPEG XS Low-Latency Image Coding, JPEG AI Image
  Coding, JPEG Trust Authentication & Provenance, 3D Audio,
  Point Cloud Coding, Neural Network Coding for Media,
  Immersive Video, Smart Contracts for Media, Common Media
  Application Format, Genomic Coding, etc.

New areas

New areas & explorations: Media trust authentication &
provenance, Video coding beyond VVC capability, Plenoptic &
holographic light fields, Lenslet image & video coding for dense
light fields, Point cloud coding, Dynamic mesh coding, Avatars
for visual communication, Generative face video coding,
Advanced genomics features & technologies, image storage as
DNA, Visual event coding, Haptics, Non-fungible tokens,
Incremental neural network compression, Neural network-
based image, video and graphics coding (JPEG AI / NNVC),
Audio and video coding for machine analysis, Immersive audio,
Media for immersive applications and immersive rendering-
based displays, Implicit neural video representation, Neural
radiance fields, Gaussian splatting

Artificial intelligence / machine learning / neural networks
in media coding and applications is a pervasive recent theme

Rules & procedures

• Basic rules
  • Directives Part 1 JTC 1 Consolidated Supplement (2024, to
    change in 2026): Procedures
  • Directives Part 2: Drafting
  • Resources
  • Drafting Standards & ISO House Style
  • New: Online Standards Development
  • ITU-T/ITU-R/ISO/IEC Common Patent Policy
  • WGs are established by SCs to develop work items
    (Therefore, a WG cannot exist without at least one work item)
• Each WG and AG has a Terms of Reference description
• SCs consist of NB delegations, operating by P-member voting
• WGs consist of P-NB-qualified experts, operating by consensus
• SCs have (term-limited, Secretariat-selected) chairs; WGs have
  (non-term-limited, NB-nominated, SC-selected) convenors

Approval: Committee, Enquiry, Publication

• International Standards (ISs): NP/NWIP (12)/Request -> [WD] -> [CD/CDAM] -> DIS/DAM -> WG Completion [FDIS/FDAM] -> Publication
• Technical Reports (TRs): Request (0-4) -> [WD] -> [CDTR] -> DTR ->
  Publication
• The NP/NWIP/Request phase can be somewhat merged with others
• Square bracket stages can sometimes be skipped

Circulation type	Preparation	Translation	Commenting	Total
CD/CDAM/CDTR			8 weeks	8 weeks
(first) DIS/DAM	2 weeks	8 weeks	12 weeks	22 weeks
Subsequent DIS/DAM	2 weeks	(none)	8 weeks	10 weeks
(first) DTR	6 weeks	2 weeks	8 weeks	16 weeks
Subsequent DTR	2 weeks	(none)	8 weeks	10 weeks
DCOR			12 weeks	12 weeks
FDIS/FDAM	6-12 weeks+		8 weeks	~16 weeks

Tools & operations

• ISO Documents
  • JPEG extensively uses the Member Area
  • Other groups document working methods there
• ISO Meetings
  • All groups should have a 2+ year planning horizon
  • Most meetings use 2 registrations in practice (ISO & Host)
  • WGs & AGs typically have 4 meetings/year (and SC 29 has 2)
  • Most meetings are (best-effort) hybrid
  • ~1 meeting per year has been virtual; this is no longer required
• ISO Projects
  • Mostly used by the official editors
• Online standards development (OSD) new drafting system
• jpeg.org & session calendar
• mpeg.org & mpeg.expert session calendar and DMS

Drafting guidance

• Drafting Standards page and Resources on ISO website
• Templates and Rice example (Foreword, Introduction, 1. Scope, 2.
  Normative references, 3. Terms and definitions)
• House Style guidance
• Verbs
  • “shall” or “is required” is a command; don’t use it in a TR or a NOTE
  • “shall not” is similar
  • “should” or “is recommended” is not, but don’t use in a TR or NOTE
  • “may” or “is permitted” is permission, not possibility
  • “can” or “is possible” is possibility
  • Don’t use “may not”
  • Avoid other words (“must”, “could”, “might”, “needs to”, “has to”)
• When in doubt, avoid using figures
  • If you use them, check and follow the rules
  • Small errors in figures can become problems and cause delays
• Lowercase titles, clause/subclause headings, and defined terms
• Use Oxford spelling (also used by the UN and thus the ITU)
  • organization, signalling, colour, neighbour, but analyse
• There are special rules for formulae (not “equations”) and pseudocode
• Automatic numbering and cross-references are problematic
• No hanging paragraphs
• No clauses and subclauses called “Introduction” (use “General”)
• Clearly refer to every annex, using “shall” if normative
• Turn off automatic language detection
• Use Canadian (or British) English language identification
• Use Cambria font (Courier New for pseudocode)
• Set field code shading to “always”
• Learn how to edit field codes (use print preview, search “not found”)
• Provide a zipped copy of the software if there is software
• Use the MXM software license; make the license(s) easy to find
• Turn on viewing of white space characters
• Turn off automatic capitalization of table cells and sentences
• References go in two different places:
  • Normative references (Clause 2) for documents used as definitions
    and in “shall” statements
  • Bibliography (at the end of the document) for informatively
    referenced documents
  • All referenced documents need to be referred to (clearly)
• If the text refers to a specific location in a referenced document, the
  reference needs to be a dated reference
• To refer to the current document, use “this document”
• All defined terms need to be used in the document
• Definitions should not start with “a” or “the”
• Don’t discuss development process or WG or plans in a standard
• Some (not all) standards & TRs can be published for free by request
• There are slightly different rules for joint standards (e.g. with ITU-T)

Some neural network-related projects

• Neural Network Coding published 2022
  • ISO/IEC 15938-17, to compress representation of a neural network
  • 2nd edition 2024 with incremental compression
• JPEG AI learning-based image coding
  • To compress pictures using neural network technology
  • ITU-T T.840.1 | ISO/IEC 6048-1, finished (v1) in 2025
• JPEG Pleno learning-based point cloud coding
  • ISO/IEC 21794-6, finished 2025
• Optimization for machine analysis of coded video content
  • ITU-T H.Sup-MACVC | ISO/IEC CDTR 23888-3
  • Draft Technical Report in progress, to finish mid 2026
• Video Coding for Machines
  • ISO/IEC CD 23888-2, completion 2026
• Feature coding for machines
  • Call for Proposals April 2025
• Neural Network Video Coding exploration in Joint Video Experts Team

Developments in JPEG AI

• JPEG AI (ITU-T T.840 | ISO/IEC 6048)
  • Version 1
    • Part 1 IS / Recommendation published in 2025
    • Part 2 Profiling IS is issued by WG 1 in July 2025
    • Part 3 Reference Software IS is issued by WG 1 in July 2025
    • Part 4 Conformance DIS under ballot
    • Part 5 File format IS is issued by WG 1 in July 2025
  • Compression efficiency gains between 12.5% to 27.9% over the
    VVC Intra coding anchor, with a wide range of complexity tradeoffs
    (7 to 216 kMAC/px) at the decoder side
  • JPEG AI Version 2 is under discussion
    • JPEG AI requirements not yet addressed in version 1, e.g.
      related to processing and computer vision tasks
    • Significantly improved solutions for JPEG AI requirements
      already addressed in version 1, e.g. compression efficiency

Developments in video coding

• Versatile video coding (ITU-T H.266 | ISO/IEC 23090-3)
  • Finished (v1 publication) in 2020
  • Now approaching v4, coordinated with VSEI
  • Roughly 50% bit rate savings over HEVC (eq. perceptual quality)
• Low-complexity enhancement video coding ISO/IEC 23094-2 in 2021
• MPEG immersive video ISO/IEC 23090-12 in 2023
• Next-generation video coding exploration in Joint Video Experts Team
  • Enhanced Compression Model: Classical-style signal processing
    • Roughly 28% savings over VVC (random access config.)
  • Neural Network Video Coding
    • Hybrid tools approach
      • Intra prediction
      • Super-resolution
      • Post-filter
      • Loop filter (VLOP 10 kMAC/pxl 6%, LOP 22 kMAC/pxl 8%, HOP 14%)
• Joint Call for Evidence issued: Response evaluation Oct 2025
• Draft Call for Proposals new std: Response submission Nov 2026

MPEG Systems technologies

New ITU Partnering: Biomedical Signals

• Signal examples:
  • ECG heart signals
  • EEG brainwave signals
  • EMG muscle signals
• Standard requested by DICOM WG32 Neurophysiology
• Project launch in ITU-T, being joined by MPEG Audio
• ITU Call for Proposals in April 2024, evaluation January 2025
• Focus on near-lossless to lossless coding
• Draft text and software available
  • Target completion July 2026

Developments in graphics coding

• Point Cloud Compression
  • Video-based PCC (V-PCC): ISO/IEC 23090-5
    Finalized v1 (2023), with ongoing improvements in V3C integration and metadata
  • Geometry-based PCC (G-PCC): ISO/IEC 23090-9
    Finalized v1 (2023), v2 near publication (support for dynamic sequences, improved
    inter-frame coding)
  • AI-based PCC (AI-PCC): in progress
    Leverages learned prediction and compression models, Compression gains
    demonstrated on sparse/dynamic content
• Dynamic Mesh Compression
  • Video-based Dynamic Mesh Compression (V-DMC): ISO/IEC 23090-29
    in progress – Efficient coding of animated meshes using video codecs
• Gaussian Splat Representation and Coding (GSC)
  • Exploratory activity under MPEG 3DGC (WG 7) and Video (WG 4)
    Focus on high-quality, real-time renderable representations
    Aligns with neural graphics pipeline
    Dataset and benchmark established; call for proposals expected in 2026
• Haptics: ISO/IEC 23090-31
  • Compression of haptic timelines and spatial signals

Genomic coding ISO/IEC 23092

• Part 1: Transport and storage of genomic information:
  • Data formats for both transport and storage of genomic information,
    including the conversion processes and the indexing technology
• Part 2: Coding of genomic information
  • Coding technology for references and genomic reads
• Part 3: Metadata & application formats interfaces (APIs)
  • Information metadata, metrics metadata, clinical data linkage metadata,
    auxiliary fields, SAM interoperability, protection metadata and APIs
• Part 4: Reference software
• Part 5: Conformance
• Part 6: Coding of genomic annotations
  • Compressed representation of genomic annotations of type: variants with
    genotyping, functional annotations, tracks, expression and contact matrices
    (from Hi-C experiments)
• Part 7: MPEG-G application formats and profiles
  • Use case oriented structuring of genomic data and associated profiles