A method for lossy image or video encoding, transmission, and decoding, the method comprising the steps of: with a first neural network, producing a latent representation of optical flow information using the first image and the second image, the optical flow information being indicative of a difference between the first image and the second image; with a second neural network, decoding the latent representation of optical flow information to produce an approximation of the optical flow information; with a third neural network, producing an output image using the optical flow information, wherein the output image is an approximation of the first image; wherein the first image and the second image comprise data arranged in a respective plurality of image channels; and wherein using the first image and the second image to produce a latent representation of optical flow information comprises using a subset of the plurality of channels.
H04N 19/186 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a colour or a chrominance component
H04N 19/537 - Motion estimation other than block-based
H04N 19/59 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution
2.
METHOD AND DATA PROCESSING SYSTEM FOR LOSSY IMAGE OR VIDEO ENCODING, TRANSMISSION AND DECODING
A method of training one or more neural networks, the one or more neural networks being for use in lossy image or video encoding, transmission and decoding, the method comprising the steps of: using a first I-frame module to produce an output I-frame from a first input image, the I-frame module comprising one or more neural networks; using a P- and/or B-frame module to produce an output P- or B-frame from a second input image and the output I-frame, the P- and/or B-frame module comprising one or more neural networks; evaluating a function based on a difference between the second input image and the output P- or B-frame; updating the parameters of the one or more neural networks of the P- and/or B-frame module based on the evaluated function; swapping the first I-frame module with a second I-frame module.
G06N 3/084 - Backpropagation, e.g. using gradient descent
H04N 19/147 - Data rate or code amount at the encoder output according to rate distortion criteria
H04N 19/159 - Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction
H04N 19/172 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a picture, frame or field
H04N 19/19 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the adaptation method, adaptation tool or adaptation type used for the adaptive coding using optimisation based on Lagrange multipliers
H04N 19/577 - Motion compensation with bidirectional frame interpolation, i.e. using B-pictures
H04N 19/154 - Measured or subjectively estimated visual quality after decoding, e.g. measurement of distortion
H04N 19/192 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the adaptation method, adaptation tool or adaptation type used for the adaptive coding the adaptation method, adaptation tool or adaptation type being iterative or recursive
H04N 19/196 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the adaptation method, adaptation tool or adaptation type used for the adaptive coding being specially adapted for the computation of encoding parameters, e.g. by averaging previously computed encoding parameters
G06N 3/088 - Non-supervised learning, e.g. competitive learning
G06V 10/82 - Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
3.
METHOD AND DATA PROCESSING SYSTEM FOR LOSSY IMAGE OR VIDEO ENCODING, TRANSMISSION AND DECODING
A method for lossy image or video encoding and transmission, and decoding, the method comprising the steps of: receiving a first image and a second image at a first computer system; with a first neural network, producing a latent representation of optical flow information using the first image and the second image, the optical flow information being indicative of a difference between the first image and the second image; transmitting the latent representation of optical flow information to a second computer system; with a second neural network, decoding the latent representation of optical flow information to produce an approximation of the optical flow information; scaling the optical flow information and with a third neural network, producing an output image using the scaled optical flow information, wherein the output image is an approximation of the first image.
A method for lossy image or video encoding, transmission and decoding, the method comprising the steps of: receiving a first image at a first computer system; encoding the first image using a first neural network to produce a latent representation; at least partially masking the latent representation; transmitting the masked latent representation to a second computer system; decoding the masked latent representation using a second neural network to produce an output image, wherein the output image is an approximation of the first image.
H04N 19/85 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression
H04N 19/86 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression involving reduction of coding artifacts, e.g. of blockiness
H04N 19/87 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression involving scene cut or scene change detection in combination with video compression
H04N 19/91 - Entropy coding, e.g. variable length coding [VLC] or arithmetic coding
5.
METHOD AND DATA PROCESSING SYSTEM FOR LOSSY IMAGE OR VIDEO ENCODING, TRANSMISSION AND DECODING
A method of training one or more neural networks for use in lossy image or video encoding, transmission and decoding. The method comprises receiving an input image at a first computer system; downsampling the input image with a downsampler to produce a downsampled input image; encoding the downsampled input image using a first neural network to produce a latent representation; decoding the latent representation using a second neural network to produce an output image, wherein the output image is an approximation of the input image; upsampling the output image with an upsampler to produce an upsampled output image; evaluating a function based on a difference between one or more of: the output image and the input image, the output image and the downsampled input image, the upsampled output image and the input image, and/or the upsampled output image and the downsampled input image.
H04N 19/33 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability in the spatial domain
H04N 19/85 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression
H04N 19/90 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using coding techniques not provided for in groups , e.g. fractals
6.
METHOD AND DATA PROCESSING SYSTEM FOR LOSSY IMAGE OR VIDEO ENCODING, TRANSMISSION AND DECODING
A method for lossy video encoding, transmission and decoding, the method comprising the steps of: receiving an input video at a first computer system; encoding an input frame of the input video to produce a latent representation; producing a quantized latent; producing a hyper-latent representation; producing a quantized hyper-latent; entropy encoding the quantized latent; transmitting the entropy encoded quantized latent and the quantized hyper-latent to a second computer system; decoding the quantized hyper-latent to produce a set of context variables, wherein the set of context variables comprise a temporal context variable; entropy decoding the entropy encoded quantized latent using the set of context variables to obtain an output quantized latent; and decoding the output quantized latent to produce an output frame, wherein the output frame is an approximation of the input frame.
A method of training one or more neural networks, the one or more neural networks being for use in lossy image or video encoding, transmission and decoding, the method comprising the steps of: receiving an input image at a first computer system; encoding the input image using a first neural network to produce a corresponding a latent representation; decoding the latent representation using a second neural network to produce an output image, wherein the output image is an approximation at a first target image quality of the input image; evaluating a function based on a difference between the output image and a corresponding image having an image quality different to the target image quality; updating the parameters of the first neural network and the second neural network based on the evaluated function to produce a first trained neural network and a second trained neural network.
H04N 19/147 - Data rate or code amount at the encoder output according to rate distortion criteria
H04N 19/172 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a picture, frame or field
8.
METHOD AND DATA PROCESSING SYSTEM FOR LOSSY IMAGE OR VIDEO ENCODING, TRANSMISSION AND DECODING USING IMAGE COMPARISONS AND MACHINE LEARNING
A method for lossy image or video encoding, transmission and decoding, the method comprising the steps of: receiving a first input image of a sequence of input images at a first computer system; comparing the first input image to a second image of the sequence of images; transmitting information from the first computer system to the second computer system; and outputting an output image at a second computer system, the output image corresponding to the first input image; wherein an amount of information associated with the second image used to produce the output image is based on a result of the comparison.
H04N 19/142 - Detection of scene cut or scene change
H04N 19/503 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
H04N 19/577 - Motion compensation with bidirectional frame interpolation, i.e. using B-pictures
H04N 19/172 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a picture, frame or field
9.
METHOD AND DATA PROCESSING SYSTEM FOR LOSSY IMAGE OR VIDEO COMPRESSION
A method of training one or more neural networks, the one or more neural networks being for use in lossy image or video encoding, transmission and decoding, the method comprising the steps of: receiving first and second input images; estimating optical flow information indicative of a difference between representations of the first and second input images; receiving first and second output images; estimating optical flow information indicative of a difference between representations of the first output image and the second output image; updating the parameters of the one or more neural networks based on the evaluated function; and repeating the above steps using a first set of input images to produce one or more trained neural network.
A method of training one or more neural networks, the one or more neural networks being for use in lossy image or video encoding, transmission and decoding, the method comprising the steps of: receiving first and second input images; estimating optical flow information indicative of a difference between representations of the first and second input images; receiving first and second output images, wherein the first and second output images comprise approximations of the first and second input images encoded and decoded by one or more neural networks; estimating optical flow information indicative of a difference between representations of the first output image and the second output image; evaluating a function based on a difference between the estimated optical flow information associated with the first and second input images and the estimated optical flow information associated with the first and second output images.
A method for lossy image and video encoding, transmission and decoding, the method comprising the steps of: receiving an input image at a first computer system; encoding the input image using a first trained neural network to produce a latent representation; performing a quantization process on the latent representation to produce a quantized latent; transmitting the quantized latent to a second computer system; decoding the quantized latent using a denoising process to produce an output image, wherein the output image is an approximation of the input image.
A method for lossy image or video encoding, transmission and decoding, the method comprising the steps of: receiving an input image at a first computer system; encoding the input image using a first trained neural network to produce a latent representation; transmitting the latent representation to a second computer system; identifying one or more values of the latent representation that have been affected by an error; replacing one or more of the identified values of the latent representation with a replacement value; and decoding the latent representation using a second trained neural network to produce an output image, wherein the output image is an approximation of the input image
H04N 19/895 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression involving methods or arrangements for detection of transmission errors at the decoder in combination with error concealment
H04N 19/46 - Embedding additional information in the video signal during the compression process
H04N 19/51 - Motion estimation or motion compensation
13.
METHOD AND DATA PROCESSING SYSTEM FOR LOSSY IMAGE OR VIDEO ENCODING, TRANSMISSION AND DECODING
A method for lossy image and video encoding, transmission and decoding, the method comprising the steps of: receiving an input image at a first computer system; encoding the input image using a first trained neural network to produce a latent representation; performing a quantization process on the latent representation to produce a quantized latent, wherein the sizes of the bins used in the quantization process are based on the input image; transmitting the quantized latent to a second computer system; decoding the quantized latent using a second trained neural network to produce an output image, wherein the output image is an approximation of the input image.
G06T 5/20 - Image enhancement or restoration using local operators
H04N 19/119 - Adaptive subdivision aspects e.g. subdivision of a picture into rectangular or non-rectangular coding blocks
H04N 19/182 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a pixel
H04N 19/91 - Entropy coding, e.g. variable length coding [VLC] or arithmetic coding
14.
IMAGE COMPRESSION AND DECODING, VIDEO COMPRESSION AND DECODING: TRAINING METHODS AND TRAINING SYSTEMS
A computer-implemented method of training an image generative network fθ for a set of training images, in which an output image {circumflex over (x)} is generated from an input image x of the set of training images non-losslessly, and in which a proxy network is trained for a gradient intractable perceptual metric that evaluates a quality of an output image {circumflex over (x)} given an input image x, the method of training using a plurality of scales for input images from the set of training images. In an embodiment, a blindspot network bα is trained which generates an output image {tilde over (x)} from an input image x. Related computer systems, computer program products and computer-implemented methods of training are disclosed.
A method for lossy video encoding, transmission and decoding, the method comprising the steps of: receiving a first frame and a second frame at a first computer system; determining a first flow between the first frame and the second frame; determining a second flow based on the first frame and the second frame; encoding an input based on the first flow and the second flow using a first trained neural network to produce a latent representation; transmitting the latent representation to a second computer system; decoding the latent representation using a second trained neural network to produce an output flow; and using the output flow to obtain an output frame, wherein the output frame is an approximation of the second frame.
H04N 7/12 - Systems in which the television signal is transmitted via one channel or a plurality of parallel channels, the bandwidth of each channel being less than the bandwidth of the television signal
G06T 3/18 - Image warping, e.g. rearranging pixels individually
H04N 19/137 - Motion inside a coding unit, e.g. average field, frame or block difference
H04N 19/172 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a picture, frame or field
16.
Method and data processing system for lossy image or video encoding, transmission, and decoding
A method for lossy image or video encoding, transmission and decoding, the method comprising the steps of: receiving an input image at a first computer system; encoding the input image using a first trained neural network to produce a latent representation, wherein the latent representation has a probability distribution described by at least one probability parameter; dividing the latent representation into a plurality of sub-latent representations, wherein each sub-latent representation has a sub-probability distribution described by at least one sub-probability parameter; entropy encoding the plurality of sub-latent representations using the plurality of at least one sub-probability parameters to produce a bitstream; transmitting the bitstream to a second computer system; entropy decoding the bitstream using the plurality of at least one sub-probability parameters to retrieve the plurality of sub-latent representations and combining the plurality of sub-latent representations to retrieve the latent representation; and decoding the latent representation using a second trained neural network to produce an output image, wherein the output image is an approximation of the input image.
H04N 19/91 - Entropy coding, e.g. variable length coding [VLC] or arithmetic coding
H04N 19/196 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the adaptation method, adaptation tool or adaptation type used for the adaptive coding being specially adapted for the computation of encoding parameters, e.g. by averaging previously computed encoding parameters
17.
METHOD AND DATA PROCESSING SYSTEM FOR LOSSY IMAGE OR VIDEO ENCODING, TRANSMISSION AND DECODING
A method for lossy video encoding, transmission and decoding, the method comprising the steps of: receiving an input frame and a previous frame at a first computer system; encoding the input frame and an input based on the previous frame using a first trained neural network to produce a first latent representation; encoding the input frame using a second trained neural network to produce a second latent representation; transmitting the first and the second latent representation to a second computer system; decoding the first latent representation using a third trained neural network to obtain an output flow map between the input frame and the previous frame; and decoding the second latent representation and an input based on the output flow map using a fourth trained neural network to produce an output frame, wherein the output frame is an approximation of the input frame.
A method of training one or more neural networks, the one or more neural networks being for use in lossy image or video encoding, transmission and decoding, the method comprising the steps of: receiving an input image at a first computer system; encoding the input image using a first neural network to produce a latent representation; decoding the latent representation using a second neural network to produce an output image, wherein the output image is an approximation of the input image; evaluating a function based on a difference between the output image and the input image; updating the parameters of the first neural network and the second neural network based on the evaluated function; and repeating the above steps using a first set of input images to produce a first trained neural network and a second trained neural network; wherein the difference between the output image and the input image is determined based on the output of a neural network acting as a discriminator; the parameters of the neural network acting as a discriminator are additionally updated based on the evaluated function; and the parameters of the neural network acting as a discriminator are updated at a first learning rate; wherein, after at least one of the updates of the parameters of the neural network acting as a discriminator, the first learning rate is updated; and the update to the first learning rate is based on an error of the output of the neural network acting as a discriminator.
A computer-implemented method for lossy image or video compression, transmission and decoding, the method including the steps of: (i) receiving an input image at a first computer system; (ii) encoding the input image using a first trained neural network, using the first computer system, to produce a latent representation; (iii) quantizing the latent representation using the first computer system to produce a quantized latent; (iv) entropy encoding the quantized latent into a bitstream, using the first computer system; (v) transmitting the bitstream to a second computer system; (vi) the second computer system entropy decoding the bitstream to produce the quantized latent; (vii) the second computer system using a second trained neural network to produce an output image from the quantized latent, wherein the output image is an approximation of the input image.
Lossy or lossless compression and transmission, comprising the steps of: (i) receiving an input image; (ii) encoding it to produce a y latent representation; (iii) encoding the y latent representation to produce a z hyperlatent representation; (iv) quantizing the z hyperlatent representation to produce a quantized z hyperlatent representation; (v) entropy encoding the quantized z hyperlatent representation into a first bitstream, (vi) processing the quantized z hyperlatent representation to obtain a location entropy parameter μy, an entropy scale parameter σy, and a context matrix Ay of the y latent representation; (vii) processing the y latent representation, the location entropy parameter py and the context matrix Ay, to obtain quantized latent residuals; (viii) entropy encoding the quantized latent residuals into a second bitstream; and (ix) transmitting the bitstreams.
H04N 19/13 - Adaptive entropy coding, e.g. adaptive variable length coding [AVLC] or context adaptive binary arithmetic coding [CABAC]
G06V 10/422 - Global feature extraction by analysis of the whole pattern, e.g. using frequency domain transformations or autocorrelation for representing the structure of the pattern or shape of an object therefor
H04N 19/42 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
21.
METHOD AND DATA PROCESSING SYSTEM FOR LOSSY IMAGE OR VIDEO ENCODING, TRANSMISSION AND DECODING
A method of training one or more neural networks, the one or more neural networks being for use in lossy image or video encoding, transmission and decoding, the method comprising steps including: receiving an input image at a first computer system; encoding the input image using a first neural network and decoding the latent representation using a second neural network to produce an output image; at least one of the plurality of layers of the first or second neural network comprises a transformation; and the method further comprises the steps of: evaluating a difference between the output image and the input image and evaluating a function based on an output of the transformation; updating the parameters of the first neural network and the second neural network based on the evaluated difference and the evaluated function; and repeating the above steps.
There is disclosed a computer-implemented method for lossy image or video compression, transmission and decoding, the method including the steps of: (i) receiving an input image at a first computer system; (ii) encoding the input image using a first trained neural network, using the first computer system, to produce a latent representation; (iii) quantizing the latent representation using the first computer system to produce a quantized latent; (iv) entropy encoding the quantized latent into a bitstream, using the first computer system; (v) transmitting the bitstream to a second computer system; (vi) the second computer system entropy decoding the bitstream to produce the quantized latent; (vii) the second computer system using a second trained neural network to produce an output image from the quantized latent, wherein the output image is an approximation of the input image. Related computer-implemented methods, systems, computer-implemented training methods and computer program products are disclosed.
A method for lossy video encoding, transmission and decoding, the method comprising the steps of: receiving an input video at a first computer system; encoding an input frame of the input video to produce a latent representation; producing a quantized latent; producing a hyper-latent representation; producing a quantized hyper-latent; entropy encoding the quantized latent; transmitting the entropy encoded quantized latent and the quantized hyper-latent to a second computer system; decoding the quantized hyper-latent to produce a set of context variables, wherein the set of context variables comprise a temporal context variable; entropy decoding the entropy encoded quantized latent using the set of context variables to obtain an output quantized latent; and decoding the output quantized latent to produce an output frame, wherein the output frame is an approximation of the input frame.
H04N 19/12 - Selection from among a plurality of transforms or standards, e.g. selection between discrete cosine transform [DCT] and sub-band transform or selection between H.263 and H.264
H04N 19/132 - Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
H04N 19/172 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a picture, frame or field
H04N 19/182 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a pixel
H04N 19/42 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
H04N 19/625 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding using discrete cosine transform [DCT]
H04N 19/63 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding using sub-band based transform, e.g. wavelets
H04N 19/88 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression involving rearrangement of data among different coding units, e.g. shuffling, interleaving, scrambling or permutation of pixel data or permutation of transform coefficient data among different blocks
24.
Image compression and decoding, video compression and decoding: methods and systems
There is disclosed a computer-implemented method for lossy image or video compression, transmission and decoding, the method including the steps of: (i) receiving an input image at a first computer system; (ii) encoding the input image using a first trained neural network, using the first computer system, to produce a latent representation; (iii) quantizing the latent representation using the first computer system to produce a quantized latent; (iv) entropy encoding the quantized latent into a bitstream, using the first computer system; (v) transmitting the bitstream to a second computer system; (vi) the second computer system entropy decoding the bitstream to produce the quantized latent; (vii) the second computer system using a second trained neural network to produce an output image from the quantized latent, wherein the output image is an approximation of the input image. Related computer-implemented methods, systems, computer-implemented training methods and computer program products are disclosed.
There is disclosed a computer-implemented method for lossy image or video compression, transmission and decoding, the method including the steps of: (i) receiving an input image at a first computer system; (ii) encoding the input image using a first trained neural network, using the first computer system, to produce a latent representation; (iii) quantizing the latent representation using the first computer system to produce a quantized latent; (iv) entropy encoding the quantized latent into a bitstream, using the first computer system; (v) transmitting the bitstream to a second computer system; (vi) the second computer system entropy decoding the bitstream to produce the quantized latent; (vii) the second computer system using a second trained neural network to produce an output image from the quantized latent, wherein the output image is an approximation of the input image. Related computer-implemented methods, systems, computer-implemented training methods and computer program products are disclosed.
There is disclosed a computer-implemented method for lossy image or video compression, transmission and decoding, the method including the steps of: (i) receiving an input image at a first computer system; (ii) encoding the input image using a first trained neural network, using the first computer system, to produce a latent representation; (iii) quantizing the latent representation using the first computer system to produce a quantized latent; (iv) entropy encoding the quantized latent into a bitstream, using the first computer system; (v) transmitting the bitstream to a second computer system; (vi) the second computer system entropy decoding the bitstream to produce the quantized latent; (vii) the second computer system using a second trained neural network to produce an output image from the quantized latent, wherein the output image is an approximation of the input image. Related computer-implemented methods, systems, computer-implemented training methods and computer program products are disclosed.
There is disclosed a computer-implemented method for lossy image or video compression, transmission and decoding, the method including the steps of: (i) receiving an input image at a first computer system; (ii) encoding the input image using a first trained neural network, using the first computer system, to produce a latent representation; (iii) quantizing the latent representation using the first computer system to produce a quantized latent; (iv) entropy encoding the quantized latent into a bitstream, using the first computer system; (v) transmitting the bitstream to a second computer system; (vi) the second computer system entropy decoding the bitstream to produce the quantized latent; (vii) the second computer system using a second trained neural network to produce an output image from the quantized latent, wherein the output image is an approximation of the input image. Related computer-implemented methods, systems, computer-implemented training methods and computer program products are disclosed.
There is disclosed a computer-implemented method for lossy image or video compression, transmission and decoding, the method including the steps of: (i) receiving an input image at a first computer system; (ii) encoding the input image using a first trained neural network, using the first computer system, to produce a latent representation; (iii) quantizing the latent representation using the first computer system to produce a quantized latent; (iv) entropy encoding the quantized latent into a bitstream, using the first computer system; (v) transmitting the bitstream to a second computer system; (vi) the second computer system entropy decoding the bitstream to produce the quantized latent; (vii) the second computer system using a second trained neural network to produce an output image from the quantized latent, wherein the output image is an approximation of the input image. Related computer-implemented methods, systems, computer-implemented training methods and computer program products are disclosed.
There is disclosed a computer-implemented method for lossy image or video compression, transmission and decoding, the method including the steps of: (i) receiving an input image at a first computer system; (ii) encoding the input image using a first trained neural network, using the first computer system, to produce a latent representation; (iii) quantizing the latent representation using the first computer system to produce a quantized latent; (iv) entropy encoding the quantized latent into a bitstream, using the first computer system; (v) transmitting the bitstream to a second computer system; (vi) the second computer system entropy decoding the bitstream to produce the quantized latent; (vii) the second computer system using a second trained neural network to produce an output image from the quantized latent, wherein the output image is an approximation of the input image. Related computer-implemented methods, systems, computer-implemented training methods and computer program products are disclosed.
There is disclosed a computer-implemented method for lossy image or video compression, transmission and decoding, the method including the steps of: (i) receiving an input image at a first computer system; (ii) encoding the input image using a first trained neural network, using the first computer system, to produce a latent representation; (iii) quantizing the latent representation using the first computer system to produce a quantized latent; (iv) entropy encoding the quantized latent into a bitstream, using the first computer system; (v) transmitting the bitstream to a second computer system; (vi) the second computer system entropy decoding the bitstream to produce the quantized latent; (vii) the second computer system using a second trained neural network to produce an output image from the quantized latent, wherein the output image is an approximation of the input image. Related computer-implemented methods, systems, computer-implemented training methods and computer program products are disclosed.
A computer-implemented method for lossy image or video compression, transmission and decoding, the method including the steps of: (i) receiving an input image at a first computer system; (ii) encoding the input image using a first trained neural network, using the first computer system, to produce a latent representation; (iii) quantizing the latent representation using the first computer system to produce a quantized latent; (iv) entropy encoding the quantized latent into a bitstream, using the first computer system; (v) transmitting the bitstream to a second computer system; (vi) the second computer system entropy decoding the bitstream to produce the quantized latent; (vii) the second computer system using a second trained neural network to produce an output image from the quantized latent, wherein the output image is an approximation of the input image. Related computer-implemented methods, systems, computer-implemented training methods and computer program products.
A method for lossy video encoding, transmission and decoding, the method comprising the steps of: receiving an input video at a first computer system; encoding an input frame of the input video to produce a latent representation; producing a quantized latent; producing a hyper-latent representation; producing a quantized hyper-latent; entropy encoding the quantized latent; transmitting the entropy encoded quantized latent and the quantized hyper-latent to a second computer system; decoding the quantized hyper-latent to produce a set of context variables, wherein the set of context variables comprise a temporal context variable; entropy decoding the entropy encoded quantized latent using the set of context variables to obtain an output quantized latent; and decoding the output quantized latent to produce an output frame, wherein the output frame is an approximation of the input frame.
H04N 19/91 - Entropy coding, e.g. variable length coding [VLC] or arithmetic coding
H04N 19/503 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
H04N 19/593 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving spatial prediction techniques
H04N 19/59 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution
H04N 19/82 - Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation involving filtering within a prediction loop
H04N 19/86 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression involving reduction of coding artifacts, e.g. of blockiness
α is trained which generates an output image {tilde over (x)} from an input image x. Related computer systems, computer program products and computer-implemented methods of training are disclosed.
G06N 3/088 - Non-supervised learning, e.g. competitive learning
H04N 19/59 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution
G06N 20/10 - Machine learning using kernel methods, e.g. support vector machines [SVM]
34.
METHOD AND DATA PROCESSING SYSTEM FOR LOSSY IMAGE OR VIDEO ENCODING, TRANSMISSION AND DECODING
A method for lossy image or video encoding, transmission and decoding, the method comprising the steps of: receiving an input image at a first computer system; encoding the input image using a first trained neural network to produce a latent representation; performing a quantization process on the latent representation to produce a quantized latent; transmitting the quantized latent to a second computer system; decoding the quantized latent using a denoising process to produce an output image, wherein the output image is an approximation of the input image.
H04N 19/90 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using coding techniques not provided for in groups , e.g. fractals
Lossy or lossless compression and transmission, comprising the steps of: (i) receiving an input image; (ii) encoding it to produce a y latent representation; (iii) encoding the y latent representation to produce a z hyperlatent representation; (iv) quantizing the z hyperlatent representation to produce a quantized z hyperlatent representation; (v) entropy encoding the quantized z hyperlatent representation into a first bitstream, (vi) processing the quantized z hyperlatent representation to obtain a location entropy parameter μy, an entropy scale parameter σy, and a context matrix Ay of the y latent representation; (vii) processing the y latent representation, the location entropy parameter μy and the context matrix Ay, to obtain quantized latent residuals; (viii) entropy encoding the quantized latent residuals into a second bitstream; and (ix) transmitting the bitstreams.
H04N 19/42 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
G06V 10/422 - Global feature extraction by analysis of the whole pattern, e.g. using frequency domain transformations or autocorrelation for representing the structure of the pattern or shape of an object therefor
36.
Image compression and decoding, video compression and decoding: methods and systems
There is disclosed a computer-implemented method for lossy image or video compression, transmission and decoding, the method including the steps of: (i) receiving an input image at a first computer system; (ii) encoding the input image using a first trained neural network, using the first computer system, to produce a latent representation; (iii) quantizing the latent representation using the first computer system to produce a quantized latent; (iv) entropy encoding the quantized latent into a bitstream, using the first computer system; (v) transmitting the bitstream to a second computer system; (vi) the second computer system entropy decoding the bitstream to produce the quantized latent; (vii) the second computer system using a second trained neural network to produce an output image from the quantized latent, wherein the output image is an approximation of the input image. Related computer-implemented methods, systems, computer-implemented training methods and computer program products are disclosed.
A method of training one or more neural networks, the one or more neural networks being for use in lossy image or video encoding, transmission and decoding, the method comprising steps including: receiving an input image at a first computer system; encoding the input image using a first neural network and decoding the latent representation using a second neural network to produce an output image;at least one of the plurality of layers of the first or second neural network comprises a transformation; and the method further comprises the steps of: evaluating a difference between the output image and the input image and evaluating a function based on an output of the transformation; updating the parameters of the first neural network and the second neural network based on the evaluated difference and the evaluated function; and repeating the above steps.
A method for lossy image or video encoding, transmission and decoding, the method comprising the steps of: receiving an input image at a first computer system; encoding the input image using a first trained neural network to produce a latent representation; identifying one or more regions of the input image associated with high visual sensitivity; encoding the one or more regions of the input image associated with high visual sensitivity using a second trained neural network to produce one or more region latent representations; performing a quantization process on the latent representation and the one or more region latent representations; transmitting the result of the quantization process to a second computer system; decoding the result of the quantization process to produce an output image, wherein the output image is an approximation of the input image.
There is provided a method for lossy image or video encoding and transmission, including the steps of receiving an input image at a first computer system, encoding the input image using a first trained neural network to produce a latent representation, performing a quantization process on the latent representation to produce a quantized latent, and transmitting the quantized latent to a second computer system.
A method for lossy image and video encoding, transmission and decoding, the method comprising the steps of: receiving an input image at a first computer system; encoding the input image using a first trained neural network to produce a latent representation; performing a quantization process on the latent representation to produce a quantized latent, wherein the sizes of the bins used in the quantization process are based on the input image; transmitting the quantized latent to a second computer system; decoding the quantized latent using a second trained neural network to produce an output image, wherein the output image is an approximation of the input image.
H04N 19/136 - Incoming video signal characteristics or properties
H04N 19/17 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
H04N 19/186 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a colour or a chrominance component
41.
Method and data processing system for lossy image or video encoding, transmission and decoding
A method for lossy image or video encoding, transmission and decoding, the method comprising the steps of: receiving an input image at a first computer system; encoding the first input training image using a first trained neural network to produce a latent representation; performing a quantization process on the latent representation to produce a quantized latent; entropy encoding the quantized latent using a probability distribution, wherein the probability distribution is defined using a tensor network; transmitting the entropy encoded quantized latent to a second computer system; entropy decoding the entropy encoded quantized latent using the probability distribution to retrieve the quantized latent; and decoding the quantized latent using a second trained neural network to produce an output image, wherein the output image is an approximation of the input training image.
y, to obtain quantized latent residuals; (viii) entropy encoding the quantized latent residuals into a second bitstream; and (ix) transmitting the bitstreams.
H04N 19/42 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
G06V 10/422 - Global feature extraction by analysis of the whole pattern, e.g. using frequency domain transformations or autocorrelation for representing the structure of the pattern or shape of an object therefor
43.
Method and data processing system for lossy image or video encoding, transmission and decoding
A method for lossy image or video encoding, transmission and decoding, the method comprising the steps of: receiving an input image at a first computer system; encoding the input image using a first trained neural network to produce a latent representation; identifying one or more regions of the input image associated with high visual sensitivity; encoding the one or more regions of the input image associated with high visual sensitivity using a second trained neural network to produce one or more region latent representations; performing a quantization process on the latent representation and the one or more region latent representations; transmitting the result of the quantization process to a second computer system; decoding the result of the quantization process to produce an output image, wherein the output image is an approximation of the input image.
There is disclosed a computer-implemented method for lossy image or video compression, transmission and decoding, the method including the steps of: (i) receiving an input image at a first computer system; ({umlaut over (υ)}) encoding the input image using a first trained neural network, using the first computer system, to produce a latent representation; (iii) quantizing the latent representation using the first computer system to produce a quantized latent; (iv) entropy encoding the quantized latent into a bitstream, using the first computer system; (v) transmitting the bitstream to a second computer system; (vi) the second computer system entropy decoding the bitstream to produce the quantized latent; (vii) the second computer system using a second trained neural network to produce an output image from the quantized latent, wherein the output image is an approximation of the input image. Related computer-implemented methods, systems, computer-implemented training methods and computer program products are disclosed.
H04N 19/12 - Selection from among a plurality of transforms or standards, e.g. selection between discrete cosine transform [DCT] and sub-band transform or selection between H.263 and H.264
H04N 19/42 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
G06V 10/422 - Global feature extraction by analysis of the whole pattern, e.g. using frequency domain transformations or autocorrelation for representing the structure of the pattern or shape of an object therefor
H04N 19/13 - Adaptive entropy coding, e.g. adaptive variable length coding [AVLC] or context adaptive binary arithmetic coding [CABAC]
46.
SYSTEM AND METHOD FOR LOSSY IMAGE AND VIDEO COMPRESSION AND/OR TRANSMISSION UTILIZING A METANETWORK OR NEURAL NETWORKS
A system and method for lossy image and video compression that utilizes a metanetwork to generate a set of hyperparameters necessary for an image encoding network to reconstruct the desired image from a given noise image, and for lossy image and video compression and transmission that utilizes a neural network as a function to map a known noise image to a desired or target image, allowing the transfer only of hyperparameters of the function instead of a compressed version of the image itself. This allows the recreation of a high-quality approximation of the desired image by any system receiving the hyperparameters, provided that the receiving system possesses the same noise image and a similar neural network. The amount of data required to transfer an image of a given quality is dramatically reduced versus existing image compression technology.
A method for lossy image or video encoding, transmission and decoding, the method comprising the steps of: receiving an input image at a first computer system; encoding the input image using a first trained neural network to produce a latent representation; identifying one or more regions of the input image associated with high visual sensitivity; encoding the one or more regions of the input image associated with high visual sensitivity using a second trained neural network to produce one or more region latent representations; performing a quantization process on the latent representation and the one or more region latent representations; transmitting the result of the quantization process to a second computer system; decoding the result of the quantization process to produce an output image, wherein the output image is an approximation of the input image.
H04N 19/14 - Coding unit complexity, e.g. amount of activity or edge presence estimation
H04N 19/167 - Position within a video image, e.g. region of interest [ROI]
H04N 19/17 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
G06N 3/04 - Architecture, e.g. interconnection topology
yyyyy y y ; and (ix) transmitting the first bitstream and the second bitstream. Related computer-implemented methods, systems, computer-implemented training methods and computer program products are disclosed.
H04N 19/59 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution
G06N 3/04 - Architecture, e.g. interconnection topology
θθαα is trained which generates an output image it from an input image x̂, in which a proxy network is trained for a gradient intractable perceptual metric that evaluates a quality of an output image x̂ given an input image x, and in which a blindspot proxy network is trained for labelling blindspot samples. Related computer systems, computer program products and computer-implemented methods of training are disclosed.
H04N 19/59 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution
G06N 3/04 - Architecture, e.g. interconnection topology
09 - Scientific and electric apparatus and instruments
42 - Scientific, technological and industrial services, research and design
Goods & Services
Computer hardware; computer software; application software; artificial intelligence programs; all of the aforesaid goods for use in relation to the compression of data; apparatus for data storage and data compression; computer software applications utilising artificial intelligence for use in relation to the compression of data; electronic data storage materials and devices; apparatus for reception, storage and transmission of data and compression thereof. Software as a service, namely, the provision of software for use in relation to data compression; application software as a service, namely, the provision of application software for use in relation to data compression; research and development services in the field of data compression; industrial analysis and research services, namely, the research and development of computer hardware and computer software which compresses data; development and implementation of software and technological solutions in respect of data compression; consultancy, information and advice in relation to the aforesaid services.
51.
IMAGE COMPRESSION AND DECODING, VIDEO COMPRESSION AND DECODING: METHODS AND SYSTEMS
There is disclosed a computer-implemented method for lossy image or video compression, transmission and decoding, the method including the steps of: (i) receiving an input image at a first computer system; (ϋ) encoding the input image using a first trained neural network, using the first computer system, to produce a latent representation; (iii) quantizing the latent representation using the first computer system to produce a quantized latent; (iv) entropy encoding the quantized latent into a bitstream, using the first computer system; (v) transmitting the bitstream to a second computer system; (vi) the second computer system entropy decoding the bitstream to produce the quantized latent; (vii) the second computer system using a second trained neural network to produce an output image from the quantized latent, wherein the output image is an approximation of the input image. Related computer-implemented methods, systems, computer-implemented training methods and computer program products are disclosed.
G06N 3/04 - Architecture, e.g. interconnection topology
H04N 19/46 - Embedding additional information in the video signal during the compression process
H04N 19/503 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
H04N 19/91 - Entropy coding, e.g. variable length coding [VLC] or arithmetic coding
52.
A SYSTEM AND METHOD FOR LOSSY IMAGE AND VIDEO COMPRESSION AND/OR TRANSMISSION UTILIZING A METANETWORK OR NEURAL NETWORKS
A system and method for lossy image and video compression that utilizes a metanetwork to generate a set of hyperparameters necessary for an image encoding network to reconstruct the desired image from a given noise image. A system and method for lossy image and video compression and transmission that utilizes a neural network as a function to map a known noise image to a desired or target image, allowing the transfer only of hyperparameters of the function instead of a compressed version of the image itself. This allows the recreation of a high-quality approximation of the desired image by any system receiving the hyperparameters, provided that the receiving system possesses the same noise image and a similar neural network. The amount of data required to transfer an image of a given quality is dramatically reduced versus existing image compression technology. Being that video is simply a series of images, the application of this image compression system and method allows the transfer of video content at rates greater than existing technologies in relation to the same image quality.
H04N 19/90 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using coding techniques not provided for in groups , e.g. fractals
G06N 3/04 - Architecture, e.g. interconnection topology
G06N 3/063 - Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using electronic means
A system and method for lossy image and video compression that utilizes a metanetwork to generate a set of hyperparameters necessary for an image encoding network to reconstruct the desired image from a given noise image.
A system and method for lossy image and video compression and transmission that utilizes a neural network as a function to map a known noise image to a desired or target image, allowing the transfer only of hyperparameters of the function instead of a compressed version of the image itself. This allows the recreation of a high-quality approximation of the desired image by any system receiving the hyperparameters, provided that the receiving system possesses the same noise image and a similar neural network. The amount of data required to transfer an image of a given quality is dramatically reduced versus existing image compression technology. Being that video is simply a series of images, the application of this image compression system and method allows the transfer of video content at rates greater than existing technologies in relation to the same image quality.
G06K 9/36 - Image preprocessing, i.e. processing the image information without deciding about the identity of the image
G06T 5/50 - Image enhancement or restoration using two or more images, e.g. averaging or subtraction
H04N 7/035 - Circuits for the digital non-picture data signal, e.g. for slicing of the data signal, for regeneration of the data-clock signal, for error detection or correction of the data signal
H04N 7/12 - Systems in which the television signal is transmitted via one channel or a plurality of parallel channels, the bandwidth of each channel being less than the bandwidth of the television signal
