Methods and systems are provided for detecting design commands from speech using natural language processing to generate design variations. In embodiments described herein, a user inputs speech indicating a desired design. Parameters of detected design commands mapped to particular design tools of a design application are detected from input speech. Design variations are generated based on the detected parameters of the detected design commands by applying the corresponding design tools. The user inputs subsequent speech indicating desired revisions to the design variations. Different parameters of detected design commands mapped to particular design tools of the design application are detected from the subsequent speech. New design variations are generated based on the detected different parameters by applying the corresponding design tools to the design variations. After the user reviews the new design variations, the design process iteratively continues until the user selects a final design.
The present disclosure relates to systems, methods, and non-transitory computer-readable media that generate a hierarchy of masks for a selected object within a digital image. For example, in some embodiments, the disclosed systems receive a digital image and user input selecting one or more pixels within an object portrayed therein. Using a segmentation neural network, the disclosed systems determine a parent token corresponding to a first semantic level for the object and a child token corresponding to a second semantic level for the object that is hierarchically lower than the first semantic level. The disclosed systems further generate, using the segmentation neural network and from the tokens, a first mask that corresponds to the first semantic level and a second mask that corresponds to the second semantic level. The disclosed systems provide, for display, at least one of the first mask or the second mask.
A method, apparatus, non-transitory computer readable medium, and system for video generation includes obtaining an input prompt describing a scene. A video generation model generates a first frame chunk and a second frame chunk based on the input prompt, wherein the first frame chunk includes a first token corresponding to a video frame and the second frame chunk includes a second token corresponding to the same video frame as the first token. Subsequently, the video generation model generates a synthetic video based on the set of frame tokens including the first frame chunk and the second frame chunk, wherein the synthetic video depicts the scene and includes the video frame.
A query system is described that supports retriever machine-learning model training data generation and implementation. The retriever machine-learning model is configured to retrieve examples for inclusion in a prompt formed from an input query that provide context for the query in performance of a corresponding task. The query system, in one or more examples, implements an alternating minimization technique for example selection that enhances in-context learning (ICL) performance for resources having few initial examples, e.g., for an initial resource. The query system is also configurable to incorporate example diversity that improves generalization and reduces bias in the retriever machine-learning model used to select the examples.
In various examples, a multimodal question answering tool generates multimodal answers to queries. For example, a machine learning model generates a text answer to a query, the text answer snippets are compared to text snippets from a source document to determine attribution. Based on the attribution, multimodal data is obtained from the document and used to generate a prompt that, as a result of being provided to the machine learning model, causes the machine learning model to generate a multimodal answer.
An image processing system auto white balances an image using an object in the image and a reference color distribution. Given an input image, a target object in the input image is identified. A reference color distribution for the object type of the target object from the input image is accessed. One or more image processing settings are determined that, when applied to the input image, minimize a difference in values between pixels of the target object and the reference color distribution. A white balanced image is generated by applying the one or more image processing settings to the input image, and the white balanced image is provided for presentation.
The present disclosure relates to systems, non-transitory computer-readable media, and methods for discovering and mitigating robustness gaps in semantic parsing models utilizing deep learning. For example, in one or more embodiments, the disclosed systems extract, utilizing a natural language processing model, predicates from a set of semantic parsing examples comprising natural language queries and corresponding structural representations. Further, the disclosed systems determine clusters of semantic parsing examples according to similarities between respective predicates from the semantic parsing examples. Moreover, the disclosed systems generate, utilizing a target semantic parsing model, predicted structural representations of the natural language queries and determine respective correctness values for the clusters based on comparing the predicted structural representations and the corresponding structural representations. In some embodiments, the disclosed systems identify robustness gaps in the target semantic parsing model based on the respective correctness values and implement mitigating actions to address the identified gaps.
This disclosure describes one or more implementations of systems, non-transitory computer-readable media, and methods that utilize a large language model as a base model to generate and utilize digital page navigation predictions with a variety of downstream user activity prediction tasks. Indeed, in one or more instances, the disclosed systems utilize the digital page navigation predictions from the large language model to derive multiple downstream predictive user activity tasks. In particular, in one or more implementations, the disclosed systems utilize the digital page navigation predictions with a variety of downstream user activity prediction models to generate user activity predictions for a user associated with the digital page navigation predictions. Moreover, in one or more implementations, the disclosed systems train a large language model to predict page sequences using a page order agnostic and contrastive measure of loss from training input-output page sequence pairs.
The present disclosure relates to systems, non-transitory computer-readable media, and methods for generating layered digital design documents from sketches utilizing deep learning. In particular, the disclosed systems determine at least one of a canvas size or an aspect ratio of a sketch image. Additionally, the disclosed systems generate, based on the at least one of the canvas size or the aspect ratio, a compositional reference of the sketch image utilizing a binarization model by normalizing lighting of the sketch image, enhancing contrast of the sketch image, and converting the sketch image to a binary format. Further, the disclosed systems generate, utilizing one or more machine learning models, a digitized digital design of the sketch image based on the compositional reference. Moreover, the disclosed systems generating a layered digital design document comprising a background layer based on the digitized digital design and editable text elements based on the compositional reference.
A method, apparatus, non-transitory computer readable medium, and system for video generation includes obtaining an input prompt describing a scene. A video generation model generates a global token based on the input prompt by performing an attention process based on a plurality of frame tokens. The video generation model generates a frame token by performing an attention process based on the global token and a subset of the plurality of frame tokens in a local window. Subsequently, the video generation model generates a synthetic video based on the frame token, wherein the synthetic video depicts the scene and includes an image frame corresponding to the frame token.
The present disclosure relates to systems, methods, and non-transitory computer-readable media that modifies parameters of a generative model based on enhancing pixels of images. Furthermore, the disclosed systems generate a modified digital image from a digital image by inpainting a region of the digital image. Moreover, the disclosed systems generate a first measure of loss based on comparing the modified digital image with a ground truth version of the digital image. Further, the disclosed systems generate a transformed modified digital image and a transformed ground truth image of the digital image by performing a color space transformation and further generates a second measure of loss. From the first measure of loss and the second measure of loss, the disclosed systems modify parameters of a generative model.
Context-based prompt generation techniques for generative machine-learning models are described. In one or more examples, a user interface (UI) selection of a UI element of first digital content is received. A processing device captures contextual information associated with the UI selection. A machine-learning model generates a prompt based on the UI selection and the contextual information. Generative artificial intelligence (AI) generates second digital content based on the prompt. The generative AI is implemented using one or more machine-learning models. The processing device then presents the second digital content for display in a user interface.
Techniques for machine-learning material data generation are described. In an example, a processing device is operable to receive a digital image depicting a material and generate a set of video frames depicting physically-based rendering properties of the material. The set of video frames is generated based on the digital image using a video generative machine-learning model. The processing device is operable to output material data based on the physically-based rendering properties of the material depicted by the set of video frames.
Methods, computer systems, computer storage media, and graphical user interfaces are provided for facilitating content generation using target keywords for performance optimization. In one implementation, a set of search result rankings for a target website and one or more competitors of the target website in association with a set of keywords are obtained. Thereafter, candidate performant keywords are identified based on a comparison of search result rankings for the target website and search result rankings for the one or more competitors. The candidate performant keywords, or a portion thereof, are used to generated content data in association with the target website via one or more generative artificial intelligence (AI) models. Such content data can be presented via a display and/or incorporated into the target website.
G06F 3/0481 - Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
G06F 16/2457 - Query processing with adaptation to user needs
A method, apparatus, non-transitory computer readable medium, and system for data processing includes receiving a document and a query requesting information about the document, identifying, using a language generation model, a relevant page of the document based on the query, and generating, using the language generation model, a response to the query based on the relevant page, wherein the response incudes the requested information about the document.
G06F 16/383 - Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
Machine-learning prompt enhancement techniques are described. In one or more examples, by generating training data from high-quality digital content meeting specific criteria, a generative artificial intelligence (AI) system is configurable to train an enhancement machine-learning model to capture features pertaining to particular tasks or scenarios. The trained enhancement machine-learning model can then extract enhancement features from inputs, enabling the formation of enhanced prompts that guide generative AI models to generate digital content as suitable for the particular tasks or scenarios.
Embodiments are disclosed for multi-color light-diffuser devices capable of displaying different colors based on stack configurations of polymer dispersed liquid crystal (PDLC) diffuser components, base layers, and filter layers. In one embodiment, a multi-color light-diffuser device can include a base layer positioned underneath a filter layer and a PDLC diffuser component positioned above the filter layer. In another embodiment, a multi-color light-diffuser can include an additional PDLC diffuser component in between the base layer and filter layer. The filter layer can be a dichroic material, a neutral density filter, or other transparent or semi-transparent material. The base layer can be an opaque material, such as paper, holographic film, iridescent film, or a mirror. Based on the activation of the light-scattering or non-light-scattering states of the PDLC diffuser components, the multi-color light-diffuser device can express different visual outputs based on the interactions/mixing of the filter layer and base layer.
An image processing system is configured to receive a three-dimensional (3D) model and a text prompt that describes a scene corresponding to the 3D model. The system may then generate a depth map of the 3D model and generate an output image based on the depth map and the text prompt. The output image may depicts a view of the scene that includes textures described by the text prompt. The output image may be generated using an image generation model.
A method, apparatus, non-transitory computer readable medium, and system for data processing include obtaining a question and a plurality of candidate answers to the question, generating an intermediate response based on the question and the plurality of candidate answers, where the intermediate response indicates that a first candidate answer of the plurality of candidate answers is invalid, and generating a response to the question based on the intermediate response, where the response indicates a second candidate answer of the plurality of candidate answers.
In implementation of techniques for generating brushstrokes with gravity-based fluid flow, a computing device implements a fluid flow system to receive an input stroke on a virtual canvas. The fluid flow system determines a gravity feature involving simulated fluid interaction with the virtual canvas using an algorithm based on a geometry of the virtual canvas. Based on the gravity feature, the fluid flow system generates a brushstroke based on a shape of the input stroke and that simulates fluid interaction on the virtual canvas. The fluid flow system then presents the brushstroke on the virtual canvas in a user interface.
In accordance with the described techniques, a query is received from a user for processing by an artificial intelligence (AI) chat agent. An ambiguity classifier model is leveraged to determine whether the query is ambiguous. Responsive to determining that the query is ambiguous, a query rewrite model is leveraged to update the query to resolve one or more ambiguities of the query, and the updated query is provided to the AI chat agent for processing. Responsive to determining that the query is unambiguous, the query is provided to the AI chat agent for processing.
Some aspects relate to technologies providing a framework for generating and optimizing digital forms. In accordance with some aspects, a digital form is presented to a user, and the interaction of the user with the digital form is monitored to obtain form usage information. The form usage information is then analyzed to determine problems with the form. From this analysis, a form hypothesis is generated using a first set of large language model agents and, based on this form hypothesis, a revised form is generated using a second set of large language model agents. This revised form is then presented to the user.
In implementation of techniques for generating a re-illuminated reconstruction of an object, a computing device implements a reconstruction system to receive digital images depicting an object from different angles and a selection of an illumination direction for virtually illuminating the object. The reconstruction system determines a geometry of the object using a machine learning model based on the digital images. Based on the geometry of the object, the reconstruction system determines illumination parameters corresponding to the illumination direction. The reconstruction system renders a reconstructed virtual object that is a virtual representation of the object based on the geometry and illuminated based on the illumination parameters.
In implementations of techniques and systems for independent alternating-input (“IAI”) devices, a control system includes multiple IAI devices and a logic circuit component. The logic circuit component includes, for example, an analog switch, a d-type latch, a shift register, and an LED driver. The control system provides a first voltage signal (e.g., a bus signal) to a first voltage input of each IAI device. In response to a control signal from the control system, the logic circuit component provides a second voltage signal to a second voltage input of each IAI device. The second voltage signal can include the bus signal or an inverted bus signal. Each IAI device is activated in response to a voltage differential across the voltage inputs. In this way, the quantity of IAI devices for large arrangements (e.g., high-resolution display systems) can be greatly increased by utilizing low-cost logic circuits.
G09G 3/20 - Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes for presentation of an assembly of a number of characters, e.g. a page, by composing the assembly by combination of individual elements arranged in a matrix
G09G 3/32 - Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes for presentation of an assembly of a number of characters, e.g. a page, by composing the assembly by combination of individual elements arranged in a matrix using controlled light sources using electroluminescent panels semiconductive, e.g. using light-emitting diodes [LED]
26.
GENERATING DESIGN LAYOUTS WITH DIFFERENT ASPECT RATIOS USING A CUSTOM GENERATIVE TRANSFORMER MODEL
The present disclosure relates to systems, non-transitory computer-readable media, and methods for generating modified digital designs in a target aspect ratio. In some embodiments, the disclosed systems utilize a transformer neural network to encode, from a digital design in an initial aspect ratio depicting one or more design elements, an input feature representation including tokenizations of the one or more design elements, an initial aspect ratio token, and a target aspect ratio token. In some embodiments, the disclosed systems utilize a transformer neural network to transform the input feature representation into an output feature representation including a modified tokenization of the one or more design elements according to the target aspect ratio. In some embodiments, the disclosed systems generate a modified digital design depicting one or more design elements according to the target aspect ratio utilizing the transformer neural network based on the output feature representation.
The present disclosure relates to systems, non-transitory computer-readable media, and methods for generating and utilizing equidistant angular guides to align a modified object relative to multiple other objects in a digital image. For example, the disclose systems identify a set of snappable segments within a graphical user interface of a digital illustration application. In some embodiments, the disclosed systems detect a user interaction for modifying a first object. The disclosed systems determine a modified alignment position for the first object that positions the first object along a non-vertical and non-horizontal angular alignment guide such that the first segment, second segment, and third segment are separated by equal distances along the angular alignment guide. In various embodiments, the disclosed systems additionally provide a snappable graphical user interface element in the graphical user interface that, upon selection, causes the first object to move into the modified alignment position.
In identifying the hero section for digital content generation, a processing device receives a user prompt to generate digital content. The user prompt generally indicates one or more objectives for the digital content. A machine-learning model identifies a hero section in a template for generating the digital content. The hero section is identified based on multiple features associated with one or more images or textual elements in the template. Images features considered for hero images include the number of elements above the candidate image, the size of the elements above the candidate image, image dimensions, vertical positioning, or aspect ratio. Text features considered for candidate textual elements include the display level, ordering, size, container level, relative size, or a size of textual elements above the candidate textual element. The machine-learning model then generates the digital content based on the user prompt with the hero section directed to the objectives.
G06V 10/764 - Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
A method, apparatus, and non-transitory computer readable medium for obtaining an input image comprising a plurality of pixels. A machine learning model generates annotation information indicating whether each of the plurality of pixels is synthetically generated. A combined image is generated based on the annotation information. In some cases, the combined image shows a synthetically generated region of the input image.
A high dynamic range editing system is configured to generate visualizations to aide digital image editing in both high dynamic ranges and standard dynamic ranges. In a first example, the visualization is generated as a histogram. In a second example, the visualization is generated to indicate high dynamic range capabilities. In a third example, the visualization is generated to indicate ranges of luminance values within a digital image. In a fourth example, the visualization is generated as a point curve that defines a mapping between detected luminance values from a digital image and output luminance values over both a standard dynamic range and a high dynamic range. In a fifth example, the visualization is generated as a preview to convert pixels from the digital image in a high dynamic range into a standard dynamic range.
Database query translation and training data generation techniques are described. In one or more example, by forming a source database and database query based on a database schema, a query annotation system establishes a foundation for accurate query processing. A natural language query is generated from the database query using machine-learning models, which is then output for editing in a user interface. By generating the database query from the database schema itself and the natural language query from the database query, the query annotation system is configured to readily adapt to database changes.
The present disclosure relates to systems, methods, and non-transitory computer-readable media that reduce the color palette of a vector artwork while retaining one or more color values of the original palette. For example, in some embodiments, the disclosed systems extract, from a vector artwork displayed within a graphical user interface of a client device, a color palette that includes a set of color values. Additionally, the disclosed systems receive, via the client device, user input for reducing the color palette of the vector artwork. In response to the user input, the disclosed systems determine a subset of color values that includes one or more color values from the set of color values. The disclosed systems further modify the vector artwork within the graphical user interface of the client device by reducing the set of color values within the vector artwork to the subset of color values.
G06F 3/04845 - Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range for image manipulation, e.g. dragging, rotation, expansion or change of colour
G06T 7/90 - Determination of colour characteristics
G06T 11/60 - Editing figures and textCombining figures or text
G06V 10/74 - Image or video pattern matchingProximity measures in feature spaces
G06V 10/762 - Arrangements for image or video recognition or understanding using pattern recognition or machine learning using clustering, e.g. of similar faces in social networks
33.
CRAFTING LAYERED DIGITAL DESIGN DOCUMENTS FROM RASTERIZED IMAGES
The present disclosure relates to systems, methods, and non-transitory computer-readable media that generates a layered digital design document from a reference image. In particular, the disclosed systems generate a design plan architecture by extracting design elements from a reference image. Furthermore, the disclosed systems generate a layered digital design document from the reference image by extracting the design elements from the reference image according to the design plan architecture and further constructing the layered digital design document from the extracted design elements. Moreover, the disclosed systems provide, to a graphical user interface of a client device, the layered digital design document.
Digital content coediting techniques are described, including detecting a change in state of digital content that is maintained as part of a coediting session involving a plurality of client devices. One or more elements of the digital content are located corresponding to the change in state and a fingerprint is generated responsive to the detecting, which is based on a hash of a serialization of the one or more elements. The fingerprint is communicated for receipt by at least one of the plurality of client devices, the fingerprint configured to cause local synchronization of the digital content.
Causal discovery is enhanced using large language models (LLMs). A causal graphical representation, such as a partial ancestral graph, can be generated from a dataset. An LLM is prompted to identify a confounder variable for the causal graphical representation. The confounder variable may be proposed by the LLM so that it has a direct relationship with two variables of the causal graphical representation initially having a dependent relationship identified from a bidirectional edge in the graph. The LLM is also prompted to provide a value for the confounder variable. The relationship strength between the confounder variable and another variable in the causal graphical representation is learned by parameter optimization using the value.
Embodiments are disclosed for music generation. The method may include receiving a text input describing music to be generated by a neural network and obtaining an initial noise laten. The neural network generates a music spectrogram based on the initial noise latent and the text input. One or more features are extracted from the music spectrogram. A loss is determined based on the one or more features from the music spectrogram and one or more target features of a target output. An optimized noise latent is obtained based on the loss and a new music spectrogram is generated using the optimized noise latent.
A method, apparatus, non-transitory computer readable medium, and system for data processing includes training a machine learning model to predict target time series data based on candidate time series data, identifying a first pair including a source indicator and the target metric, a second pair including an intermediate indicator and the target metric, and a third pair including the source indicator and the intermediate indicator, identifying a first, second, and third time horizon, computing a first, second, and third predictivity value based on the machine learning model, where the first predictivity value corresponds to the first pair and the first time horizon, the second predictivity value corresponds to the second pair and the second time horizon, and the third predictivity value corresponds to the third pair and the third time horizon, and displaying a portion of the candidate time series data based on the first, second, and third predictivity value.
A method, apparatus, non-transitory computer readable medium, and system for obtaining a query comprising a document request. A language generation model is configured to retrieve a first document based on the query and generate an expanded query based on the first document using a knowledge graph that connects the first document to a second document. A third document is retrieved based on the expanded query in response to the query.
Digital content coediting techniques are described, including detecting a change in state of digital content that is maintained as part of a coediting session involving a plurality of client devices. One or more elements of the digital content are located corresponding to the change in state and a fingerprint is generated responsive to the detecting, which is based on a hash of a serialization of the one or more elements. The fingerprint is communicated for receipt by at least one of the plurality of client devices, the fingerprint configured to cause local synchronization of the digital content.
The present disclosure relates to systems, non-transitory computer readable media, and methods for generating a set of suggested fonts matching the font of a text region of a digital image. In some embodiments, the disclosed systems generate an image embedding vector from a portion of a digital image including digital text utilizing a classifier neural network. In some embodiments, the disclosed systems generate a set of one or more embedding vectors from a set of one or more unlearned fonts based on stylized glyphs according to the set of one or more unlearned fonts by utilizing the classifier neural network. In some embodiments, the disclosed systems determine one or more suggested fonts from the set of one or more unlearned fonts based on one or more similarity scores comparing the image embedding vector to the one or more font embedding vectors and displaying the one or more suggested fonts.
G06F 40/109 - Font handlingTemporal or kinetic typography
G06T 11/60 - Editing figures and textCombining figures or text
G06V 30/18 - Extraction of features or characteristics of the image
G06V 30/244 - Division of the character sequences into groups prior to recognitionSelection of dictionaries using graphical properties, e.g. alphabet type or font
41.
SKIPPING LAYERS IN LARGE LANGUAGE MODELS UTILIZING LAYER-SPECIFIC ROUTERS WITH LOW RANK ADAPTERS
The present disclosure relates to systems, non-transitory computer-readable media, and methods for generating a response to a query using a large language model by skipping a query-specific set of layers of the large language model. In particular, the disclosed systems generate, using a large language model and in response to a query to the large language model, a first token in a prefill phase for an input sequence from the query. Additionally, the disclosed systems determine, during the prefill phase and utilizing a layer-specific router corresponding to a layer of the large language model, a probability value indicating to skip the layer of the large language model. Further, the disclosed systems generate, using the large language model, a second token in a subsequent phase by skipping the layer of the large language model based on the probability value.
Change detection and updates using probabilistic data structures are described. In one or more examples, a change is detected to a dataset record that is used as a basis to generate a first sketch as a probabilistic data structure. A second sketch is generated as a probabilistic data structure based on the change to the dataset record. The first sketch is replaced with the second sketch as stored in a database, the database supporting a probabilistic result to a query operation.
The present disclosure relates to systems, non-transitory computer-readable media, and methods for detecting generated images prompted with artist names or generated by artist-customized image generation models. In particular, in some embodiments, the disclosed systems determine, by at least one processor, a digital image generated by an image generation neural network. In addition, in some embodiments, the disclosed systems process the digital image utilizing an artist prompt prediction neural network trained to detect synthetic images generated from artist prompts comprising indications of specific artists. Moreover, in some embodiments, the disclosed systems generate, utilizing the artist prompt prediction neural network, a prompt type label for the digital image indicating whether the digital image was generated from an artist prompt comprising an indication of a specific artist.
The present disclosure relates to systems, methods, and non-transitory computer-readable media that modify digital images via scene-based editing to synthesize shadows for object(s). For instance, in one or more embodiments, the disclosed systems receive a digital image depicting a scene. The disclosed systems access an object mask of the object depicting in the digital image. The disclosed systems further combine the object mask, the digital image, and a noise representation to generate a combined representation. Moreover, the disclosed systems generate a shadow for the object from the combined representation and further generates the modified digital image by combining the shadow with the digital image.
G06T 11/60 - Editing figures and textCombining figures or text
G06F 3/0481 - Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
G06F 3/04845 - Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range for image manipulation, e.g. dragging, rotation, expansion or change of colour
The present disclosure relates to systems, non-transitory computer-readable media, and methods for training a large language model to generate coherent extractive summaries. In some embodiments, the disclosed systems generate an initial extractive summary of a digital document utilizing a large language model. In some embodiments, the disclosed systems generate a feedback set including a set of annotations indicating corrections to the initial extractive summary relative to one or more coherent summaries of the digital document and a set of quality scores for the initial extractive summary. In some embodiments, the disclosed systems adjust parameters of the large language model to reduce differences between the initial extractive summary and the one or more coherent summaries based on the feedback set.
Embodiments are disclosed for using a progressive growing variational autoencoder to boost temporal compression. The method may include receiving a request to compress an input video. The method further includes providing the input video to a progressive encoder. The progressive encoder includes a top pipeline and a bottom pipeline. The method further includes generating, by the progressive encoder, a temporally compressed representation of the input video using a first latent space representation determined by the top pipeline and a second latent space representation determined by the bottom pipeline.
H04N 19/31 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability in the temporal domain
H04N 19/436 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation using parallelised computational arrangements
The technology is directed towards receiving training data regarding a set of observations. Each observation includes a feature set, a treatment, and an outcome. A first generation of machine learning models is trained, via the training data, to predict an outcome for a feature set of a given observation. A new generation of models is generated by selecting a subset of models from the trained first generation of models based on a fitness criteria of each model to generate an intermediate layer for use in predicting a treatment. An algorithm is applied to the selected subset of models to generate the new generation of models. Transformed training data is generated using the training data and a model of the new generation of models. The transformed training data includes, for each observation, a transformed feature set comprising a representation of the feature set in a latent space of the model.
Techniques for generation of an interactive visualization are described that support analysis and dynamic display of various aspects of an entity journey. In an example, a processing device receives journey data that describes various interactions, behaviors, and/or properties of entities related to an entity journey. The processing device further receives a construction input to define an initial structure for the interactive visualization. Based on the journey data and the construction input, the processing device generates the interactive visualization for output, such as in a user interface. The interactive visualization includes various nodes that represent attributes of the entity journey that are connected by directed edges. The interactive visualization further includes visual representations of quantitative transitions between adjacent nodes. The processing device is further operable to receive a variety of interactions to update the interactive visualization in real time.
G06Q 30/0201 - Market modellingMarket analysisCollecting market data
G06F 3/04845 - Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range for image manipulation, e.g. dragging, rotation, expansion or change of colour
49.
GENERATING DOCUMENT-GROUNDED TRAINING DATA FOR GENERATIVE MODELS
The present disclosure relates to systems, non-transitory computer-readable media, and methods for generating a training dataset for AI agents by using large language models to simulate a conversation between a user and an AI agent. In some embodiments, the disclosed systems determine a synthetic persona by selecting a plurality of characteristics defining the synthetic persona. In some embodiments, the disclosed systems generate a synthetic prompt emulating text input by the synthetic persona utilizing a large language model to process a digital document associated with the synthetic persona. In some embodiments, the disclosed systems generate a synthetic response emulating text generated by an artificial intelligence agent responsive to the text input by the synthetic persona utilizing a second large language model to process the synthetic prompt. In some embodiments, the disclosed systems modify parameters of a neural network using the synthetic prompt and the synthetic response as training data.
Data enrichment and identity translation techniques using probabilistic data structures are described. In one or more examples, a plurality of datasets are received from a plurality of entities. Each dataset has a plurality of dataset records describing a respective audience. A plurality of sets of sketches are generated as probabilistic data structures, respectively, based on the plurality of datasets. A result is formed by processing a query. The result includes at least one sketch having a probabilistic data structure generated based on one or more of the plurality of sets of sketches. An entity is identified from the plurality of entities corresponding to the at least one sketch and entity is exposed for display in a user interface.
Techniques for multimodal interactive visual representation generation are described. In an example, a processing device receives a user query that includes semantic parameters that define a context of a scene. The processing device generates a subset of digital assets by correlating one or more digital assets stored in a database to the semantic parameters. The processing device generates a prompt based on the semantic parameters that includes instructions for a machine learning model to generate a visual representation based on the query and the subset of digital assets. The machine learning model processes the prompt and the subset of digital assets to generate a visual representation that depicts the subset of digital assets integrated into the scene specified by the query. The processing device is further operable to receive an interaction to the visual representation and generate an updated visual representation based on the interaction.
Differential privacy techniques using probabilistic data structures are described. In one or more examples, a dataset record is received and a sketch is generated as a probabilistic data structure by applying noise to the dataset record. The sketch is communicated to be stored in a database that supports a probabilistic result to a query operation.
In at least one implementation of systems for frame interpolation of motion graphics, a computing device implements a graphics module to receive inputs including a motion effect and a description of a digital artwork that includes at least one object. The graphics module determines a path for the motion effect. Based on the motion effect, the graphics module generates an animation sequence of the at least one object in the digital artwork across multiple frames along the path. The graphics module then outputs a digital animation of the digital artwork via a user interface. The digital animation uses the animation sequence to guide the movement of at least one object in multiple frames.
G06T 3/18 - Image warping, e.g. rearranging pixels individually
G06T 7/73 - Determining position or orientation of objects or cameras using feature-based methods
54.
FACILITATING NATURAL LANGUAGE TO STRUCTURED QUERY LANGUAGE QUERIES USING STRING-BASED ENTITY LINKING AND SEMANTIC-BASED ENTITY LINKING TO ENRICH A DATABASE
Methods and systems are provided for facilitating natural language to structured query language queries using string-based entity linking and semantic-based entity linking to enrich a database. In embodiments described herein, a user inputs a natural language query (NLQ) of a structured query language (SQL) database. A set of candidate entities are determined from entities of the database by applying (1) a string match from n-gram-based query candidates of the NLQ to string perturbations of the entities and (2) a semantic match from embeddings of n-gram-based query candidates to embeddings generated based on textual variations and classifications of the entities. A ranked set of candidate entities is determined from the set of candidate entities by a classifier trained to determine whether the candidate entity is relevant to the NLQ. A subset of the ranked set of candidate entities above a threshold score is displayed in response to the NLQ.
Methods, systems, and non-transitory computer readable storage media are disclosed for generating a lens blur effect in a digital image with in-focus edge rendering. The disclosed system generates a focal matte indicating an in-focus range of depth values of a digital image based on a focus region and a depth map of the digital image. The disclosed system generates a layered depth map comprising foreground depth values and background depth values of pixels across the digital image according to the depth map and the focal matte. The disclosed system also renders the digital image to include a lens blur effect by utilizing the focal matte and the layered depth map to determine a combination of the foreground depth values and the background depth values in connection with a splatting operation.
The present disclosure is directed toward systems, methods, and non-transitory computer readable media that generate relational vector portals and cause a vector object traversing an entry portal to exit out of a linked exit portal by performing spatial and/or style transitions on the vector object. In particular, the disclosed systems generate an entry portal aligned with a vector path. Furthermore, the disclosed systems generate an exit portal linked to the entry portal. Moreover, the disclosed systems move a vector object to at least partially traverse the entry portal. In addition, the disclosed systems generate a transformed vector object by causing a portion of the vector object traversing the entry portal to exit out of the exit portal.
A method, apparatus, non-transitory computer readable medium, and system for generating a synthetic image includes obtaining an attribute prompt that indicates a level of an attribute. An image generation prior model is configured to scale an attribute token based on the attribute prompt to obtain a scaled attribute token representing the level of the attribute and generate a condition embedding based on the scaled attribute token, wherein the condition embedding represent the level of the attribute. Subsequently, an image generation model generates a synthetic image based on the condition embedding, wherein the synthetic image depicts the attribute at the level of the attribute indicated by the attribute prompt.
A method, apparatus, non-transitory computer readable medium, and system for performing real-time image segmentation includes obtaining an image depicting a first object and a second object. Embodiments then generate, using an object detection model, a first location and a second location corresponding to the first object and the second object, respectively. The location may be, for example, a bounding box circumscribing the object or a point defining a centroid of the object. Embodiments then segment, using a segmentation model, the image based on the first location and the second location to obtain a first mask corresponding to the first object and a second mask corresponding to the second object.
A method, apparatus, non-transitory computer readable medium, and system for generating synthetic asset includes obtaining an input prompt corresponding to a target element. An intent model is configured to generate an asset generation intent based on the input prompt, wherein the asset generation intent indicates the target element. Subsequently, a language model generates an asset generation prompt based on the asset generation intent, wherein the asset generation prompt describes the target element. An image generation model is used to generate a synthetic asset depicting the target element based on the asset generation prompt.
The present disclosure relates to systems, non-transitory computer-readable media, and methods for detecting and imitating edges in digital images using an advanced corner correction algorithm. For example, the disclosed systems extract, from a digital image, a set of reference segments by segmenting a reference edge depicted in the digital image. In some embodiments, the disclosed systems determine a corner within the reference edge by classifying a reference edge segment among the set of reference edge segments as a corner segment. In certain embodiments, the disclosed systems also generate, for display, an imitation edge following a shape of the reference edge and including an imitation corner positioned according to the corner within the reference edge.
G06V 10/764 - Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
61.
USER INTERFACE FOR ADJUSTING IMAGE EFFECTS DEPICTED IN IMAGES
Disclosed embodiments may provide techniques for adjusting image effects using an image-effect control interface. A method includes providing an image-effect control interface for adjusting an image effect (e.g., a reflection effect, a flare effect) depicted in an input image. The image-effect control interface includes: (i) an effect-range user-interface element that displays a range of the image effect to be applied to the input image; and (ii) a control user-interface element configured to adjust an intensity of the image effect of the input image. The method can also include detecting user interactions that moves the control user-interface element in different directions away from the initial position and reaching updated positions of the effect-range user-interface element. The method can also include reducing or enhancing the image effect from the input image according to the updated positions of the control user-interface element.
G06F 3/04845 - Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range for image manipulation, e.g. dragging, rotation, expansion or change of colour
G06F 3/04847 - Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
H04N 5/262 - Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects
62.
METHODS AND SYSTEMS FOR GENERATIVE VIDEO PROPAGATION
A media generation system propagates editing process through frames of a media segment. The media generation system receives a sequence of frames. The media generation system also receives a first subset of frames that correspond to edited representations of one or more frames of the sequence of frames. The media generation system uses a selective content encoder to identify features of the first subset of frames that are unmodified by the editing process. The media generation system then executes a image-to-video model using the sequence of frames and the features of the first subset of frames. The image-to-video model propagates the edits applied to the one or more frames to the frames of the sequence of frames. The media generation system then outputs an edited representation of the sequence of frames.
In implementation of techniques for offsetting camera filter shift, a computing device implements an offset system to capture a first digital image using a filter at a first position relative to an image capture device and to capture a second digital image using the filter at a second position relative to the image capture device resulting from movement of the filter between the first position and the second position. The offset system determines a filter shift resulting from the movement by comparing the first and second digital images. The offset system then controls an offset of a portion of the image capture device based on the filter shift.
The present disclosure relates to systems, methods, and non-transitory computer-readable media that distribute item-based digital content across digital platforms using trend setting participants of those digital platforms. For instance, in one or more embodiments, the disclosed systems generate affinity metrics for digital items from a catalog of digital items with respect to a plurality of trend setting participants of a plurality of digital platforms using attributes of digital posts by the plurality of trend setting participants on the plurality of digital platforms and corresponding attributes of the digital items. The disclosed systems further determine predicted demand metrics for the digital items on the plurality of digital platforms using the affinity metrics. Using the predicted demand metrics, the disclosed systems distribute digital content related to the digital items for display on a plurality of client devices via the plurality of digital platforms.
The present disclosure relates to systems, non-transitory computer-readable media, and methods for inpainting digital images utilizing mask-robust machine-learning models. In particular, in one or more embodiments, the disclosed systems obtain an initial mask for an object depicted in a digital image. Additionally, in some embodiments, the disclosed systems generate, utilizing a mask-robust inpainting machine-learning model, an inpainted image from the digital image and the initial mask. Moreover, in some implementations, the disclosed systems generate a relaxed mask that expands the initial mask. Furthermore, in some embodiments, the disclosed systems generate a modified image by compositing the inpainted image and the digital image utilizing the relaxed mask.
A method, apparatus, non-transitory computer readable medium, and system for generating synthetic output includes obtaining an input image depicting a scene and an input prompt indicating an intrinsic modality of the input image, wherein the intrinsic modality determines how the scene interacts with light. A conditional image encoder encodes the input image to obtain a condition embedding representing the intrinsic modality of the input image. An image generation model generates a synthetic output based on the input prompt and the condition embedding, wherein the synthetic output comprises a visual representation of the intrinsic modality of the input image.
A method, apparatus, non-transitory computer readable medium, and system for controllable image synthesis using image elements include obtaining an image depicting a scene and encoding, using an encoder of an image generation model, a first region of the image to obtain a first encoded image element. A transformation is applied to the first encoded image element to obtain a transformed image element, where the transformation modifies an object in the scene located withing the first region of the image. A decoder of the image generation model generates an edited image depicting the scene with the modified object based on the transformed image element.
A method, apparatus, non-transitory computer readable medium, and system for performing an accelerated denoising process includes obtaining a noise map from a noise distribution. Embodiments then compute, using an image generation model, a denoising vector based on an accelerated denoising trajectory, where the accelerated denoising trajectory accelerates a denoising rate based on a diffusion timestep. Then, embodiments generate, using the image generation model, a synthetic image by denoising the noise map based on the denoising vector.
Embodiments are disclosed for context retrieval for document question answering. The method may include obtaining a plurality of digital documents and dividing the plurality of digital documents into a plurality of document chunks. A similarity between each of the plurality of document chunks is determined. A multi-document graph is constructed based on the similarity between each of the plurality of document chunks. The multi-document graph includes a plurality of nodes representing the plurality of document chunks.
G06F 16/383 - Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
Methods, computer systems, computer storage media, and graphical user interfaces are provided for facilitating identification of relevant data using data embeddings. In one implementation, a query embedding representing a query is generated. Using the query embedding, a data embedding representing data in a hyperspace that is similar to the query embedding is identified. Thereafter, the data, represented by the data embedding identified to be similar to the query embedding, is identified as relevant to the query. Content may then be generated via one or more generative artificial intelligence (AI) models based on at least a portion of the query and the data identified as relevant to the query. Such content may be displayed via a graphical user interface.
Methods, computer systems, computer storage media, and graphical user interfaces are provided for facilitating identification of audience insights based on persona representations using AI. In one implementation, a persona representation associated with a target audience is identified for a campaign asset. Thereafter, via one or more generative artificial intelligence (AI) models, an audience insight is determined in relation to the campaign asset based on the persona representation associated with the target audience. The audience insight may be displayed in relation to the campaign asset.
Some aspects relate to technologies for dynamically generating digital content for events using event data and content intent descriptors. In some aspects, when a content server identifies an event for digital content creation, the content server provides data to a user device that is based on event data for the event and a content intent descriptor. The user device generates a prompt using the received data and provides the prompt to a generative model on the user device, causing the generative model to generate a digital content item using the prompt as input. The user device then presents the digital content item.
A method, apparatus, non-transitory computer readable medium, and system for image generation include obtaining a first prompt and a second prompt, where the first prompt describes an object in a first scene and the second prompt describes the object in a second scene different from the first scene. A first attention output and a second attention output are generated, using an image generation model, by performing a cross-image attention process on the first prompt and the second prompt. The image generation model generates a first synthetic image and a second synthetic image based on the first attention output and the second attention output, respectively, wherein the first synthetic image depicts the object in the first scene and the second synthetic image depicts the object in the second scene.
In implementation of techniques for determining roughness and index of refraction of a material, a computing device implements a specular system to receive images captured of a material sample from multiple polarization angles. The specular system determines information describing light distribution and brightness of the material sample depicted in the images based on the multiple polarization angles. Based on the light distribution and the brightness of the material sample, the specular system determines a roughness property or an index of refraction property for the material sample. The specular system generates an output that visually identifies the roughness property or the index of refraction property of the material sample.
Disclosed embodiments may provide techniques for generating a differentiable mesh representation of an image object. A computer-implemented method can include accessing a set of points, in which each point is assigned with a weighted value. The method can also include receiving ground-truth input to generate a differentiable mesh representation associated a target object. The method can also include constructing an initial mesh representation of the target object by generating a plurality of simplices based on the weighted values of the points. The method can also include optimizing the initial mesh representation by iteratively adjusting the existence probability values based on a comparison between the initial mesh representation with the ground-truth input. The method can also include extracting the differentiable mesh representation of the target object by selecting a set of simplices of the plurality of simplices associated with the initial mesh representation.
Disclosed embodiments may provide techniques for adjusting color parameters using an integrated color-control interface. A computer-implemented method includes generating an integrated color-control interface for a first color parameter and a second color parameter. In some instances, the integrated color-control interface includes a visual-feedback indicator that conveys color distribution of pixels of an image. The method also includes receiving a gesture-based interaction within the integrated color-control interface. The gesture-based interaction includes using one or more gestures to adjust the first color parameter, the second color parameter, or both. The method also includes transforming the image in accordance with the adjusted first and second color parameters. The method also includes updating the integrated color-control interface by modifying the visual-feedback indicator, in which the visual-feedback indicator is modified to convey changes to the color distribution of the pixels as a result of performing the gesture-based interaction.
G06F 3/04845 - Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range for image manipulation, e.g. dragging, rotation, expansion or change of colour
G06F 3/04847 - Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
G06T 5/90 - Dynamic range modification of images or parts thereof
A method, apparatus, non-transitory computer readable medium, and system for image processing include obtaining an input image and a modification input, wherein the input image depicts an object and the modification input indicates a change to the object. A feature map is generated, and the feature map represents the object based on the input image. The feature map is transformed to obtain a transformed feature map based on the modification input. The transformed feature map represents the change to the object. A synthetic image is generated, using an image generation model, based on the input image and the transformed feature map. The synthetic image depicts the change to the object.
In implementing three-dimensional (3D) super-resolution techniques using generative video models, a processing device receives a first 3D representation of an object. The processing device generates an intermediate video of the object from multiple viewpoints of the first 3D representation. A machine-learning model then generates an upsampled video from the intermediate video. The upsampled video is in a higher resolution than the intermediate video. In one example, the machine learning model is a video-based generative upsampler. Based on 3D reconstruction of the object from the upsampled video, the processing device outputs a second 3D representation of the object in a resolution higher than the first 3D representation.
G06T 3/4053 - Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
Embodiments are disclosed for preserving a vocal identity of an input audio sequence when translating speech audio. The method may include receiving an input audio sequence that includes speech audio having a source vocal identity in a first language. The method may further comprise translating a first transcription of the speech audio to a second transcription in a second language. Using the second transcription, initial translated speech audio having a default vocal identity is generated. The method may further comprise processing the initial translated speech audio to generate a translated content embedding and translated intonation data and processing the input audio sequence to generate a source speaker embedding representing the source vocal identity of the speech audio. The method may further comprise generating final translated speech audio with the source vocal identity in the second language, using the source speaker embedding, the translated content embedding, and the translated intonation data.
A method, apparatus, non-transitory computer readable medium, and system for data classification include obtaining input data and a policy graph. The policy graph includes a decision node indicating a machine learning classifier. Embodiments then generate, using the machine learning classifier, a classification result based on the input data and the decision node. Subsequently, embodiments generate a decision label for the input data based on the classification result.
In implementations of rapid and parallelized manufacturing of modular light-diffuser devices, a method includes scoring busbar outlines on each side of a polymer dispersed liquid crystal (PDLC) film sheet for multiple petals of a modular light-diffuser system. Magnets used to prepare the PDLC film sheet for scoring are simultaneously placed and removed using a magnet lift system. A busbar on each sheet side is then exposed for multiple petals, with easy-peel tabs being utilized to simplify exposing the tabs on each petal. Liquid crystal is efficiently wiped away from the tabs and busbars using a hand tool or sponge machine attachment. The petal shapes are cut out to finish the process, with laser welding of the film sheet to a backing along the petal edges to create an air gap. One implementation eliminates the busbar peeling and cleaning steps by lining the film sheet with metal strips.
An animation system receives a curve network is bound to a rest pose of an animated object. The animation system receives an initial keyframe and a destination keyframe of the curve network. The initial keyframe represents a corresponding starting pose of the animated object and the destination keyframe represents a corresponding destination pose of the animated object. The animation system generates a sequence of intermediate curve networks representing poses of the object between the initial keyframe and the destination keyframe. The animation system generates the sequence of intermediate curve networks by interpolating from the initial keyframe to the destination keyframe using the curve network. The animation system then renders the initial keyframe, the sequence of intermediate curve networks, and the destination keyframe to generate an animation of the object from the initial keyframe to the destination keyframe.
Embodiments are disclosed for automated bulk document capture. The method may include receiving an input video comprising a plurality of frames. The input video depicts a plurality of document pages to be captured. A first machine learning model is used to determine a page turn event has been depicted in the input video based at least on a first frame of the input video. A second machine learning model is used to determine that a first frame of the input video is ready for capture. An image of a document page depicted in the first frame is then captured.
In some embodiments, a computing system accesses multiple input images of a specular object with a scene. The computing system encodes near-field interreflections of the scene on the specular object to obtain a first set of feature representations of the specular object in multiple viewing directions based on the multiple input images. The computing system encodes far-field reflections of the scene on the specular object to obtain a second set of feature representations in the multiple viewing directions based on the multiple input images. The computing system determines a set of specular color values for the specular object in the multiple viewing directions based on the first set of feature representations and the second set of feature representations using a multi-layer perceptron algorithm. The computing system renders the specular object representation at least based on the set of specular color values using a neural rendering algorithm.
Disclosed embodiments may provide techniques for performing database operations using a hierarchy-enhanced B-tree index. A computer-implemented method can include executing a query to access a data item by traversing a root node of a hierarchy-enhanced B-tree index to a particular leaf node connected to the root node. In some instances, the particular leaf node indicates a first unstructured-data record. The method can also include determining that the particular leaf node includes a sequential data structure. The sequential data structure identifies a hierarchical relationship between the first unstructured-data record and other unstructured-data records indicated by one or more other leaf nodes. The method can also include using the sequential data structure to access the one or more other leaf nodes without additional path traversals of the hierarchy-enhanced B-tree index. The method can also include retrieving the data item by parsing the other unstructured-data records.
Embodiments are disclosed for novel view synthesis using hybrid rendering. The method may include receiving a request to generate a novel view of a scene, the request including a plurality of input views and a target view. A subset of the plurality of input views is identified based on a similarity to the target view. The novel view is generated using the subset of the plurality of input views. The novel view is then rendered.
learned from digital content items and their corresponding performance metrics. In accordance with some aspects, a training dataset is accessed that comprises training samples that each include a digital content item and a performance metric, and the multimodal generative model is trained using the training data. The training can include, for a training sample, using encoders of the multimodal generative model to generate a latent representation of a digital content item from the training sample and a latent representation of a performance metric from the training sample. The latent representations are merged to provide a combined latent representation, and decoders of the multimodal generative model decode the combined latent representation to provide an output digital content item and output performance metric. Losses are determined from the outputs and used to update parameters of the multimodal generative model.
Some aspects relate to technologies for performance-guided content generation and exploration using a multimodal generative model with a joint latent space learned from digital content items and their corresponding performance metrics. In accordance with some aspects, input is received for content generation. The input includes a digital content item and is encoded by one or more encoders of the multimodal generative model into a latent representation in the joint latent space. A latent space transformation from the latent representation of the input is performed to provide a transformed latent representation, which is decoded by one or more decoders of the multimodal generative model to generate an output digital content item. In some aspects, the one or more decoders also decode the transformed latent representation to generate a predicted performance metric for the output digital content item.
Some aspects relate to technologies for generating custom digital content using a content intent descriptor from a content server and on-device contextual data maintained on a user device. In some aspects, a user device receives a content intent descriptor communicated over a network from a content server. The user device generates a prompt using the content intent descriptor and on-device contextual data maintained on the user device. A generative model is caused to generate a digital content item using the prompt, and the digital content item is presented on the user device.
Some aspects relate to technologies for generating and/or presenting digital content using on-device subscription data maintained on a user device. In some aspects, a user device receives a content intent descriptor communicated over a network from a content server. The user device performs a comparison of on-device subscription data with the content intent descriptor. Based on the comparison, the user device generates a prompt using the content intent descriptor. A generative model is caused to generate a digital content item using the prompt, and the digital content item is presented on the user device.
A method, apparatus, non-transitory computer readable medium, and system for generating images based on a target prompt and an anchor prompt include obtaining the target prompt the anchor prompt. The target prompt describes a first element, and the anchor prompt describes a second element. A first attention block of an image generation model generates a first attention output based on the target prompt and a second attention block of the image generation model generates a second attention output based on the anchor prompt. The image generation model then generates a synthetic image that depicts the first element and excludes the second element by combining the first attention output and the second attention output
A method, apparatus, non-transitory computer readable medium, and system for image generation include obtaining a concept input, a source image, and an input mask, where the concept input represents a concept, the source image depicts a scene, and the input mask indicates a location for the concept in the scene. Concept features are generated by performing a style transfer from the source image to the concept input based on the input mask. A synthetic image is generated, using an image generation model, based on the concept features. The synthetic image depicts the concept from the concept input within the scene from the source image at the location indicated by the input mask.
Systems and methods for image processing include obtaining a guidance condition representing an image element and computing a conditioned tensor based the guidance condition and an unconditioned tensor independent of the guidance condition. An adaptive guidance strength is computed based on the guidance condition and the conditioned tensor and the unconditioned tensor are combined based on the adaptive guidance strength to obtain a scoring tensor. An image generation model generates a synthetic image depicting the image element based on the scoring tensor.
Disclosed embodiments may provide techniques for modifying hue, saturation, and luminance (HSL) parameters across different colors depicted on an image. A computer-implemented method can include providing a set of color sub-spectrums extracted from an image. The method can also include displaying, on a graphical user interface, an interactive graph for a particular color sub-spectrum of the set of color sub-spectrums. The method can also include selecting one or more colors of the particular color sub-spectrum in response to a first interaction relative to a first dimension of the interactive graph. The method can also modifying HSL parameters associated with the one or more colors in response to a second interaction relative to a second dimension of the interactive graph. The method can also include generating a color-adjusted image by applying the modified HSL parameters to one or more pixels of the image.
G06F 3/04845 - Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range for image manipulation, e.g. dragging, rotation, expansion or change of colour
Embodiments are disclosed for generating a diverse training dataset for training encoders to map music and natural language text to a joint embedding space. The method may include obtaining a training audio sequence and descriptive tags associated with the training audio sequence. The disclosed systems and methods further comprise generating a plurality of different subsets of the descriptive tags. The disclosed systems and methods further comprise generating, by a large language model, a plurality of training captions describing the training audio sequence, where each training caption is generated using one of the plurality of different subsets of the descriptive tags. The disclosed systems and methods further comprise generating a plurality of negative training captions for the training audio sequence by modifying elements of the plurality of training captions. The plurality of training captions and the plurality of negative training captions can then be combined to create a training dataset.
A method, apparatus, non-transitory computer readable medium, and system for generating synthetic image includes obtaining a prompt indicating an image element. In some cases, a base generation model generates a first score function based on the prompt and an auxiliary image generation model generates a second score function based on the prompt. Additionally, the first score function and the second score function are combined to obtain a combined score function. In some cases, the combined score function includes positive guidance from the first score function and negative guidance from the second score function. A synthetic image that depicts the image element is generated based on the combined score function.
In some embodiments, a computing system receives an input image of a target in a first view. The computing system creates a single-view feature representation of the target using a trained single-view reconstruction model based on the input image. The computing system generates a multi-view feature representation of the target using a pre-trained generative model based on the single-view feature representation. The computing system determines a 3-dimensional (3D) representation of the target based on the multi-view feature representation using a neural volume rendering algorithm. The computing system generates one or more output images of the target in one or more views based on the 3D representation of the target.
Font replacement techniques based on layout are described. In one or more examples, a document is received including a font and font metadata, the font metadata indicating a layout of text using the font in the document. The font is detected as unavailable and one or more fonts are selected from a plurality of fonts based on the layout of the text indicated by the font metadata. The one or more fonts are presented in a user interface to replace the font as used for the text in the document.
A method, apparatus, non-transitory computer readable medium, and system for generating synthetic image includes obtaining an input prompt describing a first element and a second element. In some cases, an image generation model generates an intermediate output based on the input prompt and optimizes the intermediate output based on an attention contrast loss to obtain an optimized intermediate output. For example, the optimized intermediate output represents the first element at a first location and the second element at a second location. The image generation model generates a synthetic image based on the optimized intermediate output. The synthetic image depicts the first element at the first location and the second element at the second location.
Automatic layout generation is described. In one or more examples, an input including one or more visual elements, an indication of a type of a document for generation, and a size of the document are received. Based on the type of the document and the size of the document, a layout for the one or more visual elements on the document is determined using a machine learning model. One or more coordinates of one or more bounding boxes, respectively, are determined for placement of the one or more visual elements in the layout on the document using the machine learning model. The document is then generated by incorporating the one or more visual elements into the one or more bounding boxes in the layout for presentation in a user interface.
