An online system includes an interface which facilitates communication between customers and pickers who are servicing the user's order. The customer may request a modification to their order through the interface. The online system performs an inference task in conjunction with the model serving system or the interface system to continuously monitor conversations between users and pickers to infer whether a customer requested to modify their order to maintain an updated order and an updated in-store transaction estimate for the order. The online system determines if the order has been updated to account for the requested changes. If the order has not been updated, the online system automatically updates the customer's order and computes an updated in-store transaction estimate based on the changes made.
An agentic framework for solving open ended queries, a first artificial intelligence (AI) agent prompts a first large language model (LLM) to determine tasks based in part on a query received from a user device. The tasks are provided to a second AI agent, which includes one or more LLMs. The second AI agent determines results for schemes using a process. The process includes prompting, by the second AI agent, an LLM of the LLMs to determine a scheme based in part on the tasks. The scheme describes application programming interface (API) calls that are associated with at least one of the tasks. The process includes executing the API calls of the scheme to obtain a result that includes at least one item recommendation. The process is repeated until a successful result of a scheme satisfies conditions and includes item recommendations. The item recommendations are provided to the user device.
An online system uses generative artificial intelligence models to implement a voice-powered search engine. Upon receiving and processing a user's input command, the online system prompts a generative model to derive a user's intent in the form of a set of structures fields, where each structured field is associated with a corresponding search type. After that, the online system prompt a set of generative models to extract entities (or values), where each entity populates a corresponding structured field and is related to a corresponding search type. The online system then maps the set of structures fields and the extracted entities into a search query, which is used to identify, from an item database, a list of items. The online system generates, using information about the list of items, a user interface signal causing a user's device to display a user interface with the list of items for user's conversion.
An online system receives orders from users and fulfills the orders by presenting batches to pickers, who select one or more batches to fulfill. A picker views an interface generated for a batch via a picker client device to review attributes of the batch when determining whether to select the batch. However, a picker client device often has a limited display area for the interface, and different attributes of a batch have varying amounts of relevance to different pickers when determining whether to select the batch. To tailor an interface for a batch to a specific picker, the online system uses a trained batch outcome prediction model to determine probabilities of different attributes of a batch causing an attrition event for the picker selecting the batch when presented. The online system selects one or more attributes of the batch to present in the interface based on the probabilities.
An online system applies OCR techniques and a large language model to object images to more effectively extract text from those images. An online system applies an OCR model to images of physical objects and uses a multi-modal large language model to refine the text extraction from the object image. More specifically, the online system generates a text extraction prompt for the large language model that includes the object image, the text extracted using the OCR model, and instructions to generate a response with the text that is depicted in the object image. The online system receives a response from the LLM with the predicted text and stores the text from the large language model in an entry in a database. The online system may use this stored text to present content to a user related to the object in the image.
G06V 10/77 - Processing image or video features in feature spacesArrangements for image or video recognition or understanding using pattern recognition or machine learning using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]Blind source separation
G06V 10/82 - Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
G06V 20/62 - Text, e.g. of license plates, overlay texts or captions on TV images
A system performs inference in conjunction with a machine-learned language model to predict a state of a receptacle that is located at a physical location and has sensors attached to the receptacle. The predicted state of the receptacle includes a predicted list of objects that are in a storage area of the receptacle, for example. The system may generate a first predicted state of the receptacle from receptacle sensor data of the sensors. A rule-based model may be used to generate the first predicted state, for example. The machine-learned language model may be used in conjunction with the rule-based model to also predict the state of the receptacle. The machine-learned language model better understands complex user behavior compared to the rule-based model resulting in increased accuracy of the predictions.
A system determines whether an item is an acceptable force mark with an ordered item. The system receives an order that includes an item and a set of acceptable scan codes associated with the item. The system receives a selected a second item as fulfillment of the ordered item. The system identifies whether a second scan code of the second item is included in the set of acceptable scan codes associated with the ordered item. In response to identifying that the second scan code is not included in the set of acceptable scan codes, the system applies a multi-modal model to determine whether the second item is interchangeable with the ordered item. In response to receiving a response from the multi-modal model that the second item is interchangeable with the ordered item, the system provides an indication that the second item is approved for fulfillment.
The online system suggests replacement items to a user when an ordered item is unavailable. To improve the replacement suggestions for a given user, the online system obtains a set of generalized parameters for the user, where a generalized parameter provides guidance for replacing multiple different types of items. When the online system is fulfilling an order and identifies that an item in the order is unavailable, the online system uses a machine-learning model to score various candidate replacement items. The candidate replacement items can be obtained by prompting a generative model (e.g., language model) with one or more generalized parameters for the user that are relevant to the item. The generative model can be further prompted to adjust the scores of the candidate replacement items. Based on the adjusted scores, the online system selects a replacement item, and then presents the selected replacement item to the user.
An online system adjusts a guardrail setting used by a user treatment engine based on conditions faced by the online system. The online system simulates the performance of the user treatment engine using different candidate guardrail settings and computes a score for each of the guardrail settings based on the performance of the user treatment engine using each of the guardrail settings. The online system selects a new guardrail setting for the user treatment engine based on the performance scores for the candidate guardrail settings. Furthermore, the online system generates simulated training examples to initially train a user treatment engine. The online system uses a treatment performance model to simulate the effect of treatments applied to users and generates simulated training examples based on the predicted effect of the treatments. The online system retrains the user treatment engine on real training examples that are generated based on actual treatments.
G16H 50/50 - ICT specially adapted for medical diagnosis, medical simulation or medical data miningICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for simulation or modelling of medical disorders
An online concierge system generates a personalized storefront user interface to recommend items for purchase and delivery to a customer. The online concierge system obtains a user identifier for the customer and generates a set of recommended search terms that it predicts will be relevant to the customer. The recommended search terms may be identified at least in part by mapping items previously purchased by the customer to search queries that resulted in purchases of that item across a population of customers of the online concierge system. The online concierge system then executes respective search queries for the each of the set of search terms to generate respective search result sets for each of the recommended search terms. The search result sets may be presented as respective search queries on a user interface screen of a customer client device.
Disclosed herein relates to a system, comprising: at least one load receiver mounted on a shopping cart or basket and configured to receive an item placed into the shopping cart or basket for a weighing operation; a plurality of sensors configured to detect a plurality of parameters relating to the weighing operation of the item including at least one of: a relative angle between a force sensing axis of the at least one load receiver and a direction of gravity, a motion of the shopping cart or basket, and an ambient temperature surrounding the shopping cart or basket and the at least one load receiver; and a processor configured to determine an actual weight of the item based on at least a portion of the plurality of parameters.
B62B 5/00 - Accessories or details specially adapted for hand carts
B62B 3/14 - Hand carts having more than one axis carrying transport wheelsSteering devices thereforEquipment therefor characterised by provisions for nesting or stacking, e.g. shopping trolleys
12.
BOOSTING SCORES FOR RANKING ITEMS MATCHING A SEARCH QUERY
An online system receives a search query from a client device associated with a user and queries a database including item data for a set of items matching the query, in which the set of items is at a retailer location associated with a retailer type and each item is associated with an item category. For each item of the set, a machine learning model is applied to predict a probability of conversion for the user and item and a score is computed based on an expected value, in which the expected value is based on a value associated with the item and the probability. The score for each item is boosted based on the item category, retailer type, or a user segment that is based on the user’s historical order data. The items are ranked based on the boosted scores and the ranking is sent to the client device.
An online concierge system receives a search query from a client device. The online concierge system identifies a set of matching items from an item database. The matching items correspond to the received search query. The online concierge system obtains, from a hierarchical item taxonomy, a category label for each matching item. The item taxonomy relates each item in the item database to one of a plurality of category labels. The online concierge system groups the matching items by the category labels for each of the matching items into one or more groups. The online concierge system generates instructions for a user interface. The user interface includes a scrollable list of one or more carousels. Each carousel includes a scrollable list of a group of the one or more groups. The online concierge system sends the instructions of the user interface to the client device for display.
A user interface of an online system is generated based on search for relevant items that match ingredients of a recipe. After receiving, from a device associated with a user of the online system, a query for an ingredient of a recipe, the online system identifies, based on one or more attributes in the query, a set of candidate items for the ingredient. The online system generates a recipe relevance score for each candidate item by applying a weighted sum of scores, ranks the identified candidate items based on their recipe relevance scores, and selects one or more items for presentation to the user. The online system then generates a user interface of the device with a recipe page including the ingredient of the recipe and the one or more items that match the ingredient of the recipe.
An online system facilitates various functions using machine learning model microservices. A tuning mechanism tunes various configuration parameters for each microservice that control allocation of computing resources and other configurations of physical and/or virtual machines that implement the microservices. Tuning may be performed in part by executing tests under various configurations and evaluating an objective function associated with the different configurations. Furthermore, parameters of the objective function may be set based on a trained learning model that learns baseline parameters and weights of the objective function based on historical data.
An automated checkout system accesses an image of an item inside a shopping cart and a location of the shopping cart within a store. The automated checkout system identifies a set of candidate items located within a threshold distance of the location of the shopping cart based on an item map. The item map describes a location of each item within the store and the location of each candidate item corresponds to a location of the candidate item on the item map. The automated checkout system inputs visual features of the item extracted from the image to a machine-learning model to identify the item by determining a similarity score between the item and each candidate item of the set of candidate items. After identifying the item, the automated checkout system displays a list comprising the item and additional items within the shopping cart to a user.
An online system predicts the likelihood of user interactions with new items with little or no historic performance data by leveraging historic performance data of similar items. The online system identifies similar items by comparing item embeddings for new items and other, established items of the online system. Based on the historic performance data of the similar items, the online system predicts a baseline interaction rate for new items, which may be used in various ranking or selection processes for presenting the cold start items to users. As the online system gathers user interaction data for the cold start items, the new user interaction data may be blended with the predicted baseline interaction rate until sufficient data is collected.
An online system receives a request from a client device associated with a user to access a user interface for placing an order and retrieves user data for the user, in which the user data describe a first set of actions in a first domain performed by the user. The system accesses and applies a machine-learning model to predict a score for each of multiple candidate restaurants based on the user data and restaurant data for the candidate restaurant, in which the score indicates a likelihood of a second set of actions in a second domain performed by the user if presented with a recommendation associated with a restaurant. The system selects a set of restaurants from the candidate restaurants based on the scores, generates the user interface including a set of recommendations associated with the set of restaurants, and sends the user interface to the client device.
A hybrid artificial intelligence (AI) agent balances use of a plurality of predefined decision tree flows and one or more large language model (LLM) chatbots. A query is received from a user client device that is associated with a user. A routing LLM of the hybrid AI agent determines an intent of the user based in part on context associated with the query. The routing LLM selects a response model from the plurality of predefined decision tree flows and the one or more LLM chatbots based in part on the intent. The selected response model is used to determine a response to the query. The response is provided to the user client device.
G06F 16/383 - Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
09 - Scientific and electric apparatus and instruments
42 - Scientific, technological and industrial services, research and design
Goods & Services
Downloadable computer software for purchasing consumer products and services in the fields of online retail store services, personal shopping, and delivery and transportation of goods; Downloadable computer software for predictive shopping in the fields of online retail store services, personal shopping, and delivery and transportation of goods; Downloadable computer software for providing information about sales, deals and savings from favorite stores of a user in the fields of online retail store services, personal shopping, and delivery and transportation of goods; Downloadable computer software for making and sharing shopping lists to organize products and helping the consumer transact with third party retailers in the fields of online retail store services, personal shopping, and delivery and transportation of goods; Downloadable computer software for prioritizing sales, deals and savings based on preferred shopping categories of a user in the fields of online retail store services, personal shopping, and delivery and transportation of goods; Downloadable computer software for a virtual shopping assistant in the fields of online retail store services, personal shopping, and delivery and transportation of goods; Downloadable computer software for finding local sales and purchasing consumer goods from third parties in the fields of online retail store services, personal shopping, and delivery and transportation of goods; Downloadable computer software for identifying and selecting a credit card for purchasing consumer products and services in the fields of online retail store services, personal shopping, and delivery and transportation of goods; Downloadable computer software for shopping in the fields of online retail store services, personal shopping, and delivery and transportation of goods Providing temporary use of on-line non-downloadable software for purchasing consumer products and services in the fields of online retail store services, personal shopping, and delivery and transportation of goods; Providing temporary use of on-line non-downloadable software for predictive shopping in the fields of online retail store services, personal shopping, and delivery and transportation of goods; Providing temporary use of on-line non-downloadable software for providing information about sales, deals and savings from favorite stores of a user in the fields of online retail store services, personal shopping, and delivery and transportation of goods; Providing temporary use of on-line non-downloadable software for making and sharing shopping lists to organize products and helping the consumer transact with third party retailers in the fields of online retail store services, personal shopping, and delivery and transportation of goods; Providing temporary use of on-line non-downloadable software for prioritizing sales, deals and savings based on preferred shopping categories of a user in the fields of online retail store services, personal shopping, and delivery and transportation of goods; Providing temporary use of on-line non-downloadable software for a virtual shopping assistant in the fields of online retail store services, personal shopping, and delivery and transportation of goods; Providing temporary use of on-line non-downloadable software for finding local sales and purchasing consumer goods from third parties in the fields of online retail store services, personal shopping, and delivery and transportation of goods; Providing temporary use of on-line non-downloadable software for identifying and selecting a credit card for purchasing consumer products and services in the fields of online retail store services, personal shopping, and delivery and transportation of goods; Providing temporary use of on-line non-downloadable software for shopping in the fields of online retail store services, personal shopping, and delivery and transportation of goods; Software as a service (SAAS) services featuring software for purchasing consumer products and services in the fields of online retail store services, personal shopping, and delivery and transportation of goods; Software as a service (SAAS) services featuring software for predictive shopping in the fields of online retail store services, personal shopping, and delivery and transportation of goods; Software as a service (SAAS) services featuring software for providing information about sales, deals and savings from favorite stores of a user in the fields of online retail store services, personal shopping, and delivery and transportation of goods; Software as a service (SAAS) services featuring software for making and sharing shopping lists to organize products and helping the consumer transact with third party retailers in the fields of online retail store services, personal shopping, and delivery and transportation of goods; Software as a service (SAAS) services featuring software for prioritizing sales, deals and savings based on preferred shopping categories of a user in the fields of online retail store services, personal shopping, and delivery and transportation of goods; Software as a service (SAAS) services featuring software for a virtual shopping assistant in the fields of online retail store services, personal shopping, and delivery and transportation of goods; Software as a service (SAAS) services featuring software for finding local sales and purchasing consumer goods from third parties in the fields of online retail store services, personal shopping, and delivery and transportation of goods; Software as a service (SAAS) services featuring software for identifying and selecting a credit card for purchasing consumer products and services in the fields of online retail store services, personal shopping, and delivery and transportation of goods; Software as a service (SAAS) services featuring software for shopping in the fields of online retail store services, personal shopping, and delivery and transportation of goods
09 - Scientific and electric apparatus and instruments
42 - Scientific, technological and industrial services, research and design
Goods & Services
Downloadable computer software for purchasing consumer products and services in the fields of online retail store services, personal shopping, and delivery and transportation of goods; Downloadable computer software for predictive shopping in the fields of online retail store services, personal shopping, and delivery and transportation of goods; Downloadable computer software for providing information about sales, deals and savings from favorite stores of a user in the fields of online retail store services, personal shopping, and delivery and transportation of goods; Downloadable computer software for making and sharing shopping lists to organize products and helping the consumer transact with third party retailers in the fields of online retail store services, personal shopping, and delivery and transportation of goods; Downloadable computer software for prioritizing sales, deals and savings based on preferred shopping categories of a user in the fields of online retail store services, personal shopping, and delivery and transportation of goods; Downloadable computer software for a virtual shopping assistant in the fields of online retail store services, personal shopping, and delivery and transportation of goods; Downloadable computer software for finding local sales and purchasing consumer goods from third parties in the fields of online retail store services, personal shopping, and delivery and transportation of goods; Downloadable computer software for identifying and selecting a credit card for purchasing consumer products and services in the fields of online retail store services, personal shopping, and delivery and transportation of goods; Downloadable computer software for shopping in the fields of online retail store services, personal shopping, and delivery and transportation of goods Providing temporary use of on-line non-downloadable software for purchasing consumer products and services in the fields of online retail store services, personal shopping, and delivery and transportation of goods; Providing temporary use of on-line non-downloadable software for predictive shopping in the fields of online retail store services, personal shopping, and delivery and transportation of goods; Providing temporary use of on-line non-downloadable software for providing information about sales, deals and savings from favorite stores of a user in the fields of online retail store services, personal shopping, and delivery and transportation of goods; Providing temporary use of on-line non-downloadable software for making and sharing shopping lists to organize products and helping the consumer transact with third party retailers in the fields of online retail store services, personal shopping, and delivery and transportation of goods; Providing temporary use of on-line non-downloadable software for prioritizing sales, deals and savings based on preferred shopping categories of a user in the fields of online retail store services, personal shopping, and delivery and transportation of goods; Providing temporary use of on-line non-downloadable software for a virtual shopping assistant in the fields of online retail store services, personal shopping, and delivery and transportation of goods; Providing temporary use of on-line non-downloadable software for finding local sales and purchasing consumer goods from third parties in the fields of online retail store services, personal shopping, and delivery and transportation of goods; Providing temporary use of on-line non-downloadable software for identifying and selecting a credit card for purchasing consumer products and services in the fields of online retail store services, personal shopping, and delivery and transportation of goods; Providing temporary use of on-line non-downloadable software for shopping in the fields of online retail store services, personal shopping, and delivery and transportation of goods; Software as a service (SAAS) services featuring software for purchasing consumer products and services in the fields of online retail store services, personal shopping, and delivery and transportation of goods; Software as a service (SAAS) services featuring software for predictive shopping in the fields of online retail store services, personal shopping, and delivery and transportation of goods; Software as a service (SAAS) services featuring software for providing information about sales, deals and savings from favorite stores of a user in the fields of online retail store services, personal shopping, and delivery and transportation of goods; Software as a service (SAAS) services featuring software for making and sharing shopping lists to organize products and helping the consumer transact with third party retailers in the fields of online retail store services, personal shopping, and delivery and transportation of goods; Software as a service (SAAS) services featuring software for prioritizing sales, deals and savings based on preferred shopping categories of a user in the fields of online retail store services, personal shopping, and delivery and transportation of goods; Software as a service (SAAS) services featuring software for a virtual shopping assistant in the fields of online retail store services, personal shopping, and delivery and transportation of goods; Software as a service (SAAS) services featuring software for finding local sales and purchasing consumer goods from third parties in the fields of online retail store services, personal shopping, and delivery and transportation of goods; Software as a service (SAAS) services featuring software for identifying and selecting a credit card for purchasing consumer products and services in the fields of online retail store services, personal shopping, and delivery and transportation of goods; Software as a service (SAAS) services featuring software for shopping in the fields of online retail store services, personal shopping, and delivery and transportation of goods
23.
IMAGE-BASED USER POSE DETECTION FOR USER ACTION PREDICTION
A system may access a first set of images captured by cameras coupled to a shopping cart, wherein each image depicts a user associated with the shopping cart. A system may apply a pose detection model to each of the images to predict a user’s pose. A system may apply an action prediction model to the set of images and the predicted poses to predict whether the user performed an action to change the contents of a storage area of the shopping cart. A system may, responsive to predicting that the user performed a change action, apply an item identification model to a second set of images of a storage area of the shopping cart to identify an item associated with the change action. A system may update an item list of the user based on the change action and the identified item.
An online system applies an ensemble of two or more machine-learning models to accurately predict a likelihood of user interaction with one or more items on a computer system. The online system applies a first machine-learning model to generate user affinity scores describing a likelihood of user interaction with an item. The generated user affinity scores are then fed as input to a second machine-learning model, the second machine-learning model configured to predict a difference of likelihood of user interaction with the item based on a treatment modifying the item. The ensemble of machine-learning models enables the online system to predict the effects of various factors on user interactions with greater granularity.
The methods and systems of the present disclosure may: receive data indicating: one or more items to be purchased by a customer from a retailer associated with an online shopping concierge platform, and one or more items purchased by the customer from the retailer; identify one or more items offered by the retailer for subsequent consideration by the customer; determine, based at least in part on one or more machine learning (ML) models, one or more values for each item representing a likelihood that the customer will purchase the item if subsequently presented with the opportunity under specified conditions; generate data describing a graphical user interface (GUI) comprising a ranked listing of at least a portion of the item(s); and communicate the data describing the GUI such that a computing device associated with the customer renders and displays the listing.
An online concierge system trains a fine-tuned generative image model for distinct categories of items based on a generative image model that takes a textual query as input and outputs and an associated image. Training of the fine-tuned generative image model is additionally based on a small set of representative images associated with the various categories, as well as textual tokens associated with the categories. Once trained, the fine-tuned generative image model can be used to generate realistic representative images for items in a database of the online concierge system that are lacking associated images. The fine-tuned model permits the generation of different variants of an item, such as different quantities or amounts, different packaging or packing density, and the like.
G06F 16/583 - Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
A question is posed by a user of a user device as part of an online chat session with an online system. An intake artificial intelligence (AI) agent interacts with the user via the online chat session in one or more rounds of messaging to gather information that may be used by a human agent to respond to the question. At some point, the online system may identify in an output of the intake AI agent an indication that there is sufficient context regarding the question to transfer the question to the human agent. The online system provides session information (e.g., the question and gathered context) to a user device associated with the human agent. The human agent may use the session information to develop a response to the question that may be provided to the user.
A large language model (LLM) is trained to provide domain specific answers. First, item information is extracted from a catalog of items. A machine-learning model is then prompted to generate a set of queries based in part on the item information associated with the items. Training examples are generated that are associated with the items using a first subset of queries from the set. Each training example is for a corresponding item, and includes a query (that is associated with the corresponding item and is from the first subset) and some item information that is an answer to the query and that is associated with the corresponding item. The LLM is trained using the training examples. Performance of the LLM is evaluated using a second subset of the set of queries that is separate from the first subset.
An online system obtains a representation of a document that identifies items included in the document and information associated with items by the document. The online system generates a target embedding for a target item selected from the representation based on attributes of the target item and corresponding values from the representation. Based on the target embedding and embeddings for items available via the online system, the online system selects a set of candidate items. The online system selects a set of item attributes and values for each candidate item and compares the set of item attributes to attributes of the target item and associated values from the representation. Based on the comparison, the online system selects a candidate item corresponding to the target item, allowing presentation of information associated with the target item by the document in conjunction with the selected candidate item via the online system.
An online system receives a request from a client device associated with a user to place an order and retrieves user data including a constraint associated with the user. Upon receiving a notification that an item included in the order is not available at a source location, the system generates a prompt including item data for the item, a description of the constraint, contextual information associated with the order, and a request to identify a replacement for the item based on information describing a set of candidate replacement items available at the source location, the item data, the constraint, and the contextual information. The system provides the prompt to a large language model to obtain an output and extracts information describing the replacement from the output. The system generates a message including a suggestion to replace the item with the replacement and sends the message to the client device.
An online system maintains a database of items offered by the system, where the items are organized in a catalog by item categories. To generate an image for an item category without biasing the image for the item category image by branded items within the item category, the online system obtains a set of example images of items in the item category. Based on the set of example images, the online system prompts a multimodal large language model (LLM) to generate a generic description of the set of example images representing the item category. The online system prompts an image generative model to generate an example of an item within the category using the generic description from the LLM. The generated image may be evaluated and stored by the online system connection with the item category.
An online system uses a trained machine-learning model to predict a perception of a user about an expiration date of an item. Upon receiving an item signal including information about the expiration date, the online system applies the machine-learning model to the item signal, information about the user, and information about the item to generate a perception score indicative of a likelihood that the user will perceive the expiration date as unacceptable. Based on the perception score and the information about the item, the online system identifies a second item for replacing the item, the second item having a second expiration date that is later than the expiration date. The online system generates, using information about the item and the second item, a user interface signal that causes a user interface to display a notification about the expiration date and a recommendation for replacing the item with the second item.
An online system trains a machine-learning model to generate an embedding in real time for a current session of a user with the online system. The machine-learning model is trained by applying a masked language modeling algorithm to training data including a training sequence of actions and a masked action to predict a user’s action that follows the training sequence of actions. The online system captures current session data describing a sequence of actions of the user performed during the current session. The online system applies the trained machine-learning model to predict a next user’s action and generate a session embedding that encodes information about the sequence of actions and the next action. Using the session embedding, the online system ranks a list of objects. The online system generates a user interface signal causing a user’s device to display a user interface with the ranked list of objects.
An online system leverages a cart issue prediction model trained as a machine-learning model to identify predicted issues with the operation of a smart cart based on sensor data captured by sensors on the smart cart and/or feedback from a user of the smart cart. The machine-learning model is trained on historical data related to the operation of a fleet of smart carts. In response to identifying any cart issues, the online system can trigger a remedial workflow to remedy the predicted issues. The online system may transmit command signals to the smart cart to calibrate the sensors, may transmit remedial tasks to a client device of the user for prompting the user to aid in remedying the predicted issues, or may schedule a service appointment with a technician, including an issue report indicating the predicted issues.
An online system uses a voice augmented language model to create a personalized online order using voice commands from a user of the online system. The online system generates a prompt for input into the language model, the prompt including user's voice content, user's sentiment data, other user data, and source data. The language model uses the prompt to generate a list of components and metadata for each component. Upon receiving an acknowledgement signal indicating an acknowledgement of the list of components by the user, the online system converts the list of components into a list of items and generates one or more options for servicing an order including the list of items. The online system then generates a user interface signal that causes a device associated with the user to display a user interface with the list of items and the one or more options for servicing the order.
An online system uses a trained machine-learning model for personalized rearrangement of items from a physical document in a user interface. The online system obtains an electronic version of the physical document including metadata for each item in the physical document. The online system applies the machine-learning model to the metadata, information about each item, and information about a user to generate an item conversion score for each item that is indicative of the likelihood of the user converting on each item. The online system ranks, using the item conversion score for each item, items from the physical document. Based on the ranking, the online system generates a user interface signal including information about a rearrangement of each item. The user interface signal causes a device associated with the user to display a user interface with each item placed at the user interface in accordance with the rearrangement.
An online system runs experiments to test one or more variants against a control. For example, a variant differently presents content to users compared to a control presentation. To prune variants that are unlikely to test well with users, the online system obtains data for each variant by simulating the variant for real user requests to the system based on historical user interactions. The online system then extracts statistical features about the simulated results for each variant to generate a quality score. Optionally, the online system applies a visual language model (VLM) to a simulated result for a variant when generating the quality score for the variant. The online system may apply a set of rules or a trained model to the extracted statistical features for a variant to determine whether to test the variant or prune it before testing.
A website evaluation system generates a sequence of actions performed on a website based on actions performed by users of the website, in which the sequence of actions corresponds to a task. The system simulates performance of the sequence of actions on the website and receives a set of contextual information associated with the website and a response of the website to the sequence of actions. The system generates a prompt including the set of contextual information, information describing the sequence of actions, information describing the response of the website, and a request to evaluate the response to identify a set of problems with the website based on the set of contextual information and a set of objectives associated with the website. The system provides the prompt to a generative artificial intelligence model to obtain an output, extracts an evaluation of the response from the output, and stores the evaluation.
Embodiments describing automated planning and execution of functions using artificial intelligence (AI) agents are described. Functions may include, e.g., application programming interface (API) calls. A service request from a user device associated with a user is received. A planning AI agent (e.g., one or more machine-learning models) is prompted to generate, using a policy database, a sequence of one or more actions to address the service request. The policy database includes a plurality of policies for responding to various service requests. One or more functions (e.g., API calls) are identified to perform the sequence of one or more actions. An execution AI agent (e.g., one or more machine-learning models) is prompted to execute the one or more functions to resolve the service request. Once the service request has been resolved, the user device may be notified accordingly.
An online system uses a language model to create online orders from online calendar data shared by a user of the online system. The online system generates a first prompt for input into the language model including information about the online calendar and information about the user, and requests the language model to generate a first response that includes a list of consumption activities (e.g., meals). The online system generates a second prompt for input into the language model including the first response, and requests the language model to generate a second response that includes a list of components (e.g., ingredients). The online system generates, using the list of components, a list of items for user's conversion. Based on the list of items, the online system generates a user interface signal that causes a device associated with the user to display a user interface with the list of items.
A computer system uses a generative model to convert data processes of the computer system into workflows. The computer system receives, via a user interface, input data including a definition of a workflow that represents a set of one or more operations performed on data stored in a database of the computer system, and a request for generating the workflow. The computer system generates a prompt for input into the language model, the prompt including the definition of the workflow and the request for generating the workflow. The computer system requests the generative model to generate, based on the prompt input into the generative model, a response that includes a workflow code for executing the workflow. The computer system executes the workflow by deploying the workflow code.
An embedding model is trained to learn latent representations of users describing information related to conditional treatment effect for users relative to different potential treatments. The user embeddings may be used to determine the types of situations in which a user responds differently to different conditions or situations. To train this model, a plurality of experiments with users may be performed to determine user responses to different treatment conditions in the experiments. The conditional treatment effect for users in the experiments may be determined, e.g., with counterfactual predictions of a treatment not experienced by a user in the experiment. The embedding model may be trained with decoders that each predict the conditional treatment effect with respect to one of the experiments, enabling a loss for each experiment with respect to the conditional treatment effect to jointly train the embedding model.
G06Q 10/0637 - Strategic management or analysis, e.g. setting a goal or target of an organisationPlanning actions based on goalsAnalysis or evaluation of effectiveness of goals
G06Q 10/0639 - Performance analysis of employeesPerformance analysis of enterprise or organisation operations
43.
Using Language Model To Automatically Generate List Of Items At An Online System Based on a Constraint
Embodiments relate to using a large language model (LLM) to generate a list of items at an online system with a user defined constraint. The online system receives a query that includes at least one constraint. The online system generates a prompt for input into the LLM, based at least in part on the query. The online system requests the LLM to generate, based on the prompt, a set of constraints for a set of item types. The online system generates a list of candidate items by searching through a set of items stored in one or more non-transitory computer-readable media using the set of constraints for the set of item types. The online system causes a device of the user to display a user interface with the list of items for inclusion into a cart, the list of items obtained from the list of candidate items.
Automatic creation of lists of items at an online system organized around co-occurrences of items. The online system provides inputs into a computer model, the inputs including information about items purchased by a user of the online system over a defined time period, information about a catalog of items stored at one or more computer-readable media of the online system, and a plurality of recipes each including a set of co-occurring items. The online system applies the computer model to generate an indication of co-occurrence of each pair of items in each recipe. The online system generates one or more lists of items based on the indication of co-occurrence, each of the one or more lists of items associated with a respective recipe. The online system causes a device of the user to display a user interface with the one or more lists of items for presentation to the user.
A client device, or an online system communicating with the device, receives video data depicting a field of view of a display area of the device and applies machine-learning algorithms to the video data to detect objects, including portions of a body of a user of the device, within the field of view and to determine a series of body poses. The device/system uses machine-learning models to predict an action performed by the user based on the series of poses and to predict a recipe being prepared based on the objects and a predicted series of actions performed by the user. The device/system selects a suggestion associated with preparing the recipe based on candidate suggestions associated with preparing the recipe, the objects, or the predicted series of actions, and generates an augmented reality element describing the suggestion. The augmented reality element is displayed in the display area of the device.
An online system uses a voice augmented language model to create a personalized online order using voice commands from a user of the online system. The online system generates a prompt for input into the language model, the prompt including user's voice content, user's sentiment data, other user data, and source data. The language model uses the prompt to generate a list of components and metadata for each component. Upon receiving an acknowledgement signal indicating an acknowledgement of the list of components by the user, the online system converts the list of components into a list of items and generates one or more options for servicing an order including the list of items. The online system then generates a user interface signal that causes a device associated with the user to display a user interface with the list of items and the one or more options for servicing the order.
A method for predicting inventory availability, involving receiving a delivery order including a plurality of items and a delivery location, and identifying a warehouse for picking the plurality of items. The method retrieves a machine-learned model that predicts a probability that an item is available at the warehouse. The machine-learned model is trained, using machine learning, based in part on a plurality of datasets. The plurality of datasets include data describing items included in previous delivery orders, whether each item in each previous delivery order was picked, a warehouse associated with each previous delivery order, and a plurality of characteristics associated with each of the items. The method predicts the probability that one of the plurality of items in the delivery order is available at the warehouse, and generates an instruction to a picker based on the probability. An instruction is transmitted to a mobile device of the picker.
A system dynamically maps and updates item locations within a warehouse. A first set of sensors traverses the environment, capturing high-resolution spatial data the system uses to generate an item mapping for the warehouse. The item mapping maps items and structures to precise three-dimensional locations. A cart-mounted camera collects lower-granularity spatial data as it moves through the warehouse. The system identifies items in this new data, associates them with their positions, and compares the updated visual data to the item mapping. In response to detecting changes in item locations, the system updates the item mapping, enabling efficient, real-time tracking of inventory and layout changes with reduced computational resources.
An online system receives, from a computing device associated with a servicing user, an indication that items are collected for an order. The system identifies a runover by determining that a total value of the items collected for the order is greater than an expected value of the order. The system performs runover verification by: identifying features from the order, the items collected, and the runover, applying a remediation time prediction model to the features to predict an amount of time to remediate the runover, applying an uplift model to the features to predict a differential loss between a loss associated with rejecting the runover and a loss associated with verifying the runover, and verifying or rejecting the runover based on the predicted amount of time to remediate and the predicted differential loss. The online system transmits a notification indicating the verification results.
Embodiments are described for leveraging machine-learning models to determine a collection sequence of items of an order. Sequences for collecting a plurality of items of an order are determined, and each of the sequences has a different arrangement of the plurality of items. Total appeasement values are determined for each of the sequences based in part on an appeasement model. Collection times are predicted for each of the sets of sequences. The sequences of the set are scored based in part on the total appeasement values and the collection times. A sequence is selected from the set based in part on the scoring. The selected sequence is provided to a picker client device. A picker associated with the picker client device may fulfill the order and collect the plurality of items in accordance with the selected sequence.
An online system trains a multimodal machine-learning model to predict a rate of using an item that can be ordered at the online system by a user. The machine-learning model is trained by using a plurality of training examples, where each training example includes training images associated with a respective training user that are related to a respective item from the collection of items, and data related to conversion of the respective item by the respective training user. Upon receiving images of user's physical spaces that store items, the online system applies the trained machine-learning model to the images to output a rate of using a specific item by the user. Based on the predicted rate, the online system generates a user interface signal causing a device associated with the user to display a user interface with a user interface element for use by the user to restock the item.
A system for image-based error identification and remediation receives, from a computing device, a request to identify errors with a completed order and image data including one image captured by a camera assembly. The system applies a feature extraction model to the image to identify image features describing an item from the completed order. The system generates a prompt including the image features and instructions to identify any errors with the item from the completed order. The system causes execution of the prompt by a language model trained as a machine-learning model to perform error identification based on image features. The system receives a response generated by the language model indicating errors identified from the image features. The system, responsive to error identification, selects candidate remedial actions to resolve the identified errors. The system performs one remedial action to resolve the errors identified by the language model.
An online system utilizes multi-agent language models for context-aware understanding of a query. The online system receives the query submitted by a user during a user's session at the online system, and stores, during the session, information about the session. The online system generates a prompt for input into the language models, the prompt including the query and the information about the session. Each language model is tuned to infer a respective type of context of the query and generate, based on the prompt, a response including information about the respective type of context. The online system generates, using responses from the language models, a query understanding string with information about types of context of the query. The online system uses the query understanding string to identify a set of items and displays a user interface with items so that the user can order one or more items.
A system receives real-time sensor data from sensors of a smart cart. The system identifies a triggering event based on the sensor data. The system obtains a template for the triggering event, wherein the template comprises instructions for generating suggestions for the user to augment smart cart operation. The system may obtain other contextual information, e.g., order data, user data, source data about a source location, etc. The system generates a prompt by modifying the template to include the sensor data or the contextual information. The system causes execution of the prompt by a language model, which outputs a response based on the prompt. The system generates augmented content including the suggestions for the user by parsing the response. The augmented content may be multimodal, combining multiple forms of data. The system transmits the augmented content for presentation to the user to augment operation of the smart cart.
The present disclosure is directed to determining purchase suggestions for an online shopping concierge platform. In particular, the methods and systems of the present disclosure may receive, from a computing device associated with a customer of an online shopping concierge platform, data indicating one or more interactions of the customer with the online shopping concierge platform; determine, based at least in part on one or more machine learning (ML) models and the data indicating the interaction(s), a likelihood that the customer will purchase a particular item if presented, at a specific time, with a suggestion to purchase the particular item; and generate and communicate data describing a graphical user interface (GUI) comprising at least a portion of a listing of one or more purchase suggestions including the suggestion to purchase the particular item.
An online system performs an inference task in conjunction with the model serving system and/or interface system to generate relevant product images for query auto-completion and query suggestion to help users better navigate their search experience. The online system generates a collection of query suggestions using search query log mining. For each query suggestion in the collection of query suggestions, the online system retrieves one or more catalog images that depict the query suggestion from a product catalog. The online system constructs a prompt to a text-to-image model including the query suggestion, and a request to generate one or more query images based on the query suggestion. The online system receives the query images from the text-to-image model and ranks the catalog and query images to identify an image to display to the user in association with the query suggestion.
A method for selecting a smart shopping cart for update through a series of first and second networks. A method for receiving, at a charging station, a cart update from a remote server through a first network, wherein the cart update is an update to software operating on a set of smart shopping carts. The method receives cart data from the set of smart shopping carts through a second network, wherein the cart data is data describing the set of smart shopping carts, wherein the second network comprises the charging station and the set of smart shopping carts. The method proposed computes an update score for each of the set of smart shopping carts based on the cart data and a set of cart selection parameters. The method selects and transmits an update based on the computed update score.
An online system uses a trained machine-learning model for dynamically modifying an authorization buffer amount to cover additional expenses occurring during fulfillment of an online order. Upon receiving a signal indicating that a user entered an online checkout stage of the order, the online system applies the machine-learning model to generate a set of values of a metric for a set of authorization buffer amounts, each value of the metric resulting from charging the user a respective authorization buffer amount over an expected value of the order if a value of the order at delivery is greater than the expected value. The online system selects an authorization buffer amount resulting in the largest value of the metric, and generates an authorization signal that authorizes charging the user the authorization buffer amount over the expected value if the value of the order is greater than the expected value.
An automated checkout system modifies received images of machine-readable labels to improve the performance of a label detection model that the system uses to decode item identifiers encoded in the machine-readable labels. For example, the automated checkout system may transform subregions of an image of a machine-readable label to adjust for distortions in the image's depiction of the machine-readable label. Similarly, the automated checkout system may identify readable regions within received images of machine-readable labels and apply a label detection model to those readable regions. By modifying received images of machine-readable labels, these techniques improve on existing computer-vision technologies by allowing for the effective decoding of machine-readable labels based on real-world images using relatively clean training data.
G06K 7/14 - Methods or arrangements for sensing record carriers by electromagnetic radiation, e.g. optical sensingMethods or arrangements for sensing record carriers by corpuscular radiation using light without selection of wavelength, e.g. sensing reflected white light
A method for selecting a smart shopping cart for update through a series of first and second networks. A method for receiving, at a charging station, a cart update from a remote server through a first network, wherein the cart update is an update to software operating on a set of smart shopping carts. The method receives cart data from the set of smart shopping carts through a second network, wherein the cart data is data describing the set of smart shopping carts, wherein the second network comprises the charging station and the set of smart shopping carts. The method proposed computes an update score for each of the set of smart shopping carts based on the cart data and a set of cart selection parameters. The method selects and transmits an update based on the computed update score.
H04L 67/00 - Network arrangements or protocols for supporting network services or applications
H04L 67/12 - Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
61.
CLUSTERING OF ITEMS FOR MULTI-SOURCE SERVICING OF AN AGGREGATED LIST OF ITEMS
An online system performs clustering of items for multi-source servicing of an aggregated list of items for a user of the online system. Upon receiving the aggregated list and identifying that the aggregated list is unserviceable by a single source, the online system applies either a trained machine-learning model or the nearest neighbor algorithm to embeddings of items from the aggregated list to cluster items from the aggregated list into multiple clusters, each cluster of items serviced by a single source that is unique for that cluster. The online system generates, using order data for the user and the clusters of items, multiple orders and assigns the orders to sources, where each order includes items from a respective cluster. The online system uses the orders to generate a user interface signal causing a user’s device to display a user interface with information about the orders and the sources.
An online system receives a request from a client device associated with a user to place an order for pickup from a source location during a timeframe and identifies candidate remedial actions associated with the order based on the timeframe and a current time. The system retrieves user data for the user and accesses a machine-learning model. For each candidate remedial action, the system applies the model to predict, based on the user data and order data for the order, a likelihood the user will pick up the order if the candidate remedial action is taken and computes an associated value based on the likelihood. The system selects a remedial action from the candidate remedial actions based on the values, generates, based on the selected remedial action, a message associated with the order that includes a set of selectable options, and sends the message to the client device.
A system trains and deploys a unified embedding model configured to generate embeddings for a set of different entity types based on a natural language description of the entities. The system obtains training data including a plurality of pairs, wherein a pair includes a query entity and a target entity. The system divides the training data into one or more batches for training a transformer embedding model. The system, for each iteration of one or more iterations, applies parameters of the transformer embedding model to generate estimated query entity embeddings for the query entities, and to generate estimated target entity embeddings for the target entities. The system computes corresponding dot products between the estimated query entity embeddings and the estimated target entity embeddings. The system computes a loss function that is proportional to the dot product. The system updates the parameters of the transformer embedding model.
A computing platform may receive, from a user device, historical shopping information indicating previously purchased items and/or previous routes within shopping environments for a first user of the user device. The computing platform may input, into a shopping gamification model, the historical shopping information, which may output shopping recommendation information indicating one or more of: recommended items or recommended routes within a first shopping environment. The computing platform may send, to the user device, a shopping gamification interface that includes the shopping recommendation information and one or more commands directing the user device to display the shopping gamification interface. The computing platform may receive, from the user device, user feedback information indicating acceptance or rejection of the shopping recommendation information by the first user. The computing platform may update, based on the user feedback information, the shopping gamification model.
Using long-term features and short-term features to reduce latency in providing recommendations is described. A user device in a session with an online system may request a recommendation. The online system identifies a set of recommendations based in part on the request. The online system retrieves long-term features for each of the set, and determines short-term features for each of the set. The short-term features are based on the session. The online system applies the long-term features and the short-term features to a scoring model that scores the recommendations of the set. The online system selects a subset of the set based on the scores, and provides the selected subset to a selector that selects a recommendation from the subset. The recommendation may be provided to the user device prior to expiration of a latency period associated with serving the recommendation request.
H04L 41/16 - Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks using machine learning or artificial intelligence
66.
IDENTIFYING AND MODIFYING COMPONENTS OF A PHYSICAL DOCUMENT USING MACHINE-LEARNING MODELS
An online system customizes documents for a particular context, user, or set of users. The online system receives an image of a physical document and extracts components, such as text, titles, items and their metadata, from the physical document. The online system may apply rules to the metadata for one or more items to determine whether to modify at least a portion of the metadata. The online system also applies a model to generate an affinity score for a context or a user and each component of the document. If the score for a component is below a threshold, the online system prompts a generative model to generate replacement content for the component. Subsequently, the online system applies the model to the generated replacement content and updates the document with the generated replacement content for the component if the score of the generated replacement content is higher.
A search module for an online concierge system executes searches in response to a search query with respect to item databases of retailers. The search module dynamically configures a recall set size that controls a number of search results returned for a search query based in part on a query entropy representing an estimated breadth of the search term. The query entropy may be determined relative to a diversity of items in a retailer's database. The recall set size may be configured relative to the query entropy in a manner that manages a tradeoff between latency of search execution and search result quality.
An automated checkout system uses a shopping cart that is automatically charged when stacked into another shopping cart. Each shopping cart has a front charging connector and a rear charging connector. When a first shopping cart is stacked into a second shopping cart, the front charging connector of the first shopping cart connects with the rear charging connector of the second shopping cart. Electrical power can flow to the first shopping cart via the second shopping cart to charge a battery of the first shopping cart. The second shopping cart may be similarly stacked into a third shopping cart, wherein the second shopping cart receives electrical power from the third shopping cart. The second shopping cart may use this electrical power to charge its own battery or may provide some or all of the electrical power to the first shopping cart to charge the first shopping cart's battery.
B62B 3/14 - Hand carts having more than one axis carrying transport wheelsSteering devices thereforEquipment therefor characterised by provisions for nesting or stacking, e.g. shopping trolleys
A47F 10/04 - Furniture or installations specially adapted to particular types of service systems, not otherwise provided for for self-service type systems, e.g. supermarkets for storing or handling self-service hand-carts or baskets
B60L 53/16 - Connectors, e.g. plugs or sockets, specially adapted for charging electric vehicles
B62B 5/00 - Accessories or details specially adapted for hand carts
A language model is used to generate autosuggestions to complete or revise a user's partial search query. An initial partial query is applied to the language model to generate query candidates for completing the search query. The language model may generate the query candidates as additional or alternate tokens for the partial search query. When the user revises the partial query, the previously-generated candidates can be re-used to reduce subsequent processing time for generating additional candidates. The previously-generated candidates are compared with the revised partial query to select which of the candidates to be re-used and expanded for generating additional tokens. Additional tokens can be generated in parallel for the previously-generated candidates or with model values from the previous generation, enabling the tokens to be generated effectively with reduced latency consistent with user expectations for search-related autosuggestions.
An online concierge system receives location information associated with pickers and actual orders associated with a geographical zone. A model trained to predict a likelihood an actual order associated with the zone will be available for servicing within a timeframe is accessed and applied to forecasted orders. Each picker is matched to an order for servicing by minimizing a value of a function that is based on a difference between a location associated with each picker matched to an actual order and an associated retailer location, a difference between the location associated with each picker matched to a forecasted order and an associated retailer location, and the predicted likelihood. Recommendations for accepting an actual order, moving to a retailer location associated with a forecasted order, or checking back later with the system are generated based on the matches and sent for display to a client device associated with each picker.
G06Q 10/087 - Inventory or stock management, e.g. order filling, procurement or balancing against orders
G06Q 30/0202 - Market predictions or forecasting for commercial activities
71.
USING MACHINE-LEARNING MODEL OF AN ONLINE SYSTEM TO GENERATE A SOURCE-RELATED CONFIDENCE SCORE FOR SERVICING A LIST OF COMPONENTS REQUESTED BY AN ONLINE PLATFORM
An online system uses a trained machine-learning model to generate a confidence score for servicing a list of components (e.g., recipe) at a specific source. Upon receiving the list of components from an online platform, the online system identifies a set of candidate items that match each component from the list. The online system further identifies a matching score and a number of matches for each component. The online system then applies the machine-learning model to the matching score, the number of matches, and user’s conversion data to generate a confidence score for the list of components that is indicative of a likelihood that the list of components are located at the source. The online system selects the list of components for the source and sends the list of components and an identification of the source for displaying at a user’s device.
TRAINING MACHINE-LEARNING MODEL OF AN ONLINE SYSTEM TO DETERMINE A LEVEL OF MATCHING BETWEEN IDENTIFIERS OF ITEMS STORED IN A DATABASE OF ONLINE SYSTEM
An online system trains a machine-learning model to identify a level of matching (equivalency or similarity) between item identifiers (e.g., brands) stored in a database. The online system receives search data including information about a search query including a series of identifiers. The online system further receives conversion data including information about a search query in relation to a first identifier that is followed by conversion of an item having a second identifier. The online system further receives communication data exchanged between a servicing agent and a user with information about a message including identifiers of items. The online system generates, based on the search data, the conversion data, and/or the communication data, training data for the machine-learning model. The online system trains, using the training data, the machine-learning model to identify the level of matching between an identifier searched for by a user and a replacement identifier.
An online system receives, at a search interface, a search query from a user. The online system determines a recall set size for search results of the search query. The online system determines a page load size to display at least a portion of the search results by determining a query entropy associated with the search query, inputting a plurality of signals into a machine learning model, the plurality of signals comprising the query entropy, and receiving, from the machine learning model, the page load size. The online system selects a set of physical object identifiers based on the page load size. The online system generates for display a user interface that groups the selected physical object identifiers. The online system causes a device associated with the user to display the generated user interface.
An online system performs an atypical replacement recommendation task in conjunction with a model serving system or the interface system to make recommendations to a user for replacing a target item with an atypical replacement item. The online system receives a search query from a user and identifies a target item based on the search query. The online system identifies a set of candidate items for replacing the target item. The online system may select one or more atypical replacement items in the set of candidate items, and generate an explanation for each atypical replacement item. The explanation provides a reason for using the atypical replacement item to replace the target item. The online system provides the atypical replacement items and the corresponding explanations as a response to the search query.
An online system adjusts a guardrail setting used by a user treatment engine based on conditions faced by the online system. The online system simulates the performance of the user treatment engine using different candidate guardrail settings and computes a score for each of the guardrail settings based on the performance of the user treatment engine using each of the guardrail settings. The online system selects a new guardrail setting for the user treatment engine based on the performance scores for the candidate guardrail settings. Furthermore, the online system generates simulated training examples to initially train a user treatment engine. The online system uses a treatment performance model to simulate the effect of treatments applied to users and generates simulated training examples based on the predicted effect of the treatments. The online system retrains the user treatment engine on real training examples that are generated based on actual treatments.
A smart shopping cart includes internally facing cameras and an integrated scale to identify objects that are placed in the cart. To avoid unnecessary processing of images that are irrelevant, and thereby save battery life, the cart uses the scale to detect when an object is placed in the cart. The cart obtains images from a cache and sends those to an object detection machine learning model. The cart captures and sends a load curve as input to the trained model for object detection. Labeled load data and labeled image data are used by a model training system to train the machine learning model to identify an item when it is added to the shopping cart. The shopping cart also uses weight data and the image data from a timeframe associated with the addition of the item to the cart as inputs.
G01G 19/12 - Weighing apparatus or methods adapted for special purposes not provided for in groups for incorporation in vehicles having electrical weight-sensitive devices
G01G 19/40 - Weighing apparatus or methods adapted for special purposes not provided for in groups with provisions for indicating, recording, or computing price or other quantities dependent on the weight
G06V 10/25 - Determination of region of interest [ROI] or a volume of interest [VOI]
G06V 10/778 - Active pattern-learning, e.g. online learning of image or video features
G06V 10/80 - Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
Use of a language model to automatically perform visual assessment of quality of an item being fulfilled by a picker. The online system receives an image of the item and identifies a set of potential problems associated with the item. The online system generates a plurality of prompts for input into the language model including the image and one or more questions each corresponding to a respective potential problem of the set potential problems. The online system requests the language model to generate, based on the plurality of prompts, a feedback response for each potential problem. The online system generates an aggregated output by aggregating the feedback response for each potential problem, and based on the aggregated output, a second message that identifies one or more relevant problems associated with the item. The online system causes a device of the picker to display the second message.
An artificial intelligence (AI) agent generates responses customized to a user based in part on monitored actions of the user. The AI agent, formed from a machine-learning model, is instantiated with inputs that include a set of objectives, an online catalog of items, and user data associated with a user of an online system. Actions performed by a user on the online system are monitored. Action types of at least some of the monitored actions are determined. Responsive to a determination that an action of the monitored actions has an action type of a set of predetermined types of actions, the AI agent is prompted with a description of the action and a request to suggest a response to the action. The determined response is based in part on one or more of the set of objectives. The response suggested by the AI agent is invoked.
An online system uses a trained machine-learning model to predict timing of a state change of a variable state item in an order. The online system applies a trained machine-learning model to information about the variable state item and information about an ambient condition when servicing the order to predict a timing when a state of the variable state item changes from an original state at a location of a source associated with the online system to a different state. Based on the predicted timing, the online system generates a control signal that initiates at least one of a first action associated with the order or a second action associated with the variable state item. The online system performs, using the control signal, at least one of the first action associated with the order or the second action associated with the variable state item.
A shopping cart includes sensors configured to collect data about a physical interaction of a user with a product in a retail store. A set of features, such as product quality score, interaction duration, sequence of product interaction, and/or whether the product was added to the cart, are extracted from the data. These features are fed into a machine learning model to determine the user's quality preference score, indicating a likelihood that the user would be dissatisfied with the quality of the product. If a user orders online and their quality preference score surpasses a threshold, a notification is sent to the picker fulfilling the order. Furthermore, the user may send in satisfaction feedback via a client device of the user. Such feedback may subsequently be used to retrain the machine learning model.
G06V 10/44 - Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersectionsConnectivity analysis, e.g. of connected components
G06V 20/52 - Surveillance or monitoring of activities, e.g. for recognising suspicious objects
G06V 40/20 - Movements or behaviour, e.g. gesture recognition
81.
Display panel of a programmed computer system with a graphical user interface
An online system leverages a large language model (LLM) to generate a meta-catalog using online catalogs of items that are associated with sources. Items from the online catalogs are clustered. The clustering is based in part on similarity of the items, and each cluster is associated with a different meta-product. The LLM is prompted, based on descriptions of the items, to generate descriptions for meta-products that are associated with the clusters. Entries for the meta-products are generated using the generated descriptions. The meta-catalog for the meta-products is generated using the entries. The meta-catalog is provided to a third party system. A user may interact with the third party system via a user client device to select a meta-product of the meta-catalog for purchase, and the user client device is redirected to the online system to select an item corresponding to the meta-product and complete an order for the item.
An online system enables users to generate an order for items by receiving a collection of components, such as a recipe. The online system maps the components to specific items available at a source. To avoid nonsensical mappings of a specific item to a component, the online system trains a model to predict a probability of a specific item being suitable for inclusion in at least one collection of components. For example, the model generates a probability of a specific item being included in at least one recipe comprising a plurality of components. The model may be trained using users' inclusion of specific items previously selected for one or more groups of items based on collections of components by users of the online system.
An online system receives, at a search interface, a search term from a user. The system retrieves from a mapping table grouping parameters associated with the search term in the mapping table. The association between the grouping parameters and the search term in the mapping table is generated by generating a prompt including the search term and a set of physical objects that match the search term. The prompt requests grouping parameters for the set of physical objects, wherein each grouping parameter specifies a characteristic of the set of physical objects for providing search results responsive to the search term. The system receives, from the LLM, the grouping parameters and updates the mapping table. The system retrieves search results by querying a database of the online system using the search term. The system generates for display a user interface that groups the search results by the retrieved grouping parameters.
A computer-implemented method for suggesting keywords as a search term of a content item includes receiving, from a content provider, information about the content item in a database of content items. The method further includes generating a set of seed keywords related to the content item, and expanding the set of seed keywords to a plurality of candidate keywords. The plurality of candidate keywords are then scored based, at least in part, on an engagement metric measuring a user engagement with the content item in response to being presented with results from a search query comprising the candidate keyword. A candidate keyword is then selected from the plurality of candidate keywords based on the scoring, and stored relationally to the content item to define an audience for a recommendation about the content item, providing a suggestion to the content provider.
A device interfaced with an online system detects, via a physical sensor, item removal and generates a user interface with an alternative option for conversion. Upon receiving a signal from the device indicating the item removal, the online system selects a set of candidate items for replacement of the removed item, wherein each candidate item has a conversion value that is less than a conversion value of the removed item. The online system applies a trained machine-learning model to generate a conversion score for each candidate item that indicates a likelihood of conversion by the user of each candidate item. The online system selects, based on the conversion score for each candidate item, a replacement item from the set of candidate items, and generates a user interface signal that causes a user interface of the device to prompt the user to convert the replacement item.
An online system performs batch matching with synchronization between a broadcast signal and a boost signal. At a first timestep, the system notifies a first set of candidate pickers of a batch. If no picker selects the batch, the system identifies a catalyst action to facilitate matching. The system applies a decision model to determine whether to increase a broadcast signal or to increase a boost signal. If increasing the broadcast signal, the system identifies additional candidate pickers to notify of the batch. If increasing the boost signal, the system transmits the boost signal to the pickers for notification. The system may iteratively assess whether a candidate picker has selected the batch. If not, then the system may identify and perform additional catalyst actions to facilitate the matching of the batch. Eventually, the system receives a selection by one of the candidate picking users for fulfillment.
Leveraging a large language model for alternative ingredient determination is described. An online system receives, from a user device, an instruction to determine an alternative ingredient. An alternative ingredient is different from an ingredient of a recipe but has a common purpose with the ingredient in a context of the recipe. A large language model is prompted, based in part on the instruction, to determine one or more alternative ingredients for the ingredient of the recipe. An output of the large language model includes the one or more alternative ingredients. The output is processed, and at least some of the processed output is provided to the user device, and the user device presents at least one of the one or more alternative ingredients to the ingredient.
An online system that displays items from an item catalog to users supplements content displayed for one or more of the items with information extracted from images of the items. For a particular item in the item catalog, the online system performs image processing, such as optical character recognition, on one or more images of the item to extract text phrases from the images. For each extracted text phrase, the system then uses a trained model to score the text phrase as being a viable informational message. If the score for a text phrase is above a threshold, the online system augments content displayed in a user interface for the item with the text phrase. The online system may decide whether to supplement content for the item with an extracted text phrase based on the output of a predictive model.
An online system performs item redesign and engagement prediction. The system obtains item data describing characteristics of a target item for redesign. The system generates a prompt including the characteristics and directions to redesign at least one of them. The system executes the prompt on a generative model to output redesigns. Each redesign includes a modification to at least one characteristic of the target item. The system inputs features of variants, each variant include one store location and one redesign, into an engagement prediction model to output an engagement score for the variant. The engagement prediction model is trained on historical data describing levels of user engagement with items in association with the many store locations. The system identifies candidate variants based on the user engagement scores for further testing. The system transmits the candidate variants to a testing system to assess viability of the redesign.
An online concierge system applies a predictive model to predict demand of items, and facilitates preemptive picking of items in advance of receiving orders to enable efficient procurement and delivery. The online concierge system may apply a time-series model and/or machine learning model that predicts demand based on historical data. Depending on the predicted demand, items may be preemptively moved from a storage location to a staging area that enables the items to be more rapidly processed and delivered to customers when orders come in.
An online system may receive, from a content provider, a content presentation campaign that includes one or more objectives. The online system may define a set of one or more policy functions that automatically controls the content presentation campaign. A policy function may control one or more criteria in bidding content slots. The online system may monitor a realized outcome of the content presentation campaign. The online system may apply a reinforcement learning algorithm in adjusting the set of policy functions. The reinforcement learning algorithm adjusts one or more parameters in the set of policy functions to reduce a difference between the realized outcome and the desired outcome set by the content provider. The online system generates an adjusted set of policy functions and uses the adjusted set of policy functions in bidding content slots to present one or more content items provided by the content provider.
To evaluate the causal generalizability of multivariate regression models (such as marketing mix models) that evaluate a plurality of input features that may have high correlation and confounding causality, a model architecture is evaluated with respect to experimental data that varies feature values. The model architecture is trained with training data that excludes the experimental data. The trained model is then applied to predict the outcome of the experimental data inputs and the predicted outcome is scored with respect to the experimental outcome. This may be repeated across more than one experiment to evaluate how the model architecture generalizes to different types of variations in different experiments. The scores may then be used to validate the causal predictions and select or confirm a model architecture for use.
An online system receives information captured by a gaze tracking device describing a gaze point of a user and video data captured within a source location, detects a location associated with a first item that matches the gaze point based on the received information, and determines the first item is not available at the source location based on the video data. The system receives a signal indicating the user collected a second item from the source location, determines the second item is a replacement for the first item, and generates a new training example indicating the second item is an acceptable replacement for the first item for the user.
An online system receives information captured by a gaze tracking device describing a gaze point of a user and video data captured within a source location, detects a location associated with a first item that matches the gaze point based on the received information, and determines the first item is not available at the source location based on the video data. The system receives a signal indicating the user collected a second item from the source location, determines the second item is a replacement for the first item, and generates a new training example indicating the second item is an acceptable replacement for the first item for the user.
The system trains a machine-learning model to generate a score indicating whether a candidate item is an acceptable replacement for a target item for a user, in which the model is trained using training data that includes the new training example.
An online system is configured to manage messaging between artificial intelligence (AI) agents. A service request (such as a request to order items) is received at an online system from a user client device. A system AI agent and a user AI agent are instantiated with inputs that include a set of objectives or constraints that guides each of the system AI agent and the user AI agent during messaging with the other. The online system manages rounds of messaging between the system AI agent and the user AI agent, and at some point, a proposed agreement between the user and online system is extracted from the messaging. The proposed agreement may then be presented to the user or online system for approval.
An online system generates subsequent orders for users following failed attempts to purchase items. The online system receives a request to fulfill an order from a user device. The online system determines that an item from the order is unable to be fulfilled and generates a failed fulfillment signal for the item associated with the user. At a later time, the online system automatically generates a set of items for a subsequent order for the user, the set of items including at least one item substantially similar to the item that was unable to be fulfilled and predicted by a machine-learned model to be available. The online system transmits a notification to the user that the set of items is available for fulfillment.
An online system selects content for placement in positions of a display on a user device. The online system selects a first set of content items according to a first content selection process and a second set of content items according to a second content selection process. To combine the different sets of content items dynamically, the first set of content items and second set of content items are evaluated by a joint impression scoring that includes factors prioritizing user, intrinsic, and other values. The respective contribution by the different factors may be adjusted by one or more adjustable weights, enabling different situations to effect different combinations of content items from the different content selection processes.
An online system is configured to manage messaging between artificial intelligence (Al) agents. A service request (such as a request to order items) is received at an online system from a user client device. A system Al agent and a user Al. agent are instantiated with inputs that include a. set of objectives or constraints that guides each of the system Al agent and the user Al agent during messaging with the other. The online system manages rounds of messaging between the system Al agent and the user Al agent, and at some point, a proposed agreement between the user and online system is extracted from the messaging. The proposed agreement may then be presented to the user or online system for approval.
An online system receives orders from users and fulfills the orders by dispatching a picker to a physical source to obtain the items for delivery. Some items in an order may be considered “foundational,” meaning that a user who ordered the items may wish to cancel one or more other items in the order if the foundational item is unavailable (e.g., the item is a critical ingredient for a recipe). The online system predicts items in the order that are foundational using a trained machine-learning model. The online system presents the items to the picker in a sequence so the foundational items are obtained earlier by the picker. This enables the picker to observe whether the determined foundational item is available sooner in the picking process, allowing earlier performance of a remedial action and possibly avoiding replacing previously obtained items affected by the unavailability of the foundational item.
An online system uses a trained machine-learning model to detect errors in catalog data based on interactions of users of the online system with physical carts. Upon receiving an interaction signal indicating an interaction by the user with a device in a location of a source or an action signal indicating an action in the location of the source, the online system applies the trained model to the interaction signal and/or the action signal to generate an error score for an item that indicates a likelihood of an error in relation to the item. Responsive to the error score being above a threshold score, the online system generates an error checking signal for confirming that the error is present. Responsive to the confirmation of the error, the online system generates a user interface that alerts about the error and requests an action to correct the error.
G06Q 20/40 - Authorisation, e.g. identification of payer or payee, verification of customer or shop credentialsReview and approval of payers, e.g. check of credit lines or negative lists
G06Q 10/087 - Inventory or stock management, e.g. order filling, procurement or balancing against orders
G06V 10/70 - Arrangements for image or video recognition or understanding using pattern recognition or machine learning
