Computer Vision

One of our missions at Wayfair is to help our 22 million customers find the products they are looking for. For example, when a customer searches for a “modern yellow sofa” on Wayfair, we want to show the most relevant options from the tens of thousands of sofas available in our catalog. To do that, it is important to have a strong understanding of the products we sell. We use machine learning algorithms to analyze and understand the descriptive information (e.g. color, shape) of over 40 million products provided by more than 20 thousand suppliers in our catalog.

Images from product reviews can inspire and inform our customers. We created a machine learning model that helps us in prioritizing the most helpful and good looking images on the product page.

When a Wayfair customer shops for a product that is just right for her home, color plays a big role. For example, a customer might say, “I am looking for a green chair.” Or she might say, “I am looking for a teal chair.” There may be various versions of these customer color search stories. As data scientists at Wayfair, we want to solve these customer problems by algorithmically extracting the color information from the products and assigning customer-friendly color names.

High-quality lifestyle imagery is core to Wayfair’s ability to merchandise its catalog effectively and help customers shop confidently. Creating lifestyle images through 3D models enables rapid production of product visual merchandising at scale, drives simultaneous reduction in cost and speed to produce imagery, and unlocks new ways (e.g. AR, VR) for customers to engage with products. Early tests show 3D-generated images can provide high quality visual detail of our products and increase conversion rates for flagship brands (main products in Wayfair’s portfolio and to which customers most relate or identify Wayfair). Therefore, producing 3D models quickly and accurately is key to further reducing 3D imagery costs and driving revenue. In this blog, we will introduce how machine learning is utilized to aid 3D modeling at Wayfair.

‘First impressions matter’.

Introduction

Stylistic preference is an important factor when a home goods customer is deciding which product to buy, but it is very difficult to identify and define. Although designers have established different style categories, labeling a scene as adhering to a particular style is a highly subjective task. Furthermore, customers often cannot verbalize their style preferences, but can identify their preferences by looking at images. Thus, it is crucial to show products in a room context that are tailored to a customer's taste.

This week in Wayfair Data Science’s explainer series, we’re discussing object pose estimation, an important problem in robotics and augmented reality (AR) applications. In robotics, when given a 3D model of an object a mobile robot must be able to localize it in space in order to manipulate it. This localization process is also central to our AR work at Wayfair. On the Wayfair app, you can explore how our products look in your room using AR. The ability to estimate the pose of the selected item while you are moving your smartphone around your room is essential to providing the best AR experience. In this video, Wayfair data science manager Esra Cansizoglu explains how we solve this problem using perspective-n-point algorithm in a RANSAC framework.

This week in Wayfair Data Science’s explainer series, Senior Machine Learning Engineer Tim Zhang lays out what you need to know about training image synthesis. Training image synthesis is a fairly young project for the Computer Vision team at Wayfair. We have explored a few different use cases that can take advantage of this approach, such as search-with-photo, image-based 3D geometry generation, and camera perspective estimation. Looking forward, we are aiming at faster rendering speed and more intelligent domain randomization.

Last year, Wayfair launched Visual Search, a new and novel way to find products on our website. Users can now upload photos of furniture they like and find visually similar matches in an instant. If you need a little refresher on this technology, the original blog post can be found here.