Over thirty million customers from around the world shop on Wayfair to look for furniture that helps them create their unique sense of home. Wayfair serves a variety of webpages catered to niche customer sub-segments and help customers discover new products – these include pages with advice for people buying “twin futons,” or “nursery chairs with removable cushions.”
To state the obvious, it would be a near impossible feat to create pages manually for the hundreds of thousands of products for the “long tail” of Wayfair’s catalog. Wayfair’s scientists are using a technique called aspect based sentiment analysis to solve this problem.
Aspect based sentiment analysis (ABSA) is the fine grained understanding of a piece of text. ABSA is different from sentiment analysis, where we identify the sentiment from a more holistic perspective – for example, whether a string of text such as a tweet or a customer review is positive or negative.
In aspect based sentiment analysis, we go one step deeper. We identify specific entities within a text: specific aspects of a product such as “comfort,” “delivery” or “speed.” We identify the sentiments for each of these aspects or entities, and select products with aspects that have positive sentiments to build out long tail pages.
To give an example of ABSA in action, the marketing page “ Top 15 Twin Futons in 2022” draws on customer reviews to inform the product selection. The page features a variety of customer reviews, and our models draw on the noun phrases (the subjects, objects or prepositional objects) in a sentence to identify aspects.
Model for Exploiting BERT for End-to-End Aspect-based Sentiment Analysis ( from the research paper published in October 2019)
Our model builds off the transformational approach presented in BERT (Bidirectional Encoder Representations from Transformers). BERT allows our model to learn the context of a word in a more complete way, based on everything that preceded and succeeded the word. BERT is pre-trained on large amounts of unannotated text on the web, and can be plugged into a more specific model that is then fine-tuned on a natural language processing task, such as aspect-based sentiment analysis. Compared to models that do not have BERT’s generic knowledge of language built in, fine-tuning with a BERT layer can convey substantial accuracy improvements. The BERT embedding layer outputs a sequence of word embeddings which are consumed by the layers specific to the downstream task.
For the purposes of ABSA, a second layer then classifies each embedded word to determine which words or even complete phrases should be considered as aspects. In our experiments, we have found that a self-attention network utilizing a transformer architecture has worked better than anything we’ve used before in terms of identifying aspects and their associated sentiments correctly. Take the example of a customer review that says, “I love the color of this bed, but the quality is terrible.” In this case, the model would identify color as an aspect with a positive sentiment, while quality is another sentiment with a negative sentiment.
Our original method of conducting ABSA was a two-step process: identify the aspect, and then the associated sentiment, which if you think about it is very similar to the thought process employed by human beings.
At Wayfair, we essentially combined these two processes – aspect extraction and sentiment analysis – into a single layer. One drawback of the two-step approach is that you might end up with a situation where you have too many aspects, or generate aspects that people don’t really care about. To solve this problem, our model utilizes sentiment information to inform the aspects that are important to our customers. It identifies certain aspects as important because customers feel strongly about them in some way. This means our model would ignore the aspect “legs” from a sentence such as, “This table has four legs.” However it would pick up that same aspect from the review, “I love how the legs are designed for this table – their width and heft really makes the table sturdy!”
The most challenging problems we faced while training our models had to do with giving precise labeling instructions. This is primarily due to the fact that identifying aspects is a subjective process, and it’s hard to arrive at a precise definition of what an aspect is. For example, if a customer review talks about money, is it an aspect? And, more importantly: is it an aspect that is important to Wayfair? Inconsistent answers to these sorts of questions can lead to the model trying to learn conflicting things at the same time, leading to lower overall performance.
This is why we spent a lot of time refining the labeling process to ensure that our labelers arrived at a consistent set of decisions. Our guidelines, carefully crafted over multiple iterations, were designed to ensure that we were able to identify the most important aspects from the sentiment. We also asked labelers to prioritize products with longer aspects (e.g. a mid-century modern dining table). Other instructions pertained to the basic rudiments of grammar. For example, “Don’t love” is negative (not neutral), and “Off balance” is an adjective, but we might want to mark “balance” as a noun if it’s an important aspect of the product.
We have seen positive results from our self-attention network, achieving a nearly 2X improvement in the macro-F1 score as compared to our prior model. The macro-F1 score is a number between 0 and 1 that essentially measures the overall performance of a machine learning model on a classification task.
Going forward, one of the potential applications of ABSA will be to help Wayfair customers filter products during searches more effectively. For example, customers will be able to apply filters for “beds that are easy to install,” and we would be able to surface the most relevant recommendations. ABSA can also be trained on a wide variety of text inputs such as product detail pages, which would help us surface even more relevant recommendations.
We recognize that buying furniture for your home is a process that is imbued with deep relevance and meaning. When you buy a table and bring it to your home, it ceases to be just a table. It becomes a special space where you write your next great novel, or where you share dinner with your family and loved ones. That’s why we are excited to be working on aspect based sentiment analysis at Wayfair – it allows us to enable customers to dive really deep into their every nuance, and find the item of furniture that’s just right for them.
