Don’t Look Stupid

In the last few posts we discussed a number of different algorithms which can be used to generate personalised recommendations for users. Once the recommendations are generated, they often need some post-processing (component 3) before being shown to users.  At this point, it’s common for some recommendations to be filtered out and some reranking to be applied. The post-processing component is typically responsible for making sure that the recommender doesn’t look stupid. In this blog post, we’ll briefly touch on a few examples and scenarios where good business logic can be used to prevent showing poor (or even bad) recommendations to users which could lead to the recommender looking stupid.

Business logic refers to the constraints that a company places on the recommender system. Its goal is not to make the recommender more accurate such as getting better accuracy metrics (e.g. precision, recall, etc), but to better serve business and product goals as well as ensure better user experience. For example, Amazon may have rules preventing recommending out-of-stock items or items the user has already purchased. Those items might be the best recommendations for the user, but are not very useful to the user since they are not available for purchase or the user already have them. Similarly, Netflix doesn’t recommend movies to watch if you don’t have permission to stream them in your geographic location. Netflix may also not recommend adult content to children, for obvious reasons, even if children would love to see it!

Social networks also have their particular challenges. At Mendeley, for example, we recommend other users to follow in the research social network. As part of building this feature, we did add a number of different filters to check for “bad” recommendations. For example, not recommending people the user already follows, making sure we have an explanation for the recommendation, the recommended person has a public profile, and the user allows other people in the network to follow them. One tricky problem to deal with when recommending people is also in the area of duplicate detection.  In some cases, we found that users had multiple accounts and so the recommender would recommend that they follow themselves (i.e. one user account is recommended to the other user account).  Some basic business logic here helps you to avoid looking stupid, but anyone who has worked with person name disambiguation before knows that this is a pandora’s box which we won’t even try to open here.  It’s common to sacrifice recall for higher precision.

As Mark Levy, pointed out at his RecSys 2013 keynote, in most cases it is quite obvious to users when they receive a bad recommendation, although it is not easy to infer why (not relevant, too obscure, too familiar, too old, user already has it or already knows she does not like it, badly explained, etc.). Presenting users with bad recommendations has a cost associated with it, which mainly depends on the product and the users of the system. Getting it wrong could result in lost revenue, lost engaged users, amused, confused or even angry users. As a rule of thumb, most “really” bad recommendations can be caught by good business logic.

Most recommender system books don’t discuss this subject very much, but we wanted to point out that it is a core component which could really make sure that the overall recommender product gives good user experience and most of all doesn’t look stupid. It is worth noting that most often the business logic is application and use case dependent, but hopefully the examples we’ve discussed in this blog give good idea of the types of recommender shortcomings they can be used to catch and correct.