We are all surrounded by intelligent recommender systems. When you listen your LoFi hip hop playlist on Spotify while studying, or read articles on Quora, or order a wireless keyboard on Amazon you are being tracked. AI finds out what product to suggest to you or determines the optimal position for an advertisement. After the content is shown to you, the service/website/application monitors your reaction (press the like button, add to favorites, number of views, number of links clicked etc.) and reports a reward score back to the service you are using. This ensures continuous improvement of the AI model behind this service, and its ability to select the best content item based on the contextual data it collects. The list of areas where such systems are used is very vast, varying from movies, books, music, news to search queries and social labels.
How it usually works
To sum up, we can define three main approaches :
Content-based : for this, the system collects the data both from users and items. The properties of these items or users are in fact the features you feed into your model. Thus, what you really do in this approach is calculate the similitude between users and items to construct a recommendation.
Collaborative filtering : for this approach, the only thing you need is item/user identifiers and ratings given by users to the items. Thus, we learn from all the items a user liked, and all the other users who liked a specific item. Such ratings will be considered as the common points in the relationships to make a recommendation.
Hybrid recommendation: as you may have guessed from its name, the principle is quite straightforward, use both collaborative filtering and content-based recommendation for a better result. One of the examples is Matchbox recommender developed by Microsoft Research. Even though nowadays there are more sophisticated algorithms, Matchbox is an excellent starting point to understand the principles of hybrid approach.
Some systems use reinforcement learning to choose the product that suits the best a specific user based on collective behavior and reward scores across all the other clients. An example of such an item may be the content items, such as news articles, specific movies, or products.
There are also some exotic approaches based on LDA (Latent Dirichlet Allocation). LDA is a topic modeling algorithm aimed to clusterize your documents corpus into a discrete number of collections (so-called topics), for better analysis. The output is usually composed of two matrices, a topic-per-doc distribution and a term-per-topic distribution. Something like this:
document 1 - Topic1, Topic 3
document 2 - Topic 2, Topic 1
To adapt the topic modeling approach, system considers user as a document and items he ordered/liked/read as terms. Thus it tries to find similar users and similar items based on prior clustering.
The issue that is addressed the most is ethics and responsible use. Besides the GDPR rules and the fact that your personal data is being collected, there are some more important aspects of the problem.
Things like gender, race, religion or age are generally prohibited for regulatory reasons, but when they are allowed, from the ethical point of view, you should not use these parameters as features in your recommendation system. Just imagine a search engine proposing items based on your age and religion, and not showing an important information you might be interested in just because you are too old.
Zip/postal code or neighborhood information do not propagate bias on their own. But they can be highly correlated with the features from the previous paragraph, thus a smart engineer can guess your race or religion based on your location.
User perception of fairness
This is a very tricky feature to deal with. Imagine your application is built using all the ethical principles, but it's appearance is highly correlated with the features mentioned above. You should also consider the user perception and predict the different scenarios where the content of your application may vary dynamically.
Unintended bias in features
This is the most complicated aspect to figure out, and, especially, to cope with. As a NLP researcher I may provide a concrete example of bias propagation in textual features.
There exist three types of bias (preexisting, technical and emergent) and word embeddings might face all the three types. It is scientifically proven that such discriminatory parameters as sexism and ethnic stereotypes in the training set (the one you used to construct the embeddings, like Wikipedia or tweets) are transferred to the word vectors, denoting the presence of a preexisting bias. Word vectors trained by different models yield different results on benchmark tests, which corresponds to the technical bias. And finally, generalizations on social interactions based on the distance of words immanent in an embedding space, or by inserting the embeddings in another model for prediction or inference might result in the decisions deviating from common social imperatives, which is an example of emergent bias.
Trust is built on the notion of fulfilled commitments. Consequently, before developing your AI, you should analyze your audience and use case to understand all the direct and indirect commitments you might have. Here the list of use cases that might have negative effect from the ethical point of view, so you should be extremely attentive while devising your strategy:
Finance: loan, financial, and insurance products
Education: ranks for school courses and education institutions
Democracy and Civic Participation: content for users with the goal of influencing opinions
Third-party reward evaluation: items where the reward is based on a latter 3rd party evaluation of the user
Intolerance to Exploration: Any situation where the exploration behavior of your AI may cause harm
An excellent example of a tool allowing you to create a sophisticated recommendation engine, is Microsoft's Personalizer.
Azure Personalizer is a cloud-based service that helps your applications choose the best content item to show your users. You can use the Personalizer service to determine what product to suggest to shoppers or to figure out the optimal position for an advertisement. After the content is shown to the user, your application monitors the user's reaction and reports a reward score back to the Personalizer service. This ensures continuous improvement of the machine learning model, and Personalizer's ability to select the best content item based on the contextual information it receives.
What is great about this tool is that it allows to construct your system based on the best practices, such as :
Consider the legality and ethics of utilizing particular features for certain applications, as well as whether seemingly innocuous features are proxies for others you wish to avoid.
Make it clear to consumers that data analysis and algorithms are being utilized to customize the alternatives they see.
Consider this question: Would my consumers care and be pleased if I utilized this knowledge to tailor content to them? Would it be okay if I showed them how the decision to highlight or conceal specific things was made?
Use behavioral data instead of data based on other criteria for classification or segmentation.
Demographic information was traditionally used by retailers for historical reasons – demographic attributes seemed simple to collect and act upon before a digital era, - but question how relevant demographic information is when you have actual interaction, contextual, and historical data that relates more closely to the preferences and identity of users.
Consider how to avoid features from being 'spoofed' by malicious users, which if exploited in large numbers can lead to retraining your AI in misleading ways to purposefully disrupt, embarrass and harass certain categories of users.
When suitable and feasible, construct your software to permit your customers to decide in or decide out of getting sure private functions used. These may be grouped, such as "Location information", "Device Information", "Past Purchase History" etc.
In this short post we've discussed the biases that may be intentionally or unintentionally propagated in the recommendation systems you construct. We've also seen the most important principles and best practices of designing an ethical AI. Hope you will find it useful, and some of the readers, after reading this article, will have a better vision of how to implement AI in a responsible manner. Together for a better world!