Article: Current Trends in Deep Learning

In the last decade, the area of artificial intelligence (AI) has exploded with interesting and promising results. With major achievements in image recognition, speech recognition and highly complex games, AI continues to disrupt society. This blog post will discuss practical applications of AI, optimization and interpretability of deep learning models and reinforcement learning (RL), based on the 2018 REWORK Deep Learning Summit in Toronto. Four software engineers from Knowit had the pleasure of travelling to Canada to attend this conference, and with renowned speakers such as Geoff Hinton attending, it turned out be an insightful experience.

Article: Are we Asking too Much of Algorithms?

Although algorithms will get better at advising on the most difficult of judgement calls, we need to remain realistic about where the boundaries currently lie and accept that humans still have a critical role to play in decision-making.

Paper: Learning Fair Representations via an Adversarial Framework

Fairness has become a central issue for our research community as classification algorithms are adopted in societally critical domains such as recidivism prediction and loan approval. In this work, we consider the potential bias based on protected attributes (e.g., race and gender), and tackle this problem by learning latent representations of individuals that are statistically indistinguishable between protected groups while sufficiently preserving other information for classification. To do that, we develop a minimax adversarial framework with a generator to capture the data distribution and generate latent representations, and a critic to ensure that the distributions across different protected groups are similar. Our framework provides a theoretical guarantee with respect to statistical parity and individual fairness. Empirical results on four real-world datasets also show that the learned representation can effectively be used for classification tasks such as credit risk prediction while obstructing information related to protected groups, especially when removing protected attributes is not sufficient for fair classification.

Paper: Fair Classification and Social Welfare

Now that machine learning algorithms lie at the center of many resource allocation pipelines, computer scientists have been unwittingly cast as partial social planners. Given this state of affairs, important questions follow. What is the relationship between fairness as defined by computer scientists and notions of social welfare? In this paper, we present a welfare-based analysis of classification and fairness regimes. We translate a loss minimization program into a social welfare maximization problem with a set of implied welfare weights on individuals and groups–weights that can be analyzed from a distribution justice lens. In the converse direction, we ask what the space of possible labelings is for a given dataset and hypothesis class. We provide an algorithm that answers this question with respect to linear hyperplanes in $\mathbb{R}^d$ that runs in $O(n^dd)$. Our main findings on the relationship between fairness criteria and welfare center on sensitivity analyses of fairness-constrained empirical risk minimization programs. We characterize the ranges of $\Delta \epsilon$ perturbations to a fairness parameter $\epsilon$ that yield better, worse, and neutral outcomes in utility for individuals and by extension, groups. We show that applying more strict fairness criteria that are codified as parity constraints, can worsen welfare outcomes for both groups. More generally, always preferring ‘more fair’ classifiers does not abide by the Pareto Principle—a fundamental axiom of social choice theory and welfare economics. Recent work in machine learning has rallied around these notions of fairness as critical to ensuring that algorithmic systems do not have disparate negative impact on disadvantaged social groups. By showing that these constraints often fail to translate into improved outcomes for these groups, we cast doubt on their effectiveness as a means to ensure justice.

Article: AI Momentum, Maturity & Models for Success

Ethics is not a new area of focus for large organizations, so perhaps it’s no surprise that these respondents indicated that they are preparing their employees for the ethical deployment of AI. What sets AI apart, however, is the recognition that an artificially intelligent process is one for which the organization has responsibility for the outcomes – in the same way that the organization is responsible for the actions of its employees. Ethical training for technologists signals an understanding of the stakes involved with potentially unethical uses of AI. AI adopters indicated relatively strong ethical processes in place today, with 63 percent affirming that they ‘have an ethics committee that reviews the use of AI,’ and 70 percent indicating they ‘conduct ethics training for their technologists.’

Paper: Fairness-Aware Ranking in Search & Recommendation Systems with Application to LinkedIn Talent Search

Recently, policymakers, regulators, and advocates have raised awareness about the ethical, policy, and legal challenges posed by machine learning and data-driven systems. In particular, they have expressed concerns about their potentially discriminatory impact, for example, due to inadvertent encoding of bias into automated decisions. For search and recommendation systems, our goal is to understand whether there is bias in the underlying machine learning models, and devise techniques to mitigate the bias. This paper presents a framework for quantifying and mitigating algorithmic bias in mechanisms designed for ranking individuals, typically used as part of web-scale search and recommendation systems. We first propose complementary measures to quantify bias with respect to protected attributes such as gender and age. We then present algorithms for computing fairness-aware re-ranking of results towards mitigating algorithmic bias. Our algorithms seek to achieve a desired distribution of top ranked results with respect to one or more protected attributes. We show that such a framework can be utilized to achieve fairness criteria such as equality of opportunity and demographic parity depending on the choice of the desired distribution. We evaluate the proposed algorithms via extensive simulations and study the effect of fairness-aware ranking on both bias and utility measures. We finally present the online A/B testing results from applying our framework towards representative ranking in LinkedIn Talent Search. Our approach resulted in tremendous improvement in the fairness metrics without affecting the business metrics, which paved the way for deployment to 100% of LinkedIn Recruiter users worldwide. Ours is the first large-scale deployed framework for ensuring fairness in the hiring domain, with the potential positive impact for more than 575M LinkedIn members.

Paper: Predicting Economic Development using Geolocated Wikipedia Articles

Progress on the UN Sustainable Development Goals (SDGs) is hampered by a persistent lack of data regarding key social, environmental, and economic indicators, particularly in developing countries. For example, data on poverty — the first of seventeen SDGs — is both spatially sparse and infrequently collected in Sub-Saharan Africa due to the high cost of surveys. Here we propose a novel method for estimating socioeconomic indicators using open-source, geolocated textual information from Wikipedia articles. We demonstrate that modern NLP techniques can be used to predict community-level asset wealth and education outcomes using nearby geolocated Wikipedia articles. When paired with nightlights satellite imagery, our method outperforms all previously published benchmarks for this prediction task, indicating the potential of Wikipedia to inform both research in the social sciences and future policy decisions.

Article: Humans choose, AI does not

First, as Yogi Berra said, ‘It’s tough to make predictions, especially about the future.’ Where is my flying car? Second, the title reads like clickbait, but surprisingly it appears to be pretty close to the actual survey, which asked AI researchers when ‘high-level machine intelligence’ will arrive, defined as ‘when unaided machines can accomplish every task better and more cheaply than human workers.’ What is a ‘task’ in this definition? Does ‘every task’ even make sense? Can we enumerate all tasks? Third and most important, is high-level intelligence just accomplishing tasks? This is the real difference between artificial and human intelligence: humans define goals, AI tries to achieve them. Is the hammer going to displace the carpenter? They each have a purpose. This difference between artificial and human intelligence is crucial to understand, both to interpret all the crazy headlines in the popular press, and more importantly, to make practical, informed judgements about the technology. The rest of this post walks through some types of artificial intelligence, types of human intelligence, and given how different they are, plausible and implausible risks of artificial intelligence. The short story: unlike humans, every AI technology has a perfectly mathematically well-defined goal, often a labeled dataset.