Awful AI is a curated list to track current scary usages of AI - hoping to raise awareness to its misuses in society
Artificial intelligence in its current state is unfair, easily susceptible to attacks and notoriously difficult to control. Nevertheless, more and more concerning the uses of AI technology are appearing in the wild. This list aims to track all of them. We hope that Awful AI can be a platform to spur discussion for the development of possible contestational technology (to fight back!).
AI-based Gaydar - Artificial intelligence can accurately guess whether people are gay or straight based on photos of their faces, according to new research that suggests machines can have significantly better “gaydar” than humans. [summary]
Infer Genetic Disease From Your Face - DeepGestalt can accurately identify some rare genetic disorders using a photograph of a patient's face. This could lead to payers and employers potentially analyzing facial images and discriminating against individuals who have pre-existing conditions or developing medical complications. [Nature Paper]
Racist Chat Bots - Microsoft chatbot called Tay spent a day learning from Twitter and began spouting antisemitic messages.
Racist Auto Tag - a Google image recognition program labeled the faces of several black people as gorillas. Amazon's Rekognition labeled darker-skinned women as men 31 percent of the time. Lighter-skinned women were misidentified 7 percent of the time. Rekognition helps the Washington County Sheriff Office in Oregon speed up how long it took to identify suspects from hundreds of thousands of photo records. [ABC report on Rekognition bias]
Sexist Recruiting - AI-based recruiting tools such as HireVue or an Amazon internal software, scans various features such as video or voice data of job applicants and their CVs in order to tell whether they're worth hiring. In the case of Amazon, the algorithm quickly taught itself to prefer male candidates over female ones, penalizing CVs that included the word "women's," such as "women's chess club captain." It also reportedly downgraded graduates of two all-women's colleges. [summary]
PredPol - PredPol, a program for police departments that predicts hotspots where future crime might occur, could potentially get stuck in a feedback loop of over-policing majority black and brown neighborhoods. [summary]
COMPAS - is a risk assessment algorithm used in legal courts by the state of Wisconsin to predict the risk of recidivism. Its manufacturer refuses to disclose the proprietary algorithm and only the final risk assessment score is known. The algorithm is biased against blacks (even worse than humans). [summary][NYT opinion]
Infer Criminality From Your Face - A program that judges if you’re a criminal from your facial features. [summary]
Homeland Security - Homeland security, with DataRobot, is creating a terrorist-predicting algorithm trying to predict if a passenger or a group of passengers are high-risk by looking at age, domestic address, destination and/or transit airports, route information (one-way or round trip), duration of the stay, and luggage information, etc., and comparing with known instances.
iBorderCtrl - AI-based polygraph test for travellers entering the European Union (trial phase). Likely going to have a high number of false positives, considering how many people cross the EU borders every day. Furthermore, facial recognition algorithms are prone to racial bias. [summary]
Faception - Based on facial features, Faception claims that it can reveal personality traits e.g. "Extrovert, a person with High IQ, Professional Poker Player or a threats". They build models that classify faces into categories such as Pedophile, Terrorist, White-Collar Offenders and Bingo Players without prior knowledge. [classifiers][video pitch]
Cambridge Analytica - Cambridge Analytica uses Facebook data to change audience behavior for political and commercial causes. [Guardian article]
Deep Fakes - Deep Fakes is an artificial intelligence-based human image synthesis technique. It is used to combine and superimpose existing images and videos onto source images or videos. Deepfakes may be used to create fake celebrity pornographic videos or revenge porn. [AI assisted fake porn][CNN Interactive Report][Deep Nudes]
Fake News Bots - Automated accounts are being programmed to spread fake news. In recent times, fake news has been used to manipulate stock markets, make people choose dangerous health-care options, and manipulate elections, including the 2016 US presidential election. [summary]
Attention Engineering - From Facebook notifications to Snapstreaks to YouTube auto plays, they're all competing for one thing: your attention. Companies prey on our psychology for their own profit.
Social Media Propaganda - The Military is studying and using data-driven social media propaganda to manipulate news feeds in order to change the perceptions of military actions. [Guardian article]
Predicting Mass Protests - The US Pentagon funds and uses technologies such as social media surveillance and satellite imagery to forecast civil disobedience and infer location of protesters via their social networks around the world. There are indications that this technology is increasingly used to target Anti-Trump protests, leftwing groups and activists of color.
Gait Analysis - Your gait is highly complex, very much unique and hard, if not impossible, to mask in this era of CCTV. Your gait only needs to be recorded once and associated with your identity, for you to be tracked in real-time. In China this kind of surveillance is already deployed. In addition, multiple people have been convicted on their gait alone in the west. We can no longer stay even modestly anonymous in public.
SenseTime & Megvii- Based on Face Recognition technology powered by deep learning algorithm, SenseFace and Megvii provides integrated solutions of intelligent video analysis, which functions in target surveillance, trajectory analysis, population management. [summary][forbes][The Economist (video)]
Uber - Uber's "God View" let Uber employees see all of the Ubers in a city and the silhouettes of waiting for Uber users who have flagged cars - including names. The data collected by Uber was then used by its researchers to analyze private intent such as meeting up with a sexual partner. [rides of glory]
Palantir - A billion-dollar startup that focuses on predictive policies, intelligence and ai-powered military defense systems. [summary]
Social Credit System - Using a secret algorithm, Sesame credit constantly scores people from 350 to 950, and its ratings are based on factors including considerations of “interpersonal relationships” and consumer habits. [summary][Foreign Correspondent (video)][travel ban]
Health Insurance Credit System - Health insurance companies such as Vitality offer deals based on access to data from fitness trackers. However, they also can charge more and even remove access to important medical devices if patients are determined to be non compliant to unfair pricing. [ProPublica]
Misleading Show Robots - Show robots such as Sophia are being used as a platform to falsely represent the current state of AI and to actively deceive the public into believing that current AI has human-like intelligence or is very close to it. This is especially harmful as it appeared on the world's leading forum for international security policy. By giving a false impression of where AI is today, it helps defense contractors and those pushing military AI technology to sell their ideas. [Criticism by LeCun]
Zach - an AI, developed by the Terrible Foundation, claimed to write better reports than medical doctors. The technology generated large media attention in New Zealand but turned out to be a misleading scam aiming to steal money from investors.
Research to create a less awful and more privacy-preserving AI
Differential Privacy - A formal definition of privacy that allows us to make theoretical guarantees on data breaches. AI algorithms can be trained to be differentially private. [original paper]
Privacy-Preservation using Trusted Hardware - AI algorithms that can run inside trusted hardware enclaves (or private blockchains that build upon it) and train without any shareholder having access to private data.
Privacy-Preservation using Secure Computation - Using secure computation techniques like secret sharing, Yao's garbled circuits, or homomorphic encryption to train and deploy private machine learning models on private data using existing machine learning frameworks.
Fair Machine Learning & Algorithm Bias - A subfield in AI that investigates different fairness criteria and algorithm bias. A recent best paper (in ICLR18), e.g. shows that implementing specific criteria can have a delayed impact on fairness.
Adversarial Machine Learning - Adversarial examples are inputs, which cause the model to make a mistake. Research in adversarial defenses includes but is not limited to adversarial training, distillation and Defense-GAN.
These open source projects try to spur discourse, offer protection or awareness to awful AI
Data Selfie - Data Selfie is a browser extension that tracks you while you are on Facebook to show you your own data traces and reveal what machine learning algorithms could predict about your personality based on that data. [code]
AdNauseam - AdNauseam is a lightweight browser extension to fight back against tracking by advertising networks. It works like an ad-blocker (it is built atop uBlock-Origin) to silently simulate clicks on each blocked ad, confusing trackers as to one's real interests. [code]
B.S Detector - B.S. Detector is a browser extension that searches all links on a given webpage for references to unreliable sources, checking against a manually compiled list of domains. It then provides visual warnings about the presence of questionable links or the browsing of questionable websites. [code]
Snopes.com - The Snopes.com website was founded by David Mikkelson, a project begun in 1994 and has since grown into the oldest and largest fact-checking site on the Internet, one widely regarded by journalists, folklorists, and laypersons alike as one of the world’s essential resources.
Facebook Container - Facebook Container isolates your Facebook activity from the rest of your web activity in order to prevent Facebook from tracking you outside of the Facebook website via third-party cookies. [code]
TrackMeNot - TrackMeNot is a browser extension (Chrome, Firefox) that helps protect your online searches by creating fake search queries. This creates noise in data that makes it harder to track and profile user behaviour. [code]
Center for Democracy & Technology - Digital Decisions is an interactive graphic that helps you ask the right questions when designing/implementing or building a new algorithm.
License
To the extent possible under law, David Dao has waived all copyright and related or neighboring rights to this work.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent