In 2014, a team of Amazon machine learning engineers in Edinburgh, Scotland, was given a task that suited a company that had automated almost every step of retail: automate the search for top talent too. According to Reuters, the system they built rated candidates from one to five stars, just as shoppers rate products on Amazon1. The ambition, as one insider told Reuters, was to feed in a hundred CVs, have the machine return the top five, and hire them.
To teach the machine what "good" looked like, the team used the most plentiful data it had: the CVs Amazon had received over the previous ten years1. It sounded sensible. What better teacher than a decade of hiring at one of the world's most successful technology companies?
The machine learned its lesson, and the lesson was flawed
Most of those CVs came from men, reflecting the reality of the tech industry over that decade. The machine had no idea what gender was. It simply saw patterns repeated in the files of people who had been hired, and learned to score files like theirs highly.
By 2015 the team realised the system was not rating candidates for software developer and other technical jobs in a gender-neutral way1. It penalised CVs containing the word "women's", as in "women's chess club captain". It downgraded graduates of two all-women's colleges1.
What the machine valued is even more telling. According to Reuters, the team built 500 models for specific functions and locations, each taught to recognise some 50,000 terms from past CVs1. Skills almost every IT applicant shares, such as the ability to code, counted for very little. What counted was style: verbs such as "executed" and "captured", more common on male engineers' CVs1.
In other words, the machine did not learn to recognise good people. It learned to recognise people who resembled those already hired.
Fixing a word does not fix the logic
Amazon edited the programs to make them neutral to specific terms like "women's". But according to people involved, there was no guarantee the machine would not devise other ways of sorting candidates that proved discriminatory1. Block one word and the machine finds another that carries the same signal: a club, a sport, a choice of verb.
By early 2017 the team had been disbanded because executives lost hope for the project. Amazon told Reuters the tool was never used by its recruiters to evaluate candidates. The people involved said recruiters did look at its recommendations but never relied on them alone1,2.
Why a 2018 story still matters
Amazon had some of the best machine learning engineers in the world and the resources to spot the problem. It spotted it, and it stopped. Most companies buying CV-screening software today have no such team. They get a packaged product, a score, and a promise that the machine is "objective".
The machine is not objective. It is loyal to its data. If a company has spent ten years hiring mostly from a handful of universities, former employers or regions, a model trained on that history will treat those markers as markers of ability, and it will do so quickly, across thousands of files, where nobody can see.
The law is catching up. The European Union's AI Act classifies AI systems used to recruit, filter applications and evaluate candidates as high-risk, bringing requirements for data governance, record-keeping and human oversight3. In Vietnam, the Law on Personal Data Protection, in force since 1 January 2026, sets clear obligations for collecting and processing data about employees and candidates4. A CV-scoring tool nobody can explain will be increasingly hard to defend.
This is not an argument against AI
Amazon's lesson is not to keep AI out of hiring. It is the question HR must ask before switching any tool on: whose decisions is the machine learning from, and are those decisions worth copying?
A practical stance is to treat AI like a new junior screener: let it work, check its output often, and never let it decide alone who is rejected.
