The images on Eduardo Fidalgo’s computer show mundane scenes – a sofa scattered with pillows, a folded duvet on a bed, some children’s toys strewn across a floor. They depict views most of us would see around our own homes.
But these rather ordinary pictures are helping to build a new weapon in the fight against crime. Fidalgo and his colleagues are using the images to train a machine to spot clues in crime scene photographs.
When police officers visit a crime scene or a suspect’s home, they are often confronted with an overwhelming amount of visual information. And hidden amongst the everyday objects they find at these sites may be vital pieces of evidence that could link someone to a crime.
You might also like:
• The best way to stop bullies?
• The signs of disease no one can see
• Will this technology end traffic jams?
Fidalgo, a computer scientist at the University of Leon in north-west Spain, and his team have been working with the Spanish National Cybersecurity Insitute INCIBE to develop an evidence recognising tool that uses artificial intelligence to identify objects in police photographs – and to search for links with other crimes. Take, for example, a bedroom where abuse has reportedly taken place. Officers routinely take photographs of such locations, capturing key information in the process.
Matching objects at crime scenes to those from previous incidents could help find links between crimes that might otherwise be missed by detectives (Credit: Getty Images)
“Those bedrooms have objects, toys, textures on curtains, the texture of the floor – all of these sorts of things,” says Fidalgo. “Let’s say this toy could be registered by the system and if the same guy used it in other crimes, it could be retrieved.”
That might not definitively link the suspect to past offences but it could certainly open up a line of inquiry worth checking out. And it might be something that the investigating officers, depending on who was present at the crime scene, would otherwise have missed.
Fidalgo and his colleagues have developed a prototype system to do exactly that, and he’s hoping that it will be trialled by Spanish police soon. But he mentions that there are already other image-recognition tools that police forces are using right now.
One bit of kit exists in the form of a bundle of bulky suitcases filled with laptops. The computers are set up to perform analysis on huge swathes of photos downloaded from a suspect’s electronic devices. The system can automatically recognise known faces and make estimates about the age and gender of individuals in the photographs. It can also find possible images of child sexual abuse, without officers having to comb through the full library of pictures themselves.
With police budgets in many parts of the world tightening, senior officers often hope AI will help their often shrinking departments cope
This is just one of the ways that artificial intelligence is being used by police forces around the world to help them in the fight against crime. The technology is being used to analyse photographs, CCTV footage, evidence files and logs of crimes to help give them an edge over those who attempt to escape the long arm of the law. With police budgets in many parts of the world tightening, senior officers often hope AI will help shrinking departments cope. The public in turn are promised that their communities will be safer.
Scanning thousands of images from the darkest corners of the internet could help track down the criminals who exploit children (Credit: Alamy)
Such technology already is being used far more widely than many people realise. Facebook, for example, recently revealed that it used AI to unearth nearly nine million images of child nudity on its network in just three months. Almost all of those images had gone unreported before, so Facebook was able to pass on details of potential abuse to the US National Center for Missing and Exploited Children.
Nearly 200 law enforcement agencies around the US also are using an algorithm developed by researchers at the University of Southern California that scours the internet for clues pointing to victims of human trafficking and the sex trade. It searches both the open and so-called dark web for information contained in sex adverts, unravelling the information they contain to help investigators track down potential victims.
The algorithm already has trawled through 25 million pages. It has proved so successful that the US Department of Defense is experimenting with using it for in investigations such as narcotics, illegal weapons sales and counterfeit goods.
The software was recently used to help identify officials in Thailand involved in a human trafficking case
Pulling together complex strands of evidence is where artificial intelligence can really help investigators. Police in the UK are trialling software developed by digital forensics firm Cellebrite, which automatically sifts through potential evidence on a suspect’s mobile phone. It can analyse images and communication patterns, match faces and cross-reference data from multiple devices, helping officers to quickly build up a detailed picture of how a group of suspects have interacted. The firm’s software was recently used to help identify officials in Thailand involved in a human trafficking case, including police officers, three politicians and an army general.
Facial recognition uses machine learning algorithms to identify people and is already being used by some police forces (Credit: Alamy)
Algorithms also can whizz through police data to pick out a variety of possible connections between criminal cases. The idea is to help police realise that individuals, evidence or crime patterns are present, says William Wong, a professor of human-computer interaction at Middlesex University.
He helped to develop a system called Valcri – Visual Analytics for Sense-Making in Criminal Intelligence Analysis.
“We’re not asking the machine to give you an answer but to give you what could be possibly relevant,” he says. “Show me anything else within my various databases that could look similar to the way this particular crime has been committed.”
After a couple of years of basic trials with police forces in Europe, including the UK, Valcri is now well on its way to becoming an operational tool.
“We’re no longer trialling it with dummy anonymised data; we’re now trialling it with live, real data,” says Wong. The system, he says, could help to solve a real crime for the first time within the next few months.
The somewhat untapped power of police databases, just waiting to be hooked up to AI, is tantalising
The somewhat untapped power of police databases, just waiting to be hooked up to AI, is tantalising, says Ruth Morgan, a forensics expert at University College London.
“The potential is absolutely phenomenal,” she says. However, she notes that when algorithms are used, it’s not always possible to later audit their decision-making in court. Either the technology is proprietary, and the companies that own it don’t want to give up their secrets, or the system is just so convoluted that proving how it reached its conclusion is almost impossible. That’s the sort of issue that could prevent any of these technologies from becoming adopted more broadly.
By helping police unlock the information contained in their databases, artificial intelligence could become a valuable tool in the fight against crime (Credit: Alamy)
Predictive policing
Another way that artificial intelligence is changing policing is through predictive analytics. This is where computer programmes analyse data on previous crimes and individuals or places associated with them. AI then uses this to make predictions about where a crime is most likely to take place or even who might be most likely to commit a crime.
Courts in the US, for example, are already using AI in this way to help make judgements about bail and sentencing decisions. Two major tools being used to do this are PredPol and CrimeScan.
But the approach they use is highly controversial because of the risk of bias in the predictions they make. One study showed that skewed data can indeed lead to a disproportionate increase in policing within communities that are already over-policed.
In many parts of the world, however, predictive policing is already highly prized by police forces strapped of cash and looking for ways to prioritise workloads. One new tool, the National Data Analytics System, is currently being developed in the UK and could eventually be used by every force in the country.
In fact, there are actually multiple systems in use in the UK. Durham policing are trialling another AI tool that helps judge whether someone in custody should be released on bail or not.
However, a mind-boggling range of possible applications continues to be investigated.
Morgan herself is working on an image-analysing system that can count microscopic particles found on the soles of suspects’ shoes. Think pollen grains or gunshot residues. From the number of any such particles on a shoe, officers might be able to estimate how long ago the wearer was present in a certain area.
Counting these particles can involve weeks or even months of work for a human forensics expert. Getting a machine to automatically take images of these particles and counting them takes hours, says Morgan. Having tested her system to count particles, she hopes to next develop an algorithm to also reliably identify the types of particles present – what pollen species for instance.
In forensics, it is tiny traces like these that can have the biggest implications. DNA analysis has had a huge impact on criminal investigations since it was introduced 30 years ago. But DNA still poses huge challenges. Often when samples are taken from clothing, objects or a victim’s body, for example, DNA from multiple sources is scooped up. This might include genetic material from the victim, the suspect, police officers, a witness or even a pet. How do you tell them apart and count the “contributors”?
It’s the sort of work that human DNA analysts do all the time, but it’s time consuming and easy to get it wrong. A study a few years ago found that 74 of 108 forensic labs in the US detected DNA from three people in a test sample that only contained genetic material from two. In real life, that could have resulted in an innocent person being implicated in a crime.
Had the labs been more confident about the number of contributors to the samples, such a mistake would have been less likely. Michael Marciano and Jonathan Adelman at the US Forensic & National Security Institute, which is part of Syracuse University in New York State, have designed a system called Pace – Probabilistic Assessment for Contributor Estimation – to help them do this.
From the number of any such particles on a shoe, officers might be able to estimate how long ago the wearer was present in a certain area
They trained a machine learning algorithm on thousands of dummy samples containing DNA from multiple sources. It gradually learned to distinguish samples that contain DNA from two people verses those with three sets of DNA, and so on. While Pace can’t be 100% sure about the number of contributors, Marciano and Adelman claim it is slightly more accurate than competing methods for analysis.
Traces of pollen or gun powder residue on a shoe can provide vital clues to link suspects to a crime scene, but matching them has been laborious work (Credit: Alamy)
Using machine learning, however, speeds up the process to give a result in a maximum of just three minutes, says Marciano.
Sometimes the fragments left behind by a person are more than genetic, but still puzzling. Police looking for missing persons or murder victims may occasionally discover bone fragments in their searches. They might not be able to match that material to a DNA sample, but knowing what the individual’s face looked like could help identify them. Forensic anthropologists currently piece together skull fragments and build up layers of facial tissue using a medium like clay in order to reconstruct a face.
When presented with an unidentified skull, the system creates thousands upon thousands of 3D reconstructions that it then searches through to find one that matches
This work is extremely laborious and its accuracy can vary between anthropologists. Xin Li, a computer scientist at Louisiana State University, thinks machines can help.
He has been developing a system that can take three dimensional scans of a few skull fragments and put them back together again like a jigsaw puzzle that has missing pieces. The system, which was trained on the shapes and proportions of human skulls, knows how to fill in the gaps digitally with a reasonable degree of accuracy.
Just as advances in DNA fingerprinting and ballistics have allowed leaps forward in criminal investigations, experts expect artificial intelligence to do the same (Credit: Alamy)
But the next part is particularly clever. Li has also trained an algorithm on photographs of people’s faces in order to find a face that would most closely fit the reconstructed skull beneath. When presented with an unidentified skull, the system creates thousands upon thousands of 3D reconstructions that it then searches through to find one that matches.
“We collect many photos from the internet and first try to reconstruct a 3D face [for each of them],” he explains. “Then we do a so-called superimposition to match this 3D face with the skull.”
For any regions of the 3D reconstructed face that don’t map perfectly to the subject’s skull, the system is able to re-draw them, modifying the face slightly to look more like the potential victim.
“It will be interesting to see how this approach could work when taking ageing into account going forward,” says Morgan. “A face can change quite significantly over time while the skull remains stable.”
Li says he now has a working system and is hoping that forensic anthropologists will trial it within a few months.
Fragments of bone can often be difficult to identify, but machine learning is offering ways of reconstructing a victims face from the remains of their skull (Credit: Getty Images)
There are still questions over how accurate many of these technologies will prove in the long-term. They may be faster and more useful in small-scale trials, but their true test will be when they are applied to real cases. Police forces will have to show not only that there are tangible benefits from adopting such systems, but also that they are legally and ethically sound.
“Anything we utilise has to be within a legal criminal justice framework,” notes Nick Baker, deputy chief constable at Staffordshire Police in the UK. “The courts need to accept it so the public will also accept it.”
AI in policing is largely entering a period of testing, after which its real capabilities and operational usefulness will be better known
A century ago, fingerprint evidence was becoming admissible in courts around the world – but it didn’t happen overnight. The first UK criminal trial in which fingerprint evidence convicted a person took place in 1902, but it would be another nine years before such evidence became admissible in US courts.
AI in policing is largely entering a period of testing, after which its real capabilities and operational usefulness will be better known. But, as Morgan notes, tools that speed up analysis and pull together data just waiting to be analysed are likely to have a huge influence on criminal investigations in the near future.
“It will be one of the things where in a few years we’ll look back and say, ‘Can you believe we didn’t have this five years ago?’” she says.
--
Join 900,000+ Future fans by liking us on Facebook, or follow us on Twitter or Instagram.
If you liked this story, sign up for the weekly bbc.com features newsletter, called “If You Only Read 6 Things This Week”. A handpicked selection of stories from BBC Future, Culture, Capital, and Travel, delivered to your inbox every Friday.