Photo: Flickr user NASA Commons
A hurricane hits a shore town. What is the estimated property damage? A city is doing an inventory of trees. How many are there? An aid group is trying to get food to an impoverished rural population. What's the best location to make a drop?
Answering those and myriad other questions about our planet usually takes painstaking boots-on-the ground work. But satellite imagery and advanced machine learning, aka deep learning, is changing all that.
Anyone who’s ever toyed around with Google Earth knows the eyes in the sky can capture amazingly detailed images of what's on the ground. The new AI is now teaching those orbiting cameras what they're seeing.
"The challenge is, how do you convert all those pixels into meaningful information?" says Shay Har-Noy, vice president and general manager of platform at satellite imagery provider DigitalGlobe. And that's exactly what the Colorado-based company is doing.
DigitalGlobe collects about 73 terabytes of satellite images every day, adding to a 16-year running archive of photos of locations around the world.
Each of the above images contains a pool inside the pink polygon. Note the variability of color, location, size, and visibility. PoolNet is a classifier implemented on GBDX that relies on a convolutional neural network and vast amounts of crowdsourced training data to distinguish properties that contain swimming pools.[Photo: DigitalGlobe]
DigitalGlobe has developed a "geospatial big-data" platform it calls GBDX that trains computers to automatically detect relevant objects in those satellite images. Using deep learning, a technique that harnesses large sets of data to develop multilayered mathematical structures to classify images, audio, or text, DigitalGlobe's cloud-based systems can accurately pick out and analyze points of interest in the photos without a great degree of human effort.
That has let the company reach new customers who are interested in accessing structured geographical data they can work with in tools like Excel, but aren't interested in manually sifting through huge sets of raw images. Potential users range from insurers and investors looking to understand real estate development patterns, to aid organizations aiming to serve populations in sparsely mapped areas of the globe.
Airplane recognition results at Beijing Capital International Airport.[Photo: DigitalGlobe]
Among DigitalGlobe's new customers is PSMA Australia, a consortium of Australian federal, state, and territorial agencies working with the company to build a national database of information about man-made structures across Australia. The database, called Geoscape, saw its first data release last month and is ultimately slated to include satellite-imagery-derived information about more than 20 million structures across the continent.
The GBDX platform relies on the Amazon Web Services cloud to store DigitalGlobe’s enormous collection of data—the company was among the first to use Amazon’s Snowmobile service, which uses a shipping container filled with data storage equipment to transfer huge volumes of information to Amazon’s servers. The platform enables developers to process that image data with DigitalGlobe’s machine learning algorithms or their own, either for their own use or for resale to their own customers interested in the results.
The platform has also been used for processing data in the wake of natural disasters. After Hurricane Matthew struck Haiti and the southeastern US last fall, DigitalGlobe was able to detect areas covered by water before and after the hurricane and combine those regions with open-source data from OpenStreetMap to determine where roads and buildings were submerged or at heightened risk of flooding.
DigitalGlobe also maintains crowdsourcing networks of people who can quickly hand-label images after a disaster, letting aid workers know where to look for damaged buildings and people potentially in need of assistance. That technique helped the company provide guidance to nonprofits and first responders after the 2015 earthquake in Nepal, Har-Noy says.
"We were the very first company to capture imagery of Kathmandu, which had a very large amount of damage," he says. "What we were able to do is capture imagery, run it by these crowdsourcing detections in order to detect rubble, destroyed buildings, etc., and we were able to make it accessible to various NGOs that were being deployed on the ground."
Crowdsourcing can also help with training and testing machine-learning algorithms, since human workers from around the world can be paid to quickly label features on satellite images or verify labels applied by machines, he says.
And while both crowdsourcing and automation let DigitalGlobe and its customers extract more detail from satellite images than ever before, Har-Noy says the company takes steps to protect people’s privacy and safety. The resolution of commercial satellite images, regulated by the U.S. Commerce Department, means people aren’t recognizable at the level of detail the company releases, Har-Noy says. DigitalGlobe also doesn’t release images of active US combat areas.
"We take privacy very seriously with regard to the information we release, we make accessible, and also that other people derive," Har-Noy says.
Still, machine learning’s apparent success in understanding satellite images raises questions about how automated processing of higher-resolution photos, like those from aerial photography, may impact privacy in the future, says Jay Stanley, senior policy analyst at the American Civil Liberties Union.
"This is part of a larger trend, which is that machine learning and artificial intelligence techniques in general are allowing monitoring to take place on a mass scale that used to be very expensive because it required human attention," he says.
As wide-area aerial imaging, such as the controversial plane-based surveillance reportedly deployed in Baltimore last year and the balloon-mounted cameras used at last year's Summer Olympics in Rio de Janeiro, potentially becomes more common, lawmakers and the public will have to decide appropriate limits on how the technology and data are used and automatically processed, Stanley says.
"These are decisions we’re going to have to make as a society if we want to go down the road of allowing us to be monitored by machine," he says. "If you start adding more advanced analytics to a tool like that, you could do some very invasive things."
Steven Melendez, FastCompany.com