2021

It’s great to be back hosting our I/O Developers Conference this year. Pulling up to our Mountain View campus this morning, I felt a sense of normalcy for the first time in a long while. Of course, it’s not the same without our developer community here in person. COVID-19 has deeply affected our entire global community over the past year and continues to take a toll. Places such as Brazil, and my home country of India, are now going through their most difficult moments of the pandemic yet. Our thoughts are with everyone who has been affected by COVID and we are all hoping for better days ahead.

The last year has put a lot into perspective. At Google, it’s also given renewed purpose to our mission to organize the world’s information and make it universally accessible and useful. We continue to approach that mission with a singular goal: building a more helpful Google, for everyone. That means being helpful to people in the moments that matter and giving everyone the tools to increase their knowledge, success, health and happiness. 

Helping in moments that matter

Sometimes it’s about helping in big moments, like keeping 150 million students and educators learning virtually over the last year with Google Classroom. Other times it’s about helping in little moments that add up to big changes for everyone. For example, we’re introducing safer routing in Maps. This AI-powered capability in Maps can identify road, weather and traffic conditions where you are likely to brake suddenly; our aim is to reduce up to 100 million events like this every year. 

Reimagining the future of work

One of the biggest ways we can help is by reimagining the future of work. Over the last year, we’ve seen work transform in unprecedented ways, as offices and coworkers have been replaced by kitchen countertops and pets. Many companies, including ours, will continue to offer flexibility even when it’s safe to be in the same office again. Collaboration tools have never been more critical, and today we announced a new smart canvas experience in Google Workspace that enables even richer collaboration. 

At Google, we care about the energy use of our data centers. In fact, we were the first major company to be carbon-neutral way back in 2007, and we’ve been matching 100% of our annual electricity use with renewable energy purchases since 2017. But we want to go even further. By 2030, we plan to completely decarbonize our electricity use for every hour of every day. One way we can do this is by adjusting our operations in real time so that we get the most out of the clean energy that’s already available. 

And that’s exactly what our newest milestone in carbon-intelligent computing does: Google can now shift moveable compute tasks between different data centers, based on regional hourly carbon-free energy availability. This includes both variable sources of energy such as solar and wind, as well as “always-on” carbon-free energy such as our recently announced geothermal project. This moves us closer to our goal of operating on carbon-free energy everywhere, at all times, by 2030.  

Shifting compute tasks across location is a logical progression of our first step in carbon-aware computing, which was to shift compute across time. By enabling our data centers to shift flexible tasks to different times of the day, we were able to use more electricity when carbon-free energy sources like solar and wind are plentiful. Now, with our newest update, we’re also able to shift more electricity use to where carbon-free energy is available.

The amount of computing going on at any given data center varies across the world, increasing or decreasing throughout the day. Our carbon-intelligent platform uses day-ahead predictions of how heavily a given grid will be relying on carbon-intensive energy in order to shift computing across the globe, favoring regions where there’s more carbon-free electricity. The new platform does all this while still getting everything that needs to get done, done — meaning you can keep on streaming YouTube videos, uploading Photos, finding directions or whatever else.

We’re applying this first to our media processing efforts, which encodes, analyzes and processes millions of multimedia files like videos uploaded to YouTube, Photos and Drive. Like many computing jobs at Google, these can technically run in many places (of course, limitations like privacy laws apply). Now, Google’s global carbon-intelligent computing platform will increasingly reserve and use hourly compute capacity on the most clean grids available worldwide for these compute jobs — meaning it moves as much energy consumption as possible to times and places where energy is cleaner, minimizing carbon-intensive energy consumption.

Google Cloud’s developers and customers can also prioritize cleaner grids, and maximize the proportion of carbon-free energy that powers their apps by choosing regions with better carbon-free energy (CFE) scores.

To learn more, tune in to the livestream of our carbon-aware computing workshop on June 17 at 8:00 a.m PT. And for more information on our journey towards 24/7 carbon-free energy by 2030, read CEO Sundar Pichai’s latest blog post.

People love being together — to share, collaborate and connect.  And this past year, with limited travel and increased remote work, being together has never felt more important.

Through the years, we’ve built products to help people feel more connected. We’ve simplified email with Gmail, and made it easier to share what matters with Google Photos and be more productive with Google Meet. But while there have been advances in these and other communications tools over the years, they’re all a far cry from actually sitting down and talking face to face.

We looked at this as an important and unsolved problem. We asked ourselves: could we use technology to create the feeling of being together with someone, just like they’re actually there?

To solve this challenge, we’ve been working for a few years on Project Starline — a technology project that combines advances in hardware and software to enable friends, families and coworkers to feel together, even when they’re cities (or countries) apart.

Imagine looking through a sort of magic window, and through that window, you see another person, life-size and in three dimensions. You can talk naturally, gesture and make eye contact.

To make this experience possible, we are applying research in computer vision, machine learning, spatial audio and real-time compression. We’ve also developed a breakthrough light field display system that creates a sense of volume and depth that can be experienced without the need for additional glasses or headsets.

The effect is the feeling of a person sitting just across from you, like they are right there. 

Artificial intelligence (AI) has the potential to help clinicians care for patients and treat disease — from improving the screening process for breast cancer to helping detect tuberculosis more efficiently. When we combine these advances in AI with other technologies, like smartphone cameras, we can unlock new ways for people to stay better informed about their health, too.  

Today at  I/O, we shared a preview of an AI-powered dermatology assist tool that helps you understand what’s going on with issues related to your body’s largest organ: your skin, hair and nails. Using many of the same techniques that detect diabetic eye disease or lung cancer in CT scans, this tool gets you closer to identifying dermatologic issues — like a rash on your arm that’s bugging you — using your phone’s camera. 

How our AI-powered dermatology tool works 

Each year we see almost ten billion Google Searches related to skin, nail and hair issues. Two billion people worldwide suffer from dermatologic issues, but there’s a global shortage of specialists. While many people’s first step involves going to a Google Search bar, it can be difficult to describe what you’re seeing on your skin through words alone.

Our AI-powered dermatology assist tool is a web-based application that we hope to launch as a pilot later this year, to make it easier to figure out what might be going on with your skin. Once you launch the tool, simply use your phone’s camera to take three images of the skin, hair or nail concern from different angles. You’ll then be asked questions about your skin type, how long you’ve had the issue and other symptoms that help the tool narrow down the possibilities. The AI model analyzes this information and draws from its knowledge of 288 conditions to give you a list of possible matching conditions that you can then research further.

For each matching condition, the tool will show dermatologist-reviewed information and answers to commonly asked questions, along with similar matching images from the web. The tool is not intended to provide a diagnosis nor be a substitute for medical advice as many conditions require clinician review, in-person examination, or additional testing like a biopsy. Rather we hope it gives you access to authoritative information so you can make a more informed decision about your next step.

Today we’re sharing new AI research that aims to improve screening for one of the top causes of death worldwide: tuberculosis (TB). TB infects 10 million people per year and disproportionately affects people in low-to-middle-income countries. Diagnosing TB early is difficult because its symptoms can mimic those of common respiratory diseases.

Cost-effective screening, specifically chest X-rays, has been identified as one way to improve the screening process. However, experts aren’t always available to interpret results. That’s why the World Health Organization (WHO) recently recommended the use of computer-aided detection (CAD) for screening and triaging.

To help catch the disease early and work toward eventually eradicating it, Google researchers developed an AI-based tool that builds on our existing work in medical imaging to identify potential TB patients for follow-up testing. 

A deep learning system to detect active pulmonary tuberculosis  

In a new study released this week, we found that the right deep learning system can be used to accurately identify patients who are likely to have active TB based on their chest X-ray. By using this screening tool as a preliminary step before ordering a more expensive diagnostic test, our study showed that effective AI-powered screening could save up to 80% of the cost per positive TB case detected. 

Our AI-based tool was able to accurately detect active pulmonary TB cases with false-negative and false-positive detection rates that were similar to 14 radiologists. This accuracy was maintained even when examining patients who were HIV-positive, a population that is at higher risk of developing TB and is challenging to screen because their chest X-rays may differ from typical TB cases.

To make sure the model worked for patients from a wide range of races and ethnicities, we used de-identified data from nine countries to train the model and tested it on cases from five countries. These findings build on our previousresearch that showed AI can detect common issues like collapsed lungs, nodules or fractures in chest X-rays. 

Applying these findings in the real world

The AI system produces a number between 0 and 1 that indicates the risk of TB. For the system to be useful in a real-world setting, there needs to be agreement about what risk level indicates that patients should be recommended for additional testing. Calibrating this threshold can be time-consuming and expensive because administrators can only come to this number after running the system on hundreds of patients, testing these patients, and analyzing the results. 

Based on the performance of our model, our research suggests that any clinic could start from this default threshold and be confident that the model will perform similarly to radiologists, making it easier to deploy this technology. From there, clinics can adjust the threshold based on local needs and resources. For example, regions with fewer resources may use a higher cut-off point to reduce the number of follow-up tests needed. 

The path to eradicating tuberculosis

The WHO’s “The End TB Strategy” lays out the global efforts that are underway to dramatically reduce the incidence of tuberculosis in the coming decade. Because TB can remain pervasive in communities, even if a relatively low number of people have it at a given time, more and earlier screenings are critical to reducing its prevalence. 

We’ll keep contributing to these efforts — especially when it comes to research and development. Later this year, we plan to expand this work through two separate research studies with our partners, Apollo Hospitals in India and the Centre for Infectious Disease Research in Zambia (CIDRZ). 

Today, we’re sharing the  biggest update to Wear ever – built with your preferences in mind. We’ve been hard at work in three areas: building a unified platform with Samsung, delivering a new consumer experience and providing updates to your favorite Google apps.

A unified platform

Phones are at the center of our digital lives. When purchasing a phone these days, we’re buying not only a phone, but also an entire ecosystem of devices that are all expected to work together — such as TVs, laptops, cars and wearables like smartwatches or fitness trackers. In North America, the average person now has around eight connected devices, and by 2022, this is predicted to grow to 13 connected devices.

Today, we’re sharing how we’re helping make your Android phone, and all the devices connected to it, work even better together.

Pair your devices in one tap

Fast Pair helps make it easier and faster to connect to Bluetooth devices around you. So far, people have used Fast Pair over 36 million times to connect their Android phones with Bluetooth accessories from Sony, Microsoft, JBL, Philips, Google and many other popular brands.

In the coming months, we’re bringing Fast Pair to even more devices such as Beats headphones as well as cars from BMW and Ford. With a single tap, you can pair your Android phone to your favorite accessories whether it’s earbuds, speakers, wearables or cars.

Turn on your TV and find entertainment faster

Whether it’s under the couch cushions, behind your nightstand or in your refrigerator, TV remotes are often mysteriously lost. And even when you finally find it, typing a password with a remote control can be a frustrating and time-consuming process.

We’re making it easier to navigate your TV by building remote-control features directly into your Android phone, so you can watch your favorite show even if your actual remote is missing. And when you need to type a complex movie title or password, you can save time and use your phone’s keyboard to enter the text.

From the beginning, Android has always been about personalization and allowing you to select the device, service and experience that’s right for you. By providing an open ecosystem that gives you choice, Android has grown to more than 3 billion active devices around the world.

Android 12 builds on everything you love about Android, and focuses on building a deeply personal phone that adapts to you, developing an operating system that is secure by default and private by design, and making all your devices work better together.

Today, we’re releasing the first beta of Android 12, and giving you a look into some of the features that will be available in future releases.

We capture photos and videos so we can look back and remember. But having all your photos — of loved ones, screenshots, selfies — mixed together makes it hard to rediscover important moments. In fact, most of the 4 trillion photos stored in Google Photos are never viewed. 

To make it easier to look back, we’re using AI to power new features that resurface meaningful moments and bring your memories to life — while giving you control over what you relive. 

New types of memories, personalized to you

With Memories, you can already look back on important photos from years past, recent highlights, moments with your loved ones, your favorite activities and more. Using machine learning, we can now go beyond resurfacing photos based on themes to doing so based on not-so-obvious visual patterns in your photos. Starting later this summer, when we find a set of three or more photos that share things like shape or color, we’ll highlight these little patterns for you in your Memories. For example, one of our engineers received this collection featuring photos he snapped of his favorite orange backpack.

AI advancements push the boundaries of what Google products can do. Nowhere is this clearer than at the core of our mission to make information more accessible and useful for everyone.

We’ve spent more than two decades developing not just a better understanding of information on the web, but a better understanding of the world. Because when we understand information, we can make it more helpful — whether you’re a remote student learning a complex new subject, a caregiver looking for trusted information on COVID vaccines or a parent searching for the best route home.

Deeper understanding with MUM

One of the hardest problems for search engines today is helping you with complex tasks — like planning what to do on a family outing. These often require multiple searches to get the information you need. In fact, we find that it takes people eight searches on average to complete complex tasks.

With a new technology called Multitask Unified Model, or MUM, we’re able to better understand much more complex questions and needs, so in the future, it will require fewer searches to get things done. Like BERT, MUM is built on a Transformer architecture, but it’s 1,000 times more powerful and can multitask in order to unlock information in new ways. MUM not only understands language, but also generates it. It’s trained across 75 different languages and many different tasks at once, allowing it to develop a more comprehensive understanding of information and world knowledge than previous models. And MUM is multimodal, so it understands information across text and images and in the future, can expand to more modalities like video and audio.

Imagine a question like: “I’ve hiked Mt. Adams and now want to hike Mt. Fuji next fall, what should I do differently to prepare?” This would stump search engines today, but in the future, MUM could understand this complex task and generate a response, pointing to highly relevant results to dive deeper. We’ve already started internal pilots with MUM and are excited about its potential for improving Google products.

Information comes to life with Lens and AR

People come to Google to learn new things, and visuals can make all the difference. Google Lens lets you search what you see — from your camera, your photos or even your search bar. Today we’re seeing more than 3 billion searches with Lens every month, and an increasingly popular use case is learning. For example, many students might have schoolwork in a language they aren’t very familiar with. That’s why we’re updating the Translate filter in Lens so it’s easy to copy, listen to or search translated text, helping students access education content from the web in over 100 languages.

We want to help people discover, learn about and shop for the products they love — whether those products come from a big-box retailer, new direct-to-consumer brands or the mom-and-pop shop down the street. We’re supporting an open network of retailers and shoppers to help businesses get discovered and give people more options when they’re looking to buy. Two concrete steps we’ve taken to support discoverability for all merchants are eliminating commission fees and making it free for sellers on Google.

To show you the most relevant shopping information, we must have a deep understanding of the products that appear across Google and in the world around us — from images and videos to online reviews and inventory in local stores. That’s why today we shed some light on the technology behind our Shopping Graph: our most comprehensive, real-time dataset about products, inventory and merchants. 

The Shopping Graph is a dynamic, AI-enhanced model that understands a constantly-changing set of products, sellers, brands, reviews and most importantly, the product information and inventory data we receive from brands and retailers directly — as well as how those attributes relate to one another. With people shopping across Google more than a billion times a day, the Shopping Graph makes those sessions more helpful by connecting people with over 24 billion listings from millions of merchants across the web. It works in real-time so people can discover and shop for products that are available right now.