Google open-sources skin tone research to improve inclusivity

Google has been working with researchers to make its products and services more inclusive for people with darker skin tones. Now, the company is open sourcing a major part of that work. The company is making its skin tone research widely available as part of its effort at creating more ”responsible AI.” The research has so far resulted in the Monk Skin Tone Scale (MST), a scale “designed to be easy-to-use for development and evaluation of technology while representing a broader range of skin tones.”

The scale is meant to more accurately reflect the diversity of different skin tones, Google says, and was developed with Harvard professor Dr. Ellis Monk. The work will help AI more accurately “see” a wider range of skin tones, especially darker ones.

This research will be most apparent to users in Search results and in Google’s Photos app to start. For Search, Google is using the MST scale to surface results that are more inclusive of darker skin tones. For example, makeup-related searches will come with a filter for adjusting for different skin tones so users can find results that are most relevant for them.

Google is making Search results more inclusive to different skin tones.
Google

In Photos, Google is also using the MST scale to power a new set of “Real Tone filters.” According to Google, these filters are “designed to work well across skin tones” and “a wider assortment of looks.” 

Eventually, Google says it will incorporate the MST scale into more of its products and services. It’s also working to make it easier for brands, creators and publishers to label content to adapt to the scale so Search will be better able to surface results for different hair textures and colors too.

Google isn’t the first company to undertake this type of work. Pinterest has launched features to better detect diverse skin tones and different hair textures. Snapchat has also conducted research into making its camera more inclusive to darker skin tones. But with Google making its work open source, these kinds of advancements could become much more common.

Follow all of the news from Google I/O 2022 right here!

Google’s AI Test Kitchen lets you experiment with its natural language model

Google is announcing news at breakneck pace at its I/O developer conference today, and as usual it’s flexing its machine-learning smarts. In addition to unveiling its new LaMDA 2 conversational AI model, the company also showed off a new app called AI Test Kitchen. 

The app offers three demos that showcase what LaMDA 2 can do. The first is a simple brainstorm tool that asks the app to help you imagine if you were in various scenarios. During the keynote demo, Google entered “I’m at the deepest part of the ocean” as a response to the app’s prompt of “Imagine if.” The app then spit out a short paragraph describing the user in a submarine the Marianas Trench, with descriptive language.

Secondly, as a demonstration of the model being able to stay on topic, the app can have a conversation with you about something and understand context. During the demo, the app started by asking “Have you ever wondered why dogs like to play fetch so much?” In its responses to simple follow-ups like “Why is that,” the system replied with more information about dogs and their senses of smell. 

Finally, AI Test Kitchen shows how LaMDA 2 can “break down a complex goal or topic.” This section is called List It, and users can ask things like “I want to learn ukulele” or “I want to grow a garden.” LaMDA will generate lists of subtasks to help you get started, and according to Google, may even offer ideas you might not have thought of. In addition to giving you the names of vegetables you can grow, for example, AI Test Kitchen might also give you a set of steps to take or weather conditions to consider. During the demo, the app offered a tip for users with limited space, sharing the types of plants that might thrive in smaller gardens. 

According to CEO Sundar Pichai, Google is using this app in part to gather feedback on its new AI model. It will open up access “over the coming months, carefully assessing feedback from the broad range of stakeholders — from AI researchers and social scientists to human rights experts.” Pichai said these findings will be incorporated into future versions of LaMDA. He added that, over time, the company intends to “continue adding other emerging areas of AI into our AI Test Kitchen.”

Follow all of the news from Google I/O 2022 right here!

Google’s latest security upgrades include virtual credit cards

Google is using I/O 2022 to unveil (and flaunt) a host of privacy and security upgrades, including some significant features for online shopping. The company is introducing virtual payment cards on Android and Chrome that promise extra security by replacing the real card number with a digital counterpart. It should be faster, too, as you won’t have to enter the CVV or other details that frequently slow you down.

Virtual cards will be available in the US this summer for American Express, Visa and Capital One holders. Mastercard is due later in the year. This isn’t as ambitious a financial project as Google’s defunct Plex banking service, but it may be useful if you’re worried a hacker might scrape your payment details while you’re checking out.

Other additions are subtler, but potentially useful. Google now protects Workspace users against phishing and malware in Docs, Sheets and Slides, not just Gmail. You should also see the safety status in apps to let you know when your Google account is at risk.

Google is also making it easier to control data. On top of plans to let you remove contact details from search results (still in a months-long rollout), you’ll also have the option to see more or less of certain brands and categories in ads through My Ad Center. You won’t just be limited to blocking or reporting content.

The expansions come alongside ongoing efforts. Google is automatically enrolling users in two-factor authentication to reduce account hijacking. It’s also scaling back the volume of sensitive personal info, anonymizing that content and curbing access through technologies like end-to-end encryption and the secure enclaves on modern phones. Yes, Google is partly touting these features to counter long-running accusations of less-than-stellar privacy, but they might be welcome if you’re jittery about trusting the company with your data.

Follow all of the news from Google I/O 2022 right here!

Google makes its AI assistant more accessible with ‘Look and Talk’

Google Assistant is already pretty handy, filling in your payment info on take out orders, helping get the kids to school on time, controlling your stereo systems’ volume and your home’s smart light schedules. At its I/O 2022 keynote today, company executives showed off some of the new features arriving soon for the AI.

The first of these is “Look and Talk.” Instead of having to repeatedly start your requests to Assistant with “Hey Google,” this new feature relies on computer vision and voice matching to constantly pay attention to the user. As Sissie Hsiao, Google’s VP of Assistant, explained on stage, all the user has to do is look at their Nest Hub Max and state their request. Google is also developing a series of quick commands that users will be able to shout out without having to gaze longingly at their tablet screen or say “Hey Google” first — things like “turn on the lights” and “set a 10-minute alarm.”

asdf
Alphabet

All of the data captured in that interaction — specifically the user’s face and voice prints, used to verify the user — are processed locally on the Hub itself, Hsiao continued, and not shared with Google “or anyone else.” What’s more, you’ll have to specifically opt into the service before you can use it.

According to Hsiao, the backend of this process relies on a half-dozen machine learning models and 100 camera and mic inputs — i.e., proximity, head orientation and gaze direction — to ensure that the machine knows when you’re talking to it versus talking in front of it. The company also claims that it worked diligently to make sure that this system works for people across the full spectrum of human skin tones. 

Looking ahead, Google plans to continue refining its NLP models to further enhance the responsiveness and fidelity of Assistant’s responses by “building new, more powerful speech and language models that can understand the nuances of human speech,” Hsiao said. “Assistant will be able to better understand the imperfections of human speech without getting tripped up — including the pauses, ‘umms’ and interruptions — making your interactions feel much closer to a natural conversation.”

Follow all of the news from Google I/O 2022 right here!

Google brings transcripts and auto-translated captions to YouTube on mobile

Google is rolling out auto-translated video captions for YouTube on mobile devices, with support for 16 languages, the company announced during its I/O 2022 keynote today. The feature is live now. Additionally, YouTube video transcripts are now available to all Android and iOS users.

This is all part of Google’s work to make YouTube videos easier to navigate and search, building on existing features like auto-generated chapters. Google has a plan to increase the number of YouTube videos with auto-generated chapters from 8 million to 80 million by the end of the year. 

The updates to YouTube’s auto-translation features come after Google removed community captions in 2020, leaving some users high and dry, particularly in Japan and Korea. By 2021, however, Google was openly experimenting with auto-translation tools on YouTube.

Next month, Google plans to add auto-translated captions to Ukrainian YouTube content.

Follow all of the news from Google I/O 2022 right here!

Google’s Scene Exploration adds reviews and product details to searches in Lens

At its I/O 2022 keynote today, Google provided a number of updates regarding search including a powerful new feature called scene exploration. One new addition is support for Google Lens inside the default Google Search bar, which makes it easier to point your camera at an object and look it up on the web. However, to support more natural search across a range of inputs, the company is looking to enhance multisearch by making it easier to find info about local businesses with the new Near Me feature. 

That said, the big upcoming addition might actually be Scene Exploration, which uses your phone’s camera to condense info from places like cluttered store shelves and then surface it in a more digestible way. On stage, Google demoed this by showing a picture of chocolate bars in a store, before using Scene Exploration to call out additional info like the presence of nuts, user ratings and more. And in the future Scene exploration will even call out details like products that are minority-owned or come from sustainable farming methods. 

As part of its ongoing efforts to promote racial equity, Google is adding new skin tone filters to serarch.
Google

Additionally, as part of Google’s ongoing efforts to promote racial equity, the company is updating image search so that users can filter results based on specific skin tones. This should make it easier to find the right shade of makeup or an outfit that works best for you. And, later this month, Google will also add new real tone filters in Google photos, with Google also making Professor Monk’s skin tone scale available for open source collaboration. 

Meanwhile, in order to provide more context about the web pages you visit, Google is adding a new tab in the Google App that includes a description of the site sourced from both the author and other comments from across the net. 

Follow all of the news from Google I/O 2022 right here!

Google Maps adds an ‘Immersive View’ of major cities

Google Maps is getting an “Immersive View” that will offer users digitally rendered looks at major US cityscapes, Alphabet CEO Sundar Pichai told the audience at Google’s I/O 2022 keynote on Wednesday.  

The new feature uses computer vision and AI to blend Maps’ existing Street View function with aerial photography to create high-resolution models of the various buildings and urban features of a given location. “With our new immersive view, you’ll be able to experience what a neighborhood, landmark, restaurant or popular venue is like — and even feel like you’re right there before you ever set foot inside,” wrote Miriam Daniel, VP of Google Maps, in a blog post. What’s more, Maps’ other tools and features can be applied to the view as well, enabling users to see what the area looks like at different times of the day and varying weather conditions.

Immersive View will first be available for Los Angeles, London, New York, San Francisco and Tokyo later this year, with more cities to follow. The company also notes that its recently released eco-routing feature, which lets drivers in the US and Canada to pick the most fuel efficient route for their trip, has already been used to travel 86 billion miles and prevented the release of roughly half a million metric tons of carbon emissions.

Google isn’t the only company making its navigation systems more readable and user friendly. At WWDC 2021 last June, Apple rolled out a higher-fidelity version of its Maps app, offering added detail like elevation gradients, brighter road colors, more prominent location labels, and hundreds of custom icons for local landmarks.

Follow all of the news from Google I/O 2022 right here!

Google is bringing automatic summaries to Docs and Chat

Google is making it easy to catch up on long documents with a new auto summarization feature, which will soon be available on Google Docs. It relies on machine learning to break down the key points in a file and generate a readable briefing. Think of it like automatic Cliff’s Notes for all of those work reports you never read. Google originally announced the feature in March, but now it’s closer to reaching the public.

Google Docs Auto Summary
Google

As Alphabet CEO Sundar Pichai explained at today’s Google I/O keynote, the automatic summary feature relies on language understanding, information compression and natural language generation to work its magic. And while it’s coming to Docs soon, he says it’ll eventually make its way to Google Spaces and Chats, where it can summarize your conversations so you don’t have to spend hours scrolling up. If anything, it’ll be a nice way to keep tabs on those group chats you’re lurking in. (And hopefully automatic summaries will work better than Google’s AI-powered inclusive language warnings.)

As for other Google Workplace updates, the company is introducing a Portrait Restore feature to clean up crummy webcam video in Google Meet, and it showed off a customizable way to simulate studio portrait lighting. It’s also using AI to de-reverberate sound, which makes it seem like you’re talking into a high-quality microphone, and bringing automatic transcriptions to Google Meet later this year.   

Follow all of the news from Google I/O 2022 right here!

Google Translate adds support for 24 new languages

Google is adding support for 24 new languages to its Translate tool, the company announced today during its I/O 2022 developer conference. Among the newly available languages are Sanskrit, Tsongae and Sorani Kurdish. One of the new additions, Assamese, is used by approximately 25 million people in Northeast India. Another, Dhivehi, is spoken by about 300,000 people in the Maldives. 

According to Google CEO Sundar Pichai, the expansion allows the company to cover languages spoken by more than 300 million people and brings the total number of languages supported by Translate to 133. Pichai credited the breakthrough to a new monolingual AI learning approach where Google’s translation algorithm learns how to translate a section of text without first seeing a sample. In a blog post published during the event, Google admitted the approach isn’t perfect yet, but said it would keep working on the technology so that it can deliver the same experience it does with languages like Spanish and German.      

The last time Google added a significant number of languages to Translate was 2020 when the company updated the tool to support Kinyarwanda, Odia, Tata, Turkmen and Uyghur. Google still has its work cut out for itself. It’s estimated there are more than 7,151 spoken languages globally. Still, today’s additions may help people communicate in situations where they otherwise couldn’t.   

Follow all of the news from Google I/O 2022 right here!