Skip to content

News Center | Georgia Institute of Technology Georgia Institute of Technology

Search

Search form

  • Facebook
  • Twitter
  • YouTube
  • LinkedIn
  • News Home
  • Campus Map
  • Directory
  • Offices

News Center

Menu
Close
  • Calendar
  • Categories
    • Business and Economic Development
    • Campus and Community
    • Earth and Environment
    • Health and Medicine
    • Science and Technology
    • Society and Culture
    • Feature Stories
  • Media Contacts
  • Experts
    • Find an Expert
    • Featured Expert
  • Daily Digest
  • The Whistle
    • Home
    • Classifieds
    • Archives
  • Social Media
  • Subscribe
  • You are here:
  • GT Home
  • Georgia Tech NewsCenter
  • Home
  • Researchers create software for Google Glass that provides captions for hard-of-hearing users

Science and Technology

Researchers create software for Google Glass that provides captions for hard-of-hearing users

October 2, 2014 • Atlanta, GA

Captioning on Glass uses a companion Android phone app for everyday conversations. A hard-of-hearing person wears Glass while a second person speaks directly into a smartphone. The speech is converted to text, sent to Glass and displayed on the heads-up display.

A team of Georgia Institute of Technology researchers has created speech-to-text software for Google Glass that helps hard-of-hearing users with everyday conversations. A hard-of-hearing person wears Glass while a second person speaks directly into a smartphone. The speech is converted to text, sent to Glass and displayed on its heads-up display.

A group in Georgia Tech’s College of Computing created the Glassware when one of its own said he was having trouble hearing and thought Glass could help him.

“This system allows wearers like me to focus on the speaker’s lips and facial gestures," said School of Interactive Computing Professor Jim Foley. “If hard-of-hearing people understand the speech, the conversation can continue immediately without waiting for the caption. However, if I miss a word, I can glance at the transcription, get the word or two I need and get back into the conversation.”

Foley’s colleague, Professor Thad Starner, leads the Contextual Computing Group working on the project. He says using a smartphone with Glass has several benefits as compared to using Glass by itself.

“Glass has its own microphone, but it’s designed for the wearer,” said Starner, who is also a technical lead for Glass. “The mobile phone puts a microphone directly next to the speaker’s mouth, reducing background noise and helping to eliminate errors.”

Starner says the phone-to-Glass system is helpful because speakers are more likely to construct their sentences more clearly, avoiding “uhs” and “ums.” However, if captioning errors are sent to Glass, the smartphone software also allows the speaker to edit the mistakes, sending the changes to the person wearing the device.

"The smartphone uses the Android transcription API to convert the audio to text," said Jay Zuerndorfer, the Georgia Tech Computer Science graduate student who developed the software. "The text is then streamed to Glass in real time."

Captioning on Glass is currently available to install from MyGlass. More information and support can be found at the project website here.

Foley and the students are working with the Association of Late Deafened Adults in Atlanta to improve the program.

The same group is also working on a second project, Translation on Glass, that uses the same smartphone-Glass Bluetooth connection process to capture sentences spoken into the smartphone, translate them to another language and send them to Glass. The only difference is that the person wearing Glass, after reading the translation, can reply. The response is translated back to the original language on the smartphone. Two-way translations are currently available for English, Spanish, French, Russian, Korean and Japanese.

"For both uses, the person wearing Glass has to hand their smartphone to someone else to begin a conversation,” said Starner. “It’s not ideal for strangers, but we designed the program to be used among friends, trusted acquaintances or while making purchases.”

The group is working to get Translation on Glass ready for the public.

Additional Photos

  • Captioning on Glass phone display
     

    Captioning on Glass phone display

    Captioning on Glass allows a person to speak into a phone. The text is then displayed on Glass.

    Download Image
  • Captioning on Glass user display
     

    Captioning on Glass user display

    A hard-of-hearing person wears Glass while a second person speaks directly into a smartphone. The speech is converted to text, sent to Glass and displayed on its heads-up display. 

    Download Image
  • Jim Foley
     

    Jim Foley

    School of Interactive Computing Professor Jim Foley has trouble hearing and wanted an app that could help him. He and his team developed Captioning on Glass, which is now available on MyGlass.

     

    Download Image
  • Jay Zuerndorfer
     

    Jay Zuerndorfer

    Jay Zuerndorfer is a computer science graduate who helped create Captioning on Glass.

    Download Image

Related Links

  • Captioning on Glass Website
  • Install from MyGlass

Contact Information

Jason Maderer
National Media Relations
maderer@gatech.edu
404-385-2966

Categories

Science and Technology

News Categories

  • Business and Economic Development
  • Campus and Community
  • Earth and Environment
  • Health and Medicine
  • Science and Technology
  • Society and Culture

Expert Voices

James Wray

James Wray (Photo Fitrah Hamid)

More Hints of Life on Ancient Mars
James Wray
School of Earth and Atmospheric Sciences
Mariel Borowitz

Photo by Rob Felt

Half of Earth’s satellites restrict use of climate data
Mariel Borowitz
Sam Nunn School of Int'l Affairs

Featured Videos

Seth Osekre, a custodian in Facilities Management, follows safety protocols when cleaning. (Video by Allison Carter)

2020 graduate TJ Weiler talks about his journey to earning a Georgia Tech degree.

Georgia Tech Resources

  • Offices & Departments
  • News Center
  • Campus Calendar
  • Special Events
  • GreenBuzz
  • Institute Communications
  • Visitor Resources
  • Campus Visits
  • Directions to Campus
  • Visitor Parking Information
  • GTvisitor Wireless Network Information
  • Georgia Tech Global Learning Center
  • Georgia Tech Hotel & Conference Center
  • Barnes & Noble at Georgia Tech
  • Ferst Center for the Arts
  • Robert C. Williams Paper Museum

Colleges, Instructional Sites & Research

  • Colleges
  • College of Computing
  • College of Design
  • College of Engineering
  • College of Sciences
  • Ivan Allen College of Liberal Arts
  • Scheller College of Business
  • Instructional Sites
  • Georgia Tech-Lorraine
  • Georgia Tech-Savannah
  • Georgia Tech-Shenzhen
  • Georgia Tech Online
  • Professional Education
  • The Language Institute
  • Global Footprint
  • Global Engagement
  • Research
  • Georgia Tech Research Institute
  • Research at Georgia Tech
  • Executive Vice President for Research

Student & Parent Resources

  • Student Resources
  • Apply
  • BuzzPort
  • Buzzcard
  • Career Center
  • Co-ops & Internships
  • Commencement
  • Library
  • Student Life
  • Student Entrepreneurship
  • Study Abroad
  • T-Square
  • Parent Resources
  • Parent and Family Programs
  • Dean of Students
  • Scholarships & Financial Aid

Employee, Alumni, & Other Resources

  • Employees
  • Administration and Finance
  • Advising & Teaching
  • Faculty Affairs
  • Faculty Hiring
  • Human Resources
  • Office of the Provost
  • TechWorks
  • Alumni
  • Alumni Association
  • Alumni Career Services
  • Giving Back to Tech
  • Outreach
  • Startup Companies
  • Economic Development
  • Industry Engagement
  • Government & Community Partners
  • Professional Education
Map of News Center | Georgia Institute of Technology

Georgia Institute of Technology
North Avenue, Atlanta, GA 30332
Phone: (404) 894-2000

  • Contact Us
  • Site Feedback
  • Tech Lingo
  • Emergency Information
  • Legal & Privacy Information
  • Human Trafficking Notice
  • Accessibility
  • Accountability
  • Accreditation
  • Employment
Georgia Tech

© Georgia Institute of Technology