Improving Mobile App Accessibility with Icon Detection

3 · Google AI Research · Jan. 28, 2021, 6:46 p.m.
Posted by Gilles Baechler and Srinivas Sunkara, Software Engineers, Google Research Voice Access enables users to control their Android device hands free, using only verbal commands. In order to function properly, it needs on-screen user interface (UI) elements to have reliable accessibility labels, which are provided to the operating system’s accessibility services via the accessibility tree. Unfortunately, in many apps, adequate labels aren’t always available for UI elements, e.g. images and i...