EECS professor Alexei Efros (Photo by Noah Berger)The big picture
This past fall, in the wide-open spaces across campus, students from Alexei Efros’s computational photography class were easy to spot—they were the ones taking camera obscura pictures with image-capturing devices made from shoeboxes. The point, says Efros, who joined the Berkeley faculty last year as an associate professor in electrical engineering and computer sciences, was for students “to become one with the photons.”
Efros is part of the Visual Computing Lab. His research combines computer graphics and computer vision to investigate larger concepts related to the visual aspects of artificial intelligence. “It’s a multidisciplinary problem connecting philosophy, neuroscience and developmental psychology.” Efros says. “It really goes to the heart of who we are.”
The Internet is awash with visual data. Given our current ability to access and process information, most of that data is inaccessible. “I hope to make visual data a first-class citizen,” says Efros. “Right now the transfer of information is tied to language. I love literature, but there is much more to the visual world. There are so many things that we just don’t have the words for.”
Finding the Internet cat
The AverageExplorer is a tool designed to better search digital visual data. The new software, developed along with doctoral student Jun-Yan Zhu and former post-doctoral researcher Yong Jae Lee, is one of EECS professor Alexei Efros’s latest projects.
“We have this enormous collection of images on the web, but much of it remains unseen by humans. People have called it the ‘dark matter’ of the Internet. We wanted to figure out a way to quickly visualize this data by systematically ‘averaging’ the images,” Efros says about the project.
Unlike text-based data that can be organized and searched relatively easily, no intuitive system exists for indexing visual information. AverageExplorer addresses that problem by compiling millions of images into one average image. Users can then refine the search by
adding visual constraints to the query using basic image-editing tools, such as strokes, brushes and warps.
For example, to find images of a particular species of cat from among the vast sea of Internet cat images, a user starts with a massive compilation. With AverageExplorer tools they can then narrow down the results based on specific visual characteristics, such as color or ear length, yielding a new average image.
Beyond cats, potential applications for the AverageExplorer include online shopping and further refining computer vision and computer graphics systems based on data collected from users.
Using computation like cheesecloth, computer-vision researchers filter massive amounts of data into digestible nuggets of information. “To understand our visual world you need to have a lot of data,” Efros says. “You can’t understand it just with equations; you need to have the data because our world is so rich and there is so much entropy in it.”
Efros balances data-heavy computer vision research with creative computer graphics work. “You get to hack, code and play with computers. What comes out on the other end are often beautiful—sometimes bizarre, sometimes intriguing—visual representations, visual narratives even,” he says. “To me, that is very appealing aesthetically.”
Being creative while wrestling with big research questions is a priority for Efros. He once contemplated pursuing a career as a theater director. He hopes more art students will take his computational photography class in the future.
Efros’s research is also informed by his poor eyesight. “For me, this fascination for large amounts of data definitely comes from personal experience—there is this incredible power that prior experience has on what you perceive. If I have been to a place many times, then I think that I’m seeing way more than I’m actually seeing. In a familiar environment, my brain fills in the details,” he says.
“I realized how having lots of data makes all of the difference, and this is true for everyone, not just for people with poor sight.”