With prototypes of driverless cars hitting the road around the world, one can imagine a perhaps not-so-distant future where machines will replace humans behind the wheel. But before driverless cars can make it to a driveway near you, carmakers and tech firms will need to develop systems that can react to the complex dynamics on the road, and predict the (sometimes unpredictable) behaviour of other drivers and pedestrians.
University of Ottawa professor Robert Laganière, of the School of Electrical Engineering and Computer Science, and PhD student Rytis Verbickas have developed a new algorithm to detect pedestrians which harnesses the power of deep learning and opens the door to cheaper, more efficient systems to guide driverless cars.
While existing systems for autonomous vehicles require a wide array of expensive sensors and radars, the new system developed by Laganière and Verbickas relies on a far more familiar technology: cameras. “Cameras are simple to install and use, while being small, energy-efficient and affordable,” explains Verbickas. “With our new algorithm, the forward-facing camera of a car is turned into a sensor with a grid of cells that react to the presence of pedestrians within their field of view.”
The “deep learning” process used to teach the car to recognize and react to pedestrians involves repeatedly showing the model images and telling it where pedestrians are located within those images. Over time, the model builds an internal representation — or “learning” — of what pedestrians look like and how they behave. The more images are shown to the system, the better it analyzes images it has never seen before.
“The idea is to be able to detect pedestrians at a great enough distance that an autonomous vehicle will have time to decide the best course of action,” says Laganière. “The exploit here is that we can not only detect pedestrians at a high level of accuracy, but also with a model that runs on the low-power processors that will equip the next generation of intelligent cars.”
In addition to forgoing the bulky and generally expensive sensors used by driverless cars, which can cost over $10,000 each, the new system developed by Laganière and Verbickas requires considerably less power and memory. “Existing systems that use deep learning tend to draw significant amounts of power — 10 watts or more — while consuming hundreds of megabytes of memory,” says Verbickas. “By comparison, our model, applied to next-generation automotive processors, consumes less than two watts of power and is roughly the same size as an MP3 file containing a single song, or about four megabytes.”
The team is working with NXP Semiconductors’ Ottawa Technology Center, focused on the design of Processors for Vision and Artificial Intelligence, to apply its model to specialized automotive processors and analyze incoming camera data. The technology could be used for autonomous driving or collision warning systems by applying a decision-making logic to the incoming data.
Laganière and Verbickas are also looking to “train” their model to recognize pedestrians of different sizes and detect other cars and buses, and to test their model’s performance under changing weather, lighting levels and image quality. “It’s interesting to note that our system was originally ‘trained’ using pedestrian images captured in California,” says Laganière. “When we ran our cameras along the uOttawa campus, it was the first time it saw pedestrians with winter coats and hats.”
The University of Ottawa—A crossroads of cultures and ideas
The University of Ottawa is home to over 50,000 students, faculty and staff, who live, work and study in both French and English. Our campus is a crossroads of cultures and ideas, where bold minds come together to inspire game-changing ideas. We are one of Canada’s top 10 research universities—our professors and researchers explore new approaches to today’s challenges. One of a handful of Canadian universities ranked among the top 200 in the world, we attract exceptional thinkers and welcome diverse perspectives from across the globe.
Media Relations Officer