First half goals from Kerstin Garefrekes and Okoyino da Mbabi put the hosts ahead before Canada captain Christine Sinclair cut the deficit late in the game with a well-struck freekick.
In the other game on Sunday, France beat Nigeria 1-0 in Group A. "We had a good start to the game but started combining badly towards the end of the first half," Germany coach Silvia Neid told reporters.
"In the second half we created a lot of chances but somehow we forgot how to score. Then Canada came back but we have now won the game and I am very happy with the start to the tournament."
In front of a 73,680 sellout crowd and with German President Christian Wullf, Chancellor Angela Merkel and FIFA President Sepp Blatter in the stands of the Olympic stadium, Germany, who won the 2007 title without conceding a goal, were initially on the back foot.
Canada, one of the tournament's dark horses, tried to sucker-punch them with quick attacks early on but unmarked Sinclair missed a gilt-edged chance on six minutes, firing wide from 12 metres.
Germany, who won the 2003 and 2007 tournaments, struck after 12 minutes with Garefrekes, who seconds earlier had failed to score form close range, rising above her marker to head in a Babett Peter cross at the far post and wrong-foot keeper Erin McLeod.
McLeod was kept busy throughout as Canada's midfield caved in, doing well to snatch the ball off Germany captain Birgit Prinz as the 33-year-old was about to tap in.
But McLeod stayed far too long on her line when Okoyino da Mbabi raced clear, giving the striker enough time to slot in three minutes before the break.
Canada found no way forward in the second half and needed a a 22-metre curled free-kick from Sinclair to pull one back on 82 minutes as Germany held on for victory.
"It is difficult to keep up with them (Germans)," said Canada midfielder Sophie Schmidt.
