Lleyton Hewitt's tournament hit the buffers, however, Australia's remaining hope skidding out 7-5, 6-3, 6-3 to third seed Novak Djokovic.
While Wimbledon champion Williams and fourth seed Ivanovic made light work of their opposition, world number one Federer was again made to fight hard to defend his title.
Pushed to five sets on Saturday, he prevailed 6-4, 7-6, 6-3 against 13th seed Tomas Berdych but did not have things all his own way.
"He was playing better (than me) in the second set, honestly," Federer said in a courtside interview.
"He got an early lead, then played well in the breaker. In the end, maybe he made a wrong shot selection. It's unfortunate for him, but jeez, it's good for me to win in straight sets."
Next up for the Swiss is American James Blake who reached his first grand slam quarterfinal outside the United States with a solid 6-3, 6-4, 6-4 victory over Croatia's Marin Cilic.
While Federer put Saturday's epic behind him, Hewitt was unable to bounce back from his five-set third-round thriller. The former world number one took until 4:33am on Sunday morning to oust Marcos Baghdatis and failed to rouse himself for the Djokovic challenge.
Serbian Djokovic overcame a nervy start to clinch victory in two hours, 26 minutes and set up a last-eight clash with fifth seed David Ferrer of Spain.
"In the first set I was lucky, but then afterwards I relaxed and tried to be aggressive and take control of the match which I did," Djokovic said.
"Of course, Lleyton was very tired from the Baghdatis match a couple of nights ago, so I took my opportunities."
Ferrer through
Ferrer beat Juan Carlos Ferrero 7-5, 3-6, 6-4, 6-1 to reach the last eight.
"I beat Juan Carlos playing very well, it was a very tough match," Ferrer told reporters. "I am very happy. Now I am a little bit tired."
Djokovic's fellow Serb Ivanovic booked her quarterfinal with Venus by ousting Caroline Wozniacki 6-1, 7-6.
The world number three has not lost a set during the tournament and is relishing the prospect of playing the American. "She's definitely a tough opponent. Last two grand slams I lost to her. Exactly the time for some revenge," she smiled. "It will be important for me to serve well, and, yeah, stay in the court and don't give her too much."
Williams advanced with a routine 6-4 6-4 victory over talented Polish qualifier Marta Domachowska.
Agnieszka Radwanska became the first Polish woman to reach a grand slam quarterfinal when she came from a set and 3-0 down to beat an ailing Nadia Petrova.
Russian 14th seed Petrova led 6-1 3-0 but a groin strain hampered her movement in the third set and Radwanska stormed back to win 1-6 7-5 6-0 and set up a clash with ninth seed Daniela Hantuchova of Slovakia.
The 18-year-old Radwanska is the youngest of the eight quarterfinalists.
Slovak Hantuchova recovered from a slow start to beat Russian Maria Kirilenko 1-6 6-4 6-4.
Results (fourth round, prefix denotes seeding): Men’s singles: 3-Novak Djokovic (Serbia) bt 19-Lleyton Hewitt (Australia) 7-5, 6-3, 6-3; 5-David Ferrer (Spain) bt 22-Juan Carlos Ferrero (Spain) 7-5, 3-6, 6-4, 6-1; 1-Roger Federer (Switzerland) bt 13-Tomas Berdych (Czech Republic) 6-4, 7-6 (9-7), 6-3; 12-James Blake (US) bt Marin Cilic (Croatia) 6-3, 6-4, 6-4.
Women’s singles: 9-Daniela Hantuchova (Slovakia) bt 27-Maria Kirilenko (Russia) 1-6, 6-4, 6-4; 8-Venus Williams (US) bt Marta Domachowska (Poland) 6-4, 6-4; 29-Agnieszka Radwanska (Poland) bt 14-Nadia Petrova (Russia) 1-6, 7-5, 6-0; 4-Ana Ivanovic (Serbia) bt Caroline Wozniacki (Denmark) 6-1, 7-6 (7-2).