The decision came at a meeting of senior leaders at party president Nitin Gadkari's residence to discuss the crisis in Karnataka.
Though there was no official word from the party, BJP sources said there was little option left but to remove Yeddyurappa as his continuance in office was taking the sting out of the party's attack on the UPA government in parliament on corruption.
Senior party leaders L K Advani, Sushma Swaraj, Arun Jaitely and Rajnath Singh were present at the meeting.
Yeddyurappa was on Friday summoned to Delhi by Gadkari to explain his position to the central leadership.
The BJP Karnataka core committee including state unit president KS Vishwarappa, his predecessor Sadanand Gowda, Rural Development and Panchayati Raj Minister Jagdish Shettar also attended the meeting along with senior central leaders Arun Jaitley and Anant Kumar.
After the meeting, Jaitley had told reporters that the central leadership has heard the views of Yeddurappa and other Karnataka leaders and would take a decision after due deliberations with senior leaders L K Advani and others.