"His continued presence is the biggest overhang on Microsoft's stock," Einhorn said in reference to Ballmer.
The comments by outspoken Einhorn, who made his name warning about Lehman Brothers' financial health before the investment bank's collapse, are the most pointed yet from a high-profile investor against Microsoft's leadership.
Microsoft shares, which have been static for over a decade, gained 0.87 percent in after-hours trading after Einhorn's comments, the most of any Dow Jones industrial average component.
The software giant, which was the largest U.S. company by market value in the late 1990s, has since been overtaken by Apple Inc and IBM in market value, and is no longer seen as a dominating force in technology after a failure to capitalize on new Internet and mobile computing markets.
The stock is down 6 percent in the last two weeks alone after Microsoft agreed to pay $8.5 billion for Internet phone service Skype, a move which mystified many investors.
Speaking at the annual Ira Sohn Investment Research Conference in New York on Wednesday, Einhorn said it was time for Ballmer -- who succeeded co-founder Bill Gates in 2000 -- to step aside and "give someone else a chance."
Einhorn's comments echo what some investors have said for some years in private.
A Microsoft spokesman declined comment on Einhorn's remarks.
RECENT BUYER
Einhorn's Greenlight Capital hedge fund has been a recent buyer of Microsoft stock, which at under 10 times expected earnings is regarded by many as undervalued.
Greenlight held about 9 million shares in Microsoft, or 0.11 percent of the company's outstanding shares, at the end of the first quarter, according to Thomson Reuters data.
Einhorn also said it was time for Microsoft to consider strategic alternatives for its money-losing online business, which has so far failed to win share from online search leader Google Inc.
The online services unit, which runs the Bing search engine and MSN web portal, had a loss of $726 million last quarter and has now lost $7 billion in four years.
Bing has made some progress, raising its U.S. Internet search market share to 14 percent from 8 percent in the two years since launch, but has not taken any share from Google, which has held on to its 65 percent share, according to research firm comScore.
Einhorn declined to comment further.
OLD FOES APPLE, IBM REVIVED
On Tuesday, Microsoft was overtaken by IBM in market value for the first time in 15 years, chiefly because of Microsoft's static share price. Apple roared past it last year to become the world's most valuable tech company.
An investor who put $100,000 into Microsoft stock 10 years ago would now have about $69,000 worth.
Einhorn, the president of Greenlight Capital, which had $7.8 billion of assets as of Jan. 1, made his name with the prescient call on Lehman's accounting troubles.
In the spring of 2008, Einhorn said Lehman -- and its then-Chief Financial Officer Erin Callan -- had understated its own problems and needed to raise capital to support a balance sheet peppered with risky assets.
Einhorn's public speeches on the matter in April and May 2008 -- including one at the Ira Sohn conference that year -- touched a nerve with other investors and are widely credited as leading to Callan's departure from the company a few months before its collapse.
Microsoft shares, which gained 4 cents in normal trading, ended up a further 12 cents at $24.31 in after-hours activity.
