Forty-nine per cent of those who responded to a survey by the Asahi daily said Kan should step down immediately.
Since taking office last June, public support for Kan Japan's fifth leader in less than four years has quickly evaporated, with voters citing a lack of leadership.
Recent polls showed support for Kan's Cabinet plummeting to below 20 per cent, raising questions about how long he may last.
A separate survey by the Mainichi daily showed public support for Kan dropping to an all-time low of 19 percent, down 10 percentage points from a month ago.
In the Mainichi survey, around 34 per cent of respondents said Kan lacked leadership, while another 34 per cent said their expectations of the prime minister were low.
The results underline deepening public frustration with the government and raise questions about Kan's political longevity.
They cast a pall over Kan's ability to formulate and implement measures to tackle Japan's pressing problems, including a lackluster economy, a rapidly aging population and a ballooning public debt.
The Asahi said its telephone survey covered 3,484 randomly selected households. It received valid responses from 2,082 individuals on Saturday and yesterday.
No margin of error was given, but a poll of that size would generally have a margin of error of about three percentage points.
The Mainichi said its telephone survey covered 1,492 randomly selected households. It received valid responses from 989 individuals on Saturday and yesterday.
The paper gave no margin of error, but a poll of that size would generally have a margin of error of about five percentage points.