"You either fight them or not... The Pakistani government has to understand you can't have halfway measures with terrorists," former National Security Advisor Stephen Headley said.
Headley said ISI's links with terrorism has been an issue between the US and Pakistan ever since former President Pervez Musharraf's regime.
"When President Musharraf nine years ago was told you need to break with the Taliban and join us with the war on terror, he said he would and he did largely.
"But for nine years there's always been this suspicion -- more suspicion than hard evidence but some hard evidence -- that there are elements within the Pakistani security services that are maintaining some kind of relationship," he said.
Pakistan joined the US in its war in Afghanistan in 2001 and has been an ally but ISI's long time links with Taliban and other militant groups in Afghanistan have occasionally led to differences between the two countries.
"... and it is an issue that the US does and must continue to push the Pakistani government," Headley said.
He said US-Pak ties have always been difficult to manage, and they go from time to time periodic crisis.
He said the issue of whether Pakistan is continuing to give "quiet support" to some of the extremist groups in Afghanistan will be a crucial issue.
"Secondly, this issue that they're willing to allow the US to do some things in Pakistani territory - these much reported press reports of the predator attacks -- but doing something operationally with our forces across the border.
"That's a red line and of course that's what was implicated on this helicopter incident here of last week," he said admitting there have been tensions which have to be managed. Headley said based on past experiences he hoped the US will get through this latest period of tension.
Responding to a question as to who is running the show in Pakistan – the democratic elected government or the military, he said it is actually a coalition of the two.
"Well you know there is a civilian government but as I said it is highly dependent on the military. Does the military have influence? Absolutely right. Does the military want to take over again in as they did in the Musharraf era? Clearly not," he said.
He said in the present situation the military wants the democratic government of Pakistan to succeed but the government continues to be dependent on the military for the humanitarian relief and for the fighting terrorists.