Two of the PM's most senior colleagues have told David Cameron to accept the failure of the government's manifesto pledge to reduce migration into the UK.
Michael Gove and Boris Johnson said the pledge was "corrosive of public trust" while Britain remained in the EU.
But Number 10 said their claim was an "attempt to distract" from the fact that a Brexit would be "disastrous".
It comes as a survey suggests that nine in 10 of the UK's top economists say leaving the EU would be damaging.
In an open letter to Mr Cameron, published in The Sunday Times, Mr Gove and Mr Johnson said the migration pledge had become unachievable.
They said: "Voters were promised repeatedly at elections that net migration could be cut to tens of thousands.
"This promise is plainly not achievable as long as the UK is a member of the EU and the failure to keep it is corrosive of public trust in politics."
They also said they were "particularly concerned about the impact of free movement in the future on public services".
"Class sizes will raise and waiting lists will lengthen if we don't tackle free movement," they wrote.
Net migration — the difference between the number of people coming to the UK for at least a year and those leaving — rose to 333,000 in 2015, according to Office for National Statistics estimates. The government is aiming to cut this figure to under 100,000.
The figure for EU-only net migration was 184,000, equalling its record high, and 188,000 for non-EU.
In response to the letter, Number 10 said: "This is a transparent attempt to distract from the fact that the overwhelming majority of economists and businesses believe leaving the single market would be disastrous for jobs, prices and opportunities for people."
Nine in 10 of the UK's top economists agree that leaving the EU would damage the country's economy, according to a survey published in the Observer.
According to an IPSOS Mori poll of more than 600 economists, 88% of those who replied said leaving the EU single market would damage Britain's growth prospects over the next five years.
Britain Stronger in Europe campaign director Will Straw said: "This is the final nail in the coffin of the Leave campaign's economic credibility.
"It is becoming clear that leaving is a risk we simply cannot afford to take."
It comes as former Conservative Prime Minister Sir John Major criticised the Vote Leave campaign in the Mail on Sunday.
He wrote: "As the referendum vote comes nearer, I again ask the senior figures of Vote Leave to correct the inaccuracies and falsehoods they are peddling to the British people."
On the issue of immigration from Europe, he said the Leave campaign "seems more focused on raising fears than setting out facts".
"Their cavalier exaggeration of likely immigration flows has been the most distasteful aspect of this referendum campaign," he said.
"Vote Leave has consistently failed to tell us how they see the UK outside Europe. They have glib slogans, but no solid detail."