Boris Johnson says he will meet a group of families bereaved by coronavirus when their "litigation" with the government has "concluded".
But Covid-19 Bereaved Families for Justice say it is "simply not true" they are in litigation with the government and it is a "poor excuse".
They have repeated their demand for a meeting with the PM to discuss their call for an inquiry into the pandemic.
Downing Street said it had nothing to add to the PM's comments.
Covid-19 Bereaved Families for Justice say they have written five times to Mr Johnson to request a meeting.
Asked about the letters, in a live TV interview last week, the prime minister said he would "of course" meet anyone bereaved by Covid-19.
But days later he wrote to the group to say he was "unable" to meet them.
The group, which says it represents 1,600 families, said it was "devastated" to receive the letter, which it has made public.
At Prime Minister's Questions, Labour Leader Sir Keir Starmer - who met the group in July - asked if Mr Johnson would change his mind, saying the PM "will understand the frustration and the hurt of those families that he said one thing to camera and another thing to them".
Mr Johnson said: "Of course I would be very happy to meet the families of the bereaved and sympathise deeply with all those who have lost loved ones throughout this pandemic and we all feel their pain and their grief.
"It turns out that this particular group are currently in litigation with the government. I will certainly meet them once that litigation is concluded."
Elkan Abrahamson, a lawyer for Covid Bereaved Families for Justice, said it was "simply not true" that the group is in litigation with the government.
The group is considering seeking a judicial review of the decision not to hold an immediate inquiry into the pandemic.
On 26 August, it sent the government's legal department a "letter before action" calling for a meeting with the PM, which says "the families wish to resolve these issues without the need to resort to litigation".
Mr Abrahamson said the group had yet to receive a reply to this letter.
"Even if there was litigation parties are always encouraged to meet in person to try and resolve their differences so I don't really see that as an excuse," he added.
In his letter to the group, Mr Johnson said that because they had instructed solicitors, and were "engaged in pre-action correspondence", all further correspondence on the issue "should pass through the respective legal teams".
"I understand that the Government Legal Department will be responding to your most recent letter on this topic within the timeframe you requested," he added.
Jo Goodman, co-founder of Covid-19 Bereaved Families for Justice, who lost her dad Stuart to the virus, said: "It feels like we're the wrong kind of bereaved people - like the prime minister only wants to meet with bereaved people who won't ask difficult questions.
"There's no legal action happening, so that can't be the reason he's not meeting with us.
"So will he now commit to meet with us so we can share our experiences of serious issues - from deaths in care homes to inadequate protective equipment - that can help him learn crucial lessons and save lives?"
Covid-19 Bereaved Families for Justice is calling for an independent, judge-led, statutory public inquiry into coronavirus deaths, which it wants to begin imminently.
A Downing Street spokesperson said earlier that the PM remained committed to meeting people who have been bereaved as a result of Covid-19, and that he was "resolute in his determination to beat this virus" to prevent further "dreadful loss".
In July the prime minister committed to an independent inquiry into the pandemic, but has repeatedly said now was not the right time.
What could an inquiry look like?
Independent inquiries can take many forms - from full public inquiries that can take years, or in some cases decades, and cost millions of pounds; to smaller scale, more fleet-footed investigations.
The idea is to hold the powerful to account and try to learn lessons from decisions that have gone wrong, and how to avoid repeating scandals and tragic events in the future.
Recent examples of judge-led inquiries include the Leveson inquiry into media standards, or the ongoing inquiry into the Grenfell Tower catastrophe.
Other inquiries, such as 2009 Iraq inquiry, headed by a retired senior civil servant Sir John Chilcot, do not take evidence under oath.
The government of the day is normally expected to adopt many, if not all, of the recommendations of an official inquiry, although it does not always work out like that in practice.