A popular prison cafe, which offers work experience serving the public for inmates, could face enforcement action.
Cafe Britannia opened at Norwich Prison five years ago, when it was deemed as an ancillary use of the prison, not requiring planning consent.
But the city council is reviewing its policy following complaints about its increased scale and popularity.
Britannia Enterprises, which runs the cafe, said it has worked on a way forward with the council.
The cafe is based in a prison property outside the main walls and it was previously an officers' mess for prison staff.
There was one complaint in 2014 when the building was converted but, at the time, council bosses said it was exempt from planning consent due to its scale and links to the prison.
Following further complaints, Norwich City Council's planning department revisited the case.
A report, which goes before the authority's planning committee on Thursday, said: "It was determined in 2016 that evidence from site visits and comments from neighbouring occupiers showed that a significant increase in activity from people entering and leaving the building via the access on Britannia Road occurred when the cafe opened."
The report said the council sought legal advice on the matter and had concluded the cafe was no longer ancillary to the prison, and consequently it would need planning permission to change its use.
The committee will decide whether enforcement action is needed.
The report said the committee has three options: to do nothing, close the cafe or allow it to continue with certain conditions, with officers recommending the latter.
The conditions include adding a new pedestrian entrance, cycle access and limiting opening hours to 07:30-22:00.
Britannia Enterprises said it was "confident" it was could comply with the city council's request.