Justice chiefs have dismissed as “rumours” fears that a prison visitors centre was to be set up in the middle of a Suffolk housing estate.

Concerned villagers claim to have heard of plans to turn Hollesley Bay open prison’s former social club for officers in Oak Hill into a centre for inmates to meet with relatives.

The old social club was built amid a group of houses, which were once owned by the prison for staff, but have since been sold off as open market housing.

Joel Stone, who lives in Oak Hill, said people in the estate were concerned by the prospect of the prison “bringing convicts up to our residential area”.

“We’ve had enough of this prison riding rough shod over us,” he added. “The residents of Oak Hill are fuming.”

When asked about the possible plans for the old social club, a Prison Service spokesman said there were “no plans” for a visitors centre and insisted any comments otherwise were just “rumours”.

Mr Stone said he was not assured by the comments, particularly given the Prison Service’s handling of recent plans to bring sex offenders to Hollesley Bay. “They haven’t exactly got a reputation for being honest and up front,” he said.

Mr Stone has been working on the campaign to reverse the decision for sex offenders to be accommodated in the open prison.

The Prison Service confirmed this summer Hollesley Bay was among several open prisons that would be used to accommodate the growing number of sex offenders in the criminal justice system.

Despite assurances offenders would be “suitably risk assessed”, the proposals prompted a major backlash.

Mr Stone has set up a petition to Government to reverse the decision and is rallying the villagers for a march,

Meanwhile, Patrick Jacob, another villager, has written an open letter to Suffolk Coastal MP Therese Coffey asking for her support in getting the decision overturned. Dr Coffey said the letter raised “some legitimate concerns” which she has the Prison’s Minister to respond to.

The Prison Service said the changes were necessary as more sex offenders were going to jail. “A wider range of rehabilitation options for these men as they prepare to re-enter society is the best way of minimising the risk to the public,” a spokesman added