Government borrowing increased last month after the state was forced to pay higher interest on its debt.
Public sector net borrowing, excluding state-owned banks, rose to £6.9bn in June, up £2bn from a year earlier.
The government's debt costs jumped by more than a third in June from a year earlier after rising inflation pushed up interest on index-linked bonds.
For the financial year to date, borrowing is up £1.9bn to £22.8bn, the Office for National Statistics said.
The Office for Budget Responsibility (OBR) - the fiscal watchdog - has forecast that borrowing will be £58.3bn during the current financial year. In the financial year to March 2017, borrowing was £46.2bn according to the latest estimates from the ONS.
Analysis: Jonty Bloom, BBC business correspondent
With the Chancellor, Philip Hammond, under pressure to relax austerity and pay public sector workers more, from within the Cabinet let alone the Opposition, the Treasury has seized on these figures as evidence that it does not have the money to do that. It insists national debt is still too high and the country continues to be vulnerable to future economic shocks.
But that doesn't mean there is no room for manoeuvre, although public sector borrowing is expected to rise this year overall, it is still not likely to be by as much as forecast. The OBR thinks the government will have to borrow £58bn this year and many think that is a rather pessimistic estimate.
So come the Autumn Budget and despite his protests at the moment that money is short, the chancellor may find that he does have some room to be generous.
Scott Bowman, UK economist at Capital Economics, said: "June's figures suggest that the public sector finances have started to deteriorate a little. This could limit the scope for an easing in austerity and mean that fiscal policy will still provide a significant drag on GDP growth over the next few years."
Last week, the OBR published its first Fiscal Risks report, which identified possible dangers to the public finances.
It pointed out that the UK was "much more sensitive" to higher inflation and interest rates, given that the country's debt is higher than before the 2008 financial crisis.
The ONS data showed that total government debt, excluding public sector banks, stood at £1.75 trillion at the end of June, which is equivalent to 87.4% of gross domestic product (GDP).
John Hawksworth, chief economist at PwC, the accountancy firm, said: "Looking beyond the current financial year, we would expect the decline in the budget deficit to resume if current tax and spending plans are maintained.
"This should give the chancellor some room for manoeuvre in his Autumn Budget to ease up on austerity in priority areas like health, social care, policing and housing investment.
"But he will wish to do this in a measured way given the uncertainties around the economic environment as the Brexit process continues and the high initial level of the public debt to GDP ratio."