Piero Mingoia scored the winner at Portsmouth as Accrington Stanley made it three victories from five games.
Rob Atkinson lashed home Luke Joyce's long ball before John O'Sullivan added a second from the edge of the area.
Marcus Bean responded with a curling shot and Ryan Taylor hit the post for Portsmouth before Craig Westcarr levelled with a lobbed shot.
While Joyce had a penalty saved after Matt Fish fouled O'Sullivan in the box, Mingoia won it with a low drive.
Portsmouth manager Andy Awford told BBC Radio Solent:
"We were left a mountain to climb after 25 minutes being two down.
"We climbed half of it at half time and the lads showed some character in the first half, but to concede a first goal like we did is again very frustrating and disappointing, so again the defensive unit has let us down today.
"The players have been prepared properly as can be and we move onto Cheltenham now."