Ballymena United made it six wins in their last seven league visits to the Oval thanks to Tony Kane's late 25-yard strike into the top left-hand corner.
Alan Teggart put the visitors in front with a penalty on 17 minutes after Jimmy Callacher fouled Allan Jenkins.
Andy Waterworth hit the post with a 31st-minute penalty, but Hill scored with his spot-kick on 63 after Dwayne Nelson brought down Waterworth.
Ballymena move to within two points of seventh-placed Portadown.
Gary Thompson missed a glorious chance early in the game after Aaron Hogg fumbled the ball, but the goalkeeper recovered to tip away the midfielder's effort.
Referee Alan Black then adjudged that Callacher's challenge on Jenkins merited a penalty award and Teggart found the net to open the scoring.
The referee pointed to the spot again after Chris Rodgers fouled Martin Murray, but Waterworth hit the woodwork.
The Glens striker attempted to make amends but saw his shot deflected over the bar before half-time.
Nelson smothered another Waterworth effort after the break, but Hill netted the equaliser with his penalty.
Eddie Patterson's side dominated possession in the second half, with Jim O'Hanlon heading wide and Ross Black clearing an attempt off the line.
At the other end, Peter Duffin's goalbound effort was also cleared off the line, while Hogg saved a long-range effort from substitute Alan Davidson.
Kane's late strike sealed a dramatic late win for the Sky Blues.