A fumble by goalkeeper Craig Morris cost Llanelli a place in the second qualifying round of the Europa League.
Morris dropped the ball under pressure and Aleksi Paananen capitalised to drill the ball home from 18 yards.
The KuPS goal came 10 minutes after Llanelli had taken the lead in the second leg when Luke Bowen converted a penalty he won himself.
Having lost the first leg 2-1, Bowen's goal put Llanelli ahead on away goals, but the Finns hit back to progress.
Both sides had chances to open the scoring in the first half of an even contest. Home striker Craig Williams saw a header well saved, with Lee Surman's header saved on the line from the resulting corner.
At the other end, left-back Lloyd Grist cleared off the line from KuPS striker Ilja Venalainen.