Gary Locke could not hide his frustration as Hearts moved closer to the drop with at Rugby Park.
Dale Carrick cancelled out Danny Wilson's own goal, however a Kris Boyd double put the home side in control.
Sam Nicholson brought the visitors back into it but Michael Gardyne sealed the three points for Killie.
Locke said: "It's very frustrating, especially when you see some of the basic things we get wrong at times."
Hearts remain 20 points behind Ross County, with only nine fixtures left to play.
"It was certainly a big improvement on last week," Locke told BBC Scotland. "Last week we weren't at the races at all, but today we competed really well.
"If it hadn't been for such a poor start to the second half we should have got something out of the game.
"At half-time we praised them because I felt they deserved to get praised for their first half performance.
"In the first five minutes of the second half we switch off, and we gave Kris Boyd the type of chance he'll put away like he did.
"We showed great spirit, got back in the game, and then for me the fourth goal was a really poor goal. Jordan McGhee - he's got to stop the lad coming inside him. Dylan [McGowan] has got to get across a wee bit quicker to help him out."
Nicholson capped another fine performance with a low drive in the 69th minute, but Locke reckons the 19-year-old prospect is not protected by the officials.
"He's a fantastic talent," said Locke. "I don't know how many times he got fouled.
"The referees have got to give him a wee bit more protection because I felt some of the treatment today was quite poor.
"Everyone forgets Jordan McGhee - he's only just turned 17.
"It's a tough learning curve for us but hopefully, as I've said all along, it'll make them better players and make them have better futures."