Garrett Wilhelm told police he was using FaceTime on the iPhone 6 when he crashed into the back of a family car.
Five-year-old Moriah Modisette died from the injuries she sustained in the 2014 Christmas Eve accident in Texas.
Her family held Apple responsible for not enabling "lockout" technology on the phone.
The car was so badly damaged that Moriah and her father James had to be extracted from it, both in a critical condition.
Moriah later died of her injuries in a local children's hospital.
Police found Mr Wilhelm's intact phone still running the FaceTime application after the accident.
The Modisette family alleged Apple was responsible for the accident as it had considered utilising technology to detect motion on its phones and disable certain functions when driving.
Apple had patented this technology but it was not included on the iPhone 6.
The lawsuit states Apple's iPhone 6 was "defective" and shouldn't have been shipped without the lock-out feature.
It said Apple should pay damages for its "wrongful failure to install and implement the safer, alternative design for which it sought a patent in December 2008".
Apple was granted the patent in April 2014 by the US patent office.
No "duty of care"
In May, a court had dismissed the case, leading to the appeal.
The appeals court agreed with the earlier decision, concluding Apple "did not owe the Modisettes a duty of care".
It said it was not up to the tech giant to take responsibility for actions of individuals using its applications.
The ruling determined the Modisettes "cannot establish that Apple's design of the iPhone constituted a proximate cause of the injuries they suffered".