IAAF president Lord Coe accused Nestle of hypocrisy after it ended its sponsorship of the world governing body's Kids' Athletics programme.
The Swiss food and drink company said it feared the doping scandal engulfing the IAAF could damage its reputation.
But Coe pointed to Nestle's backing of the Tour de France during a time when cycling had notable doping problems.
"We were left confused and angry. I didn't understand the rationale behind it," the Briton told BBC Sport.
"Clearly it wasn't a decision made about reputation because, since 2001, they've been the global partner to the Tour de France and renewed at a moment when cycling was in its worst position around doping.
"We felt this was hypocritical."
He added that the IAAF would not accept Nestle's decision.
"Our legal teams have clearly taken a view that we need to be tough about this," he said. "And we will be.
"We will pursue this strongly and vigorously because we see no sensible rationale and we don't believe this is simply about reputational damage."
Nestle's sponsorship of the Tour de France began in 2001, principally in the guise of its Vittel water brand.
In 2013, the company extended its deal with cycling's most prestigious road race until 2018.
|More on the IAAF scandal|
|'IAAF scandal worse than Fifa's'|
|Adidas to end IAAF deal early|
|Watch: Trust may not return soon - Coe|
'We need to be tough about this'
Nestle had sponsored the Kids' Athletics programme since 2012 but on Wednesday ended the agreement - which was in its final year - with immediate effect.
The Kids Athletics programme, which began in 2005, is designed to increase participation in athletics in schools and to educate children in a balanced and healthy lifestyle.
Nestle is the second notable backer - after Adidas - to end its relationship with athletics' world governing body.
The sportswear giant's withdrawal is understood to be as a result of the doping scandal, which came to light as a result of an independent commission set up by the World Anti-Doping Agency (Wada).
The Wada report detailed claims of "state sponsored doping" within Russia and stated that "corruption was embedded" within the IAAF under former president Lamine Diack.
Coe said the IAAF's marketing partners and global partners had "in large part" said they wanted to help it "reshape" the sport.
"Nestle were not a global partner," he added. "They were there to fund a specific part of our programme and that was limited to children.
"This is not a partner or a logo that will appear in a World Championship stadium.
"They came to the table because they claimed they had a profound interest in the lives of young people. I guess that's what we're left to question at this moment."
'The only people who will suffer are the kids'
In a statement on Wednesday, the IAAF emphasised the number of children that would be affected by Nestle's decision.
"In 2016, IAAF Kids' Athletics plans to reach a further 15 countries, training 360 lecturers, instructing 8,640 physical education teachers, with three million children participating by the end of the activation," it said.
Coe added: "We will continue to fund as well as we possibly can those grassroots programmes but this is going to leave a massive hole in the programme and the only people who will suffer here are the kids."
The former Olympic champion was elected president in August 2015 and is seeking to restore the battered image of athletics following a number of harmful doping cases and allegations.