Frank Warren believes Amir Khan will have his work cut out to beat American Lamont Peterson in a rematch.
Khan is lodging a complaint after Peterson snatched his WBA and IBF light-welterweight belts with a split-decision win in Washington DC.
But Warren said: "He was making extremely hard work of what a lot of people felt would be quite comfortable.
"I think the rematch will be a tougher fight for him because Peterson is holding all the cards now."
Warren, Khan's former promoter, also has reservations about the Bolton boxer moving up a weight to take on WBC welterweight champion Floyd Mayweather.
He told BBC Radio 5 live: "All this hysteria about he's going to be the best pound-for-pound fighter on the planet is a bit premature and I really don't think he should be going up a weight to even think about Mayweather unless he was looking for a big payday.
"His defence is not as good as it should be. He went to America to work on that and to me it's not something that seems to be successful as far as he's concerned."
Khan was unhappy with referee Joseph Cooper twice deducting him a point for pushing, and also with the judges' scoring of the fight in Peterson's home town.
Immediately after Saturday's fight Khan said he wanted a rematch, but not in Washington DC.
Warren added: "I think more importantly if there is a rematch, what I was concerned with was how easy Peterson, who is not a big, big puncher, was finding it to catch Khan, especially with the right hand.
"It seems to me in his last few fights, against (Marcos) Maidana and Peterson, he's making real tough work of fights people thought he would win much easier."