Mark Clattenburg will take charge of Southampton's Premier League trip to West Ham on Saturday, the first time he will referee a Saints game since they made an official complaint about him.
During a 29 December defeat by Everton, the 38-year-old told Saints' Adam Lallana: "You are very different now, since you've played for England - you never used to be like this."
Southampton said it would be difficult for Clattenburg, who was cleared of any wrongdoing by referees' governing body PGMOL, to officiate in future matches.
They believe he insulted captain Lallana after the player challenged his decision not to award a penalty during their 2-1 loss at Goodison Park.
Clattenburg, who has been on the Fifa list of elite referees since 2006, has experience of taking charge of matches featuring teams who have complained about him.
Following a heated Premier League game against Manchester United in October 2012, Chelsea alleged Clattenburg had racially abused their midfielder Jon Obi Mikel.
The Football Association subsequently fully exonerated the Durham official.
Regarding Southampton's complaint, an FA statement last month read: "The FA can confirm it received a complaint from Southampton concerning referee Mark Clattenburg and his alleged comments made to their player Adam Lallana.
"The FA has given its full consideration to this matter and it is our view that the comments, as disclosed in Southampton's letter of complaint, do not constitute misconduct under the FA's rules.
"Consequently, The FA will not be taking any further action in relation to this matter and now considers this case to be closed."