The retired Wales wing, 34, played in both defeats and the 26-26 draw at Treviso that preceded Sarries' wins.
Williams has gone no further than the quarter-finals with the Ospreys and is likely to finish his career there at the end of next season.
"It's starting to slip away this season now by the looks of it," said Williams.
"But we've got to keep optimistic. We're not completely out of it yet, but it's going to be a tough ask."
Third-placed Ospreys host bottom team Treviso in mid-January before travelling to second-placed Biarritz for their last Pool Five game a week later.
Biarritz are the only team the Welsh region have beaten in the tournament this season.
Williams admits progress to the knock-out stages is "not looking likely".
However, he believes the Ospreys' performance in their latest set-back was a big improvement on the one they produced in the 31-26 defeat at Wembley.
And Williams is confident they can improve as they face a Pro12 festive derby double against at the Scarlets on Boxing Day and against Cardiff Blues at Liberty Stadium on New Year's Day.
"The last two weeks we haven't played our best rugby. Maybe the pressure got to us a little bit, I don't know," Williams added.
"Hopefully we can up the performances. We're going to have to because the Scarlets are playing particularly well at the moment and the Blues are no pushovers either.
"So that's going to be a tough Christmas and hopefully we can pick up the performances and have a couple of victories."