A late Ander Monro penalty for Canada denied Japan their first victory of the Rugby World Cup.
Canada took the lead in the seventh minute when DTH Van der Merwe powered over before Japan levelled thanks to Shota Horie's try.
Japan took control with a penalty before Kosuke Endo touched down.
Phil Mackenzie went over for Canada but two penalties increased Japan's lead which they held until Monro scored a try before slotting over.
Japan coach John Kirwan:
"We probably had the game - I thought we should have won it. I don't know whether to be happy or sad. The boys played their hearts out."
Canada captain Pat Riordan:
"Great play by the boys, great fight at the end to come back. We got out of it with a couple of points."
Canada: Pritchard, Evans, van der Merwe, Smith, P. Mackenzie, Monro, Fairhurst, Buydens, Riordan, Marshall, Sinclair, Cudmore, Kleeberger, Carpenter, O'Toole.
Replacements: Trainor for Pritchard (21), White for Fairhurst (71), Hamilton for Riordan (60), Franklin for Marshall (67), Hotson for Cudmore (74), Kyne for O'Toole (64). Not used: Hirayama.
Japan: Webb, Endo, Tupuailei, Nicholas, Onozawa, Arlidge, Tanaka, Hirashima, Horie, Fujita, Thompson, T. Kitagawa, Vatuvei, Leitch, Kikutani.
Replacements: Williams for Webb (73), Robins for Tupuailei (13), Hiwasa for Tanaka (64), Hatakeyama for Fujita (25), Taufa for T. Kitagawa (64), Ono for Vatuvei (64). Not used: Aoki.