The Blue Jackets then held on against a Maple Leafs team that didn't look particularly interested much of the night, despite some late-game scoring chances, including a glorious opportunity for Colby Armstrong on a 2-on-1 with Gustavsson pulled.
Kristian Huselius scored the other Columbus goals in front of 19,148 fans at Toronto's Air Canada Centre. Antoine Vermette added two assists for Columbus.
``It's always tough to lose, especially tight games like this,'' Gustavsson said. ``But we can't feel sorry for ourselves. We have to keep going and find a way to win the next game.''
The win was the second in a row for the Blue Jackets, who improved to 19-15-3 on the season.
Columbus goaltender Steve Mason stopped 20 of 22 shots for the victory, while Gustavsson made 29 saves.
``To come into Toronto and finally win one in regulation with all my friends and family here, it's pretty special,'' Nash said.
Columbus got off to a quick start in a game highlighted by defensive gaffes and giveaways by Maple Leafs blue-liners. Nash, from nearby Brampton, Ont., looked particularly dangerous early.
However, it was Kulemin who opened the scoring with his 13th of the season that came on a 2-on-1 with Clarke MacArthur. Toronto Marlies call-up Darryl Boyce chipped the puck off the boards to Kulemin who passed off to MacArthur. The winger held the puck as long as he could before sliding it across the crease to Kulemin, who shovelled it home.
Huselius answered less than 90 seconds later with his seventh of the season on a clean breakaway from centre ice. After Toronto's Phil Kessel gave the puck away in the Columbus zone, a streaking Huselius took a pass, skated in on Gustavsson and beat him through his pads.
Bozak's fifth of the season on the power play restored Toronto's one-goal lead with 18 seconds to play in the period. With Columbus' Rostislav Klesla in the box for boarding, Kessel picked up the puck behind Blue Jackets' goalie Steve Mason and tapped it to Bozak at the side of the net, who caught Mason looking the wrong way and chipped it in.
The tide began to shift in Columbus' favour in the second period. Without dangerous forward Mikhail Grabovski, whose his girlfriend went into labour Thursday, the Maple Leafs were unable to mount much of an attack and were hemmed into their own end by the Blue Jackets much of the period. Columbus out-shot Toronto 11-4 in the frame.
Huselius' second goal tied the game 2-2 6:47 into the period. It was a gift from Toronto Francois Beauchemin, who had trouble with the puck behind his net.
The puck took a funny bounce off the boards on a dump around and deflected off the Beauchemin's skate right onto Huselius' stick to the right of Gustavsson. The shifty winger made a quick forehand-backhand move and beat the Toronto netminder.
``We've got to work for those bounces, and we've got to get more sustained pressure in their end,'' MacArthur said.
Columbus was 1-for-2 on the power play, while Toronto went 1-for-1 with the man advantage.
The Maple Leafs play next on New Year's Day at Ottawa after Columbus hosts the Senators New Year's Eve.
Notes: Brett Lebda leads all current Toronto players in career assists (8) and points (11) versus Columbus. He was a healthy scratch...the Leafs are now 2-6-0 on Thursdays this season...with two assists Monday against Minnesota, Rick Nash became Columbus' career leader in that category...