Kane opened the scoring inside the first 11 minutes before Ferjani Sassi struck later in the first half from the penalty spot to draw level in Volgograd.
That was how it stayed as England struggled to break down the resilient north Africans, before they finally found a way through the massed defensive ranks when Kane swivelled to head home in added time.
That's the World Cup.
A huge peak audience of 18.3 million were watching BBC One as England scored a last minute goal to take the lead. "We kept fighting and got the late goal and a big win", England midfielder Jordan Henderson said.
If Ireland had qualified for this World Cup and we had beaten Tunisia with a last-minute Seani Maguire goal, similar scenes would have been played in Tallaght, Tullamore to Tuam and yet England's expectations should be a little more inflated.
Woman shouting "Allahu Akbar" stabs two in French supermarket
A sense of edginess has been with France since a murderous series of killings in 2015 in the name of the Islamic State group. Also, earlier, on 15 June in Munich, Germany, a woman died after being stabbed that she was stabbed by an unknown man.
Harry Kane is setting his sights on challenging Cristiano Ronaldo and Lionel Messi for the title of best player on the planet. We have not done well in recent years and there is not much else than could go wrong.
Lose, and it's very likely you will need to win both remaining group games just to qualify. "And may I remind you that I have 1 more Golden Boot than you".
It's a great start for England, but the evening didn't pass quite as smoothly as it could have for Gareth Southgate's men.
A group of Swiss World Cup fans booked a hotel in the wrong Russian city, confusing Rostov-on-Don where Switzerland played Brazil on Sunday, with Rostov Veliky, 1281 km (795 miles) away.
Invited to do so in the aftermath of his decisive turn in Volgograd, Southgate instead indicated his own desire to keep demanding more.