Vardy scored just before the hour mark to make it 3-2, Aaron Ramsey rallied Arsenal with a goal seven minutes at time to make it 3-3. Olivier Giroud and Theo Walcott also came on to replace Rob Holding and Danny Welbeck.
"I opened the door for him at some stage as it's important for him to play but I don't want him to go", Wenger said at the post-match news conference.
"Not only as a player, he is a fantastic player, but as well in the group - he has a big weight in the squad and it was a relief for me that he made a decision to stay".
Xhaka sent the ball in and it fell to Giroud, who powered in a header off the underside of the bar, sparking wild celebrations at the Emirates.
An admiring Wenger said Giroud was "in his prime" and "at the top of his game".
Football is used to watching successful centre-back, centre-midfield or centre-forward pairings - but Arsenal's manager-defender duo of Arsene Wenger and Per Mertesacker is something else.
"We were punished on every single mistake by a Leicester team that was very efficient going forward".
Joe Hart: 'Not many options for permanent move'
You can either moan about it and you can make smart comments and try to work out why it was done, or you can just get on with it. But that wasn't meant to be and West Ham have been absolutely fantastic towards me. "I don't think about it in a negative way".
"I'm always looking for the positives, we came to the Emirates and the players executed the game plan. He is a collective player, an intelligent player as well".
"Even people will say we conceded three goals, yes, but we scored four as well and the game was of quality". Shinji Okazaki was the initial beneficiary with a fifth-minute equaliser.
After Wilfred Ndidi had survived a penalty shout for handball, Arsenal levelled in first-half stoppage time, Danny Welbeck tapping in from Kolasinac's square pass after Lacazette's shot was blocked.
"I'm happy the goals we conceded we can work on together".
Shakespeare said: "He hasn't given me one problem where I've had to pull him to the side and say, 'Oi, pull your finger out'".
"We were quite good a year ago on corners".
"At the moment, sitting here, I've got a dejected dressing room".
Former Manchester City striker Kelechi Iheanacho is likely to start for the Foxes after recovering from a toe injury. We all know how good he is but a change of scenery would have helped.