West Ham, struggling in the top-flight under new boss David Moyes and now without a win in four games, were given a lifeline when West Brom’s Semi Ajayi was sent off in the 72nd minute after his push earned a second yellow card.
Sebastien Haller should have equalised with a headed chance and Manuel Lanzini then dragged his shot wide, but Albion held on.
West Ham were jeered off by their furious fans, while Croatian Bilic, who also played for the east Londoners, celebrated a sweet success with his Albion players.
“We put a strong team out. I was well aware it was a home tie so I wanted to try to make it work, but on the day we did not play well enough,” Moyes said.
“We gave up opportunities in the early part of the game and they got their goal. Too many poor passes and not enough good play.”
At St Mary’s, Christian Eriksen was left out amid reports in Italy suggesting the Tottenham midfielder has agreed a move to Inter Milan, while Gedson Fernandes made his full debut for Jose Mourinho’s men.
Tottenham’s last visit to Southampton was a painful one, with Harry Kane suffering a torn hamstring in a Premier League defeat on New Year’s Day, a clash that saw Mourinho embroiled in a touchline row with a member of the Saints’ coaching staff.
Mourinho’s side struck in the 58th minute on their return to the south-coast.
South Korea forward Son Heung-min fired home from Erik Lamela’s pass for his 12th goal of the season.
But Southampton hit back to equalise through Sofiane Boufal’s powerful strike from Danny Ings’ cross in the 87th minute.
“They are a good team. They put some pressure on us in the last minutes and scored. But we are alive,” Mourinho said.
“We don’t need one more match but we have it so we are going to fix it. It’s better to draw than to be out.”
Newcastle had to settle for an embarrassing 0-0 draw with third tier Oxford at St James’ Park as their wretched FA Cup form haunted them again.
Steve Bruce’s side are looking to make the fifth round for the first time since 2006, but they needed a replay to beat minnows Rochdale in the previous round and they will have to go the distance again after this limp display.
Burnley and Norwich made a combined total of 12 changes at Turf Moor, with the Canaries running out 2-1 winners.
Grant Hanley’s 53rd minute goal and a 57th minute strike from Josip Drmic put Norwich in control before Erik Pieters reduced the deficit in the 74th minute.
Cardiff fans were warned not to sing “racist or homophobic songs” after two incidents were reported during their 1-1 draw at fellow Championship side Reading.
Sheffield United won 2-0 at second tier Millwall through Muhamed Besic’s 62nd minute goal and Oliver Norwood’s 84th minute strike.
Kelechi Iheanacho’s early goal sealed a 1-0 win for Leicester at Championship side Brentford.
Brendan Rodgers’ team are chasing a trip to Wembley on two fronts and the League Cup semi-finalists took the next step in the FA Cup.
Leicester, who have never won the FA Cup, are hoping to reach the final for the first time since 1969.
Despite Jamie Vardy’s absence with a glute injury, it took Leicester just four minutes to go ahead when Nigerian striker Iheanacho met James Justin’s cross with a clinical close-range finish.
Chelsea travel to second tier Hull in Saturday’s late game.