Chelsea coach Jose Mourinho handed Juan Mata a rare start at third-tier Swindon Town and he created the opening goal in the 29th minute with a shot that was parried by Wes Foderingham for Fernando Torres to tap home.
Ramires doubled the visitors' lead six minutes later, poking the ball over the advancing Foderingham from Torres's astute pass to complete the scoring in a 2-0 victory.
Ramires had only come on as a substitute for the injured Marco van Ginkel in the 10th minute but he had to cede his place at half-time, with John Terry coming on.
"We all know the cups are important and to win away against teams in the lower divisions is difficult," Torres told Sky Sports.
"For the players who have not played in the last few matches, it was good to play. Everyone is happy. It is one step forward."
Fresh from their 4-1 drubbing of derby rivals Manchester United on Sunday, City enjoyed a similarly straightforward evening with a 5-0 win at home to Wigan Athletic, the side who beat them in last season's FA Cup final.
With Micah Richards back in the team after injury, City took a 33rd-minute lead when Edin Dzeko finished from Fernandinho's through ball.
Montenegro striker Stevan Jovetic claimed his first goal for the club on the hour before Yaya Toure curled home another fine free-kick -- his third of the season -- with 14 minutes to play.
Jovetic added a fourth, with Jesus Navas also getting on the score-sheet to record his first goal in his new side's colours.
Jermain Defoe scored twice as Tottenham comfortably won 4-0 at Premier League rivals Aston Villa, with close-season signings Paulinho and Nacer Chadli also hitting the target.
"We have great strength in depth, we have great team spirit and the players are all buzzing for opportunities like this," said Spurs coach Andre Villas-Boas.
In their first game since the dismissal of Di Canio as manager on Sunday, Sunderland beat third-tier Peterborough 2-0 at the Stadium of Light to record only their second win of the season.
Two Di Canio signings settled the game, Emanuele Giaccherini breaking the deadlock in the 33rd minute from a Lee Cattermole cross before Valentin Roberge headed in a second mid-way through the second period.
Fulham came from behind to win 2-1 at home to Everton, with strikers Dimitar Berbatov and Darren Bent scoring second-half goals after Steven Naismith had put the visitors ahead in the 12th minute.
It was Roberto Martinez's first loss as Everton manager.
West Ham United, meanwhile, needed an 88th-minute goal from Ricardo Vaz Te to win 3-2 against Cardiff City, who had hit back to level the scores after conceding two goals in the first eight minutes at Upton Park.
Gary Hooper was one of the stars of the night, scoring his first two Norwich City goals on his full debut in a perilous 3-2 win at second-tier Watford.
A close-season signing from Celtic, the 25-year-old netted a 95th-minute equaliser to force extra time at Vicarage Road before claiming the winner in the 115th minute.
There were also wins for Southampton, Hull City, Burnley and Leicester City.
Luis Suarez is expected to make his return from a 10-game ban for biting Chelsea's Branislav Ivanovic on Wednesday when Liverpool face arch rivals Manchester United in the stand-out tie of the round.
Arsenal visit Premier League rivals West Bromwich Albion while holders Swansea City begin the defence of their maiden trophy at second-tier Birmingham City.