Around 25 travellers had to exit the aircraft via the slides with "no significant injuries" reported by the carrier.
American Airlines flight 731 had been due to leave for Charlotte, North Carolina from Heathrow yesterday morning (June 26) when the incident caused the service to be grounded.
A Heathrow spokesperson said a "technical issue" with the aircraft had been discovered and was later resolved.
In total 277 passengers and 12 crew members were onboard at the time.
Footage of the smoke was tweeted by a passenger sitting onboard a flight from Helsinki, which was parked next to the aircraft.
Ross Hiscock told the BBC he saw "smoke coming from the tail-end of the plane" and said emergency services crews had been called to the scene.
He said: "I saw a lot of smoke come out from the tail-end of the aircraft. I wondered if it was normal, whether it was just exhaust fumes.
"Then the door opened on the right side of the aircraft and crews deployed slides from the rear and the middle of the plane.
"Next we wondered whether it was a drill, but then we saw they were real passengers. I saw at least one carrying a child.
"It all happened in a matter of seconds. Everyone seemed to be OK."
In a statement, American Airlines said: "American Airlines Flight 731 Heathrow (LHR) to Charlotte (CLT) experienced smoke in the cabin while still on the ground in LHR.
"Three slides were deployed on the aircraft and approximately 25 passengers deplaned via the slides. The rest of the passengers and crew safely deplaned via the jet bridge.
"A small number of the passengers and crew are being seen by medical personnel but no significant injuries have been reported."
