On Tuesday, the Seimas voted 72 to five with one abstention to remove Zemaitaitis from office.
At least 71 "yes" votes were needed to pass the motion tabled by MPs from the ruling coalition parties, including Zemaitis' former fellow Order and Justice members.
According to the 44 initiators of the motion, Zemaitaitis could not continue as vice-speaker after losing support from the parliamentary group that had nominated him to the position.
Zemaitaitis refused to step down, saying he had first to obtain approval from his party's council.
The Lithuanian Farmers and Greens Union, the biggest party in the parliament, ended its coalition with the the Order and Justice party in late October.
In September, all of the Order and Justice MPs, except for Zemaitaitis, set up a new political group, named "For Lithuania's Welfare", in the Seimas. Most of the new group's members have also left the party.
"For Lithuania's Welfare" has signed a cooperation agreement with the other three ruling bloc parties.
Last month, the ruling bloc failed in its attempt to oust Viktoras Pranckietis, who left the Farmers and Greens Union earlier this year, as the speaker of the Seimas.