Prime Minister Fredrik Reinfeldt was seeking to become the first center-right leader to win reelection after serving a full term in a Scandinavian welfare nation dominated for decades by the left-wing Social Democrats.

But the Islam-bashing Sweden Democrats held the balance of power after winning 5.7 percent of the votes for 20 seats in the 349-seat legislature, according to results. Final official results are expected later this week.

Reinfeldt’s four-party coalition won 172 seats, three short of a majority, while the left-wing opposition got 157 seats.

His coalition has been boosted by popular tax cuts and healthy public finances that stand out in debt-ridden Europe.

The 45-year-old prime minister said his government would stay in office and seek support from the small opposition Green Party, to avoid having to rely on the Sweden Democrats.

“I have been clear on how we will handle this uncertain situation: We will not cooperate, or become dependent on, the Sweden Democrats,’’ Reinfeldt said.

Maria Wetterstrand, a Green Party leader who campaigned with the Social Democrats and the former communist Left Party, at first rejected the idea, saying she couldn’t envision supporting a government “that doesn’t have a climate policy.’’

The result suggested a hung Parliament, because both blocs have ruled out governing with the Sweden Democrats, who want sharp cuts in immigration and have called Islam Sweden’s biggest foreign threat since World War II.

If Reinfeldt fails to solve the impasse he will be left with a fragile minority government that could be forced to resign if it fails to push crucial legislation through Parliament.

Jimmie Akesson, Sweden Democrats leader, said his party had “written political history’’ in the election.

“Party colleagues, we’re in Parliament!’’ he told jubilant supporters in Stockholm.

Large waves of immigration from the Balkans, Iraq, and Iran have changed the demography of the once-homogenous Scandinavian country, and 1 in 7 residents are now foreign-born. The Sweden Democrats say immigration has become an economic burden that drains the welfare system.

But prevote surveys suggested Swedish voters were more concerned about unemployment (at 8.5 percent in July), the economy, and the environment than they were about immigration.

Siamak Shoukri, a 52-year-old electrical engineer who moved to Sweden from Iran, said he believes the financial crisis has helped foment hostility against immigrants.

“When there is a crisis, unemployment, mass unemployment . . . they believe that foreigners have caused it,’’ said Shoukri, who voted for the Left Party.

The Electoral Authority said 82 percent of 7.1 million eligible voters turned out for the election.

Reinfeldt’s coalition ousted the Social Democrats in 2006 and kept its promises to lower taxes and trim welfare benefits.