Friends and supporters say that the son of an immigrant from Russia, who went from being the lone Jewish kid in Russel, Kan., to the halls of power in Washington, always relished the role of underdog, even as he came to epitomize the consummate insider.
"Specter is the ultimate fighter and the ultimate winner," said Joseph Smukler, a Philadelphia lawyer and philanthropist who has known and supported Specter for more than 40 years.
Smukler was referring not only to Specter's campaign battles, but to his triumph over a brain tumor and Hodgkins disease.
But the longtime senator's winning streak came to a halt on Tuesday, when U.S. Rep. Joe Sestak triumphed in the Democratic primary.
Now that his career in politics appears to have come to an end, many in the Jewish community are mourning the loss and hailing his decades of public service, during which he was often seen as a "go-to" guy for the Jewish community on Israel and a host of other issues.
"Anything I ever asked Arlen Specter to do with respect to Israel or Jews, he has always done it," said Morton Klein, the longtime head of the Zionist Organization of America, who stuck by Specter through thick and thin, even when he disagreed with the senator over his efforts to reach out to Iran and Syria.
Many expressed admiration for his years of service and his attention to other Jewish causes as well, from advocating for Soviet Jewry to securing funding for Holocaust awareness and restitution to survivors.
Examples abound of Specter defying political odds. Perhaps none illustrate his sheer grit and determination than his victory back in 1992. That's when public animosity over Specter's grilling of law professor Anita Hill during Clarence Thomas' Supreme Court confirmation hearings was so great that he looked like a sure loser against activist Lynn Yeakel.
But when the November election came, he eked out a narrow win.
Twelve years later, it appeared as if an increasing conservative Republican electorate was ready to back Pat Toomey and drive the pro-choice Specter from office. With the help of then fellow Sen. Rick Santorum and President George W. Bush, Specter prevailed by a slim margin.
He also threw his hat into the presidential ring in 1995, confident that the country would turn toward a pro-choice Republican who railed against the religious right. Part of his platform was moving the U.S. Embassy in Israel from Tel Aviv to Jerusalem. The campaign floundered quickly.
"My father came to this country in 1911 for religious freedom," Specter said in a news conference with members of the Jewish media at the time. The longtime member of Congregation Mikveh Israel in Center City, who later joined Har Zion Temple in Penn Valley to be with his grandchildren, often invoked his father's journey from tsarist Russia to the American heartland.
"I'm just not willing to stand by and see this fringe take over our party," he said, explaining why he was seeking the presidency.
At the news conference where he ended his short-lived presidential candidacy, Specter said: "I come before you this morning with head slightly bloody, but totally unbowed."
At the time, he complained to a reporter that the Jewish community did not provide as much support as he would have expected.
But throughout his career, and especially in close races, Jewish support often seemed to prove the difference between victory and defeat. That happened in 1992 and again in 2004, when Specter beat Toomey by nearly 14,000 votes in heavily Jewish Montgomery County.
That race was decided by less than 17,000 votes.
His failure to catch on with Republican primary voters in 1995 may have foreshadowed his larger troubles in 2004, and ultimately, following his deciding vote in favor of the 2009 federal stimulus package, his fleeing the GOP in the hopes of political survival.
But the biggest gambit of all for the longtime Jewish Republican -- abandoning his party of five decades to run as a Democrat to try to secure a sixth term -- has now failed. He's fallen victim to anti-incumbency fervor, the distrust of Democrats of the longtime Republican, a tough-as-nails challenger and the sentiment among some that it was time for a change.
In the larger political sphere, the discussion of his legacy could last for years to come. Should he be remembered as the lawmaker who put principle before party, pragmatism over ideology? After all, he's steered untold millions in federal dollars to his home state, and been an ardent advocate for stem-cell and cancer research.
Or will history portray Specter as a lawmaker who shifted with the political winds -- who sided with President George W. Bush when it was in his interests and then quickly, President Barack Obama when it suited his purposes?
Ultimately, the lawmaker who helped torpedo the Supreme Court nomination of conservative Robert Bork in 1987 and ensure the success of Clarence Thomas in 1991 might best be remembered as a contradiction, an enigma.
Even those who disagreed with the senator at times over the years express admiration for his service.
Among them is Betsy Sheerr, a communications specialist and political donor who supports promotes a pro-Israel foreign policy and a liberal domestic agenda.
Sheerr -- who sits on the board of the Jewish Publishing Group, which oversees the Jewish Exponent -- recalled first meeting Specter in 1992, which became known as the "Year of the Woman" in politics. That's because five women -- motivated in part by Specter's harsh treatment of Hill -- ran for the Senate. All won except Yeakel, who had challenged Specter.
Sheerr said that Specter had tried to explain his rationale for backing Thomas and grilling Hill, but Sheerr wasn't buying it. But she grew to appreciate Specter's ability to get funding for cancer research, his maintaining a pro-choice stance in the increasingly pro-life centered GOP and his commitment to Israel's security.
Over the years, she not only donated to his campaigns, but briefly switched party registrations to back him against more conservative Republican challengers. But after his party switch, which shocked the political world last spring, Sheerr said that she thought long and hard about whether to continue to support him.
In the end, she backed Sestak, citing the perception that Specter prizes holding onto power above all else.
"I'm not sure what compass guides him anymore, and that really disturbed me," she said.
But for Klein, his commitment to the senator was steadfast. Their paths also first crossed back in the 1992 Senate race when Klein, then an economist and a nascent Israel activist, raised questions about Yeakel's association with the Bryn Mawr Presbyterian, which hosted a series of events that he and others deemed anti-Israel.
Many have said that questions surrounding Yeakel's church proved just enough of a wedge issue to swing the election to Specter -- the man that many Democrats desperately wanted out of office.
Not long after, Klein became president of the ZOA and they have remained close ever since. Klein has publicly disagreed with Specter more than once, including over the senator's meetings with the Assad family in Syria and his attempts to arrange a congressional delegation to Iran.
But Klein said that Specter has done much for Israel behind the scenes, including fighting to maintain U.S. military assistance to the Jewish state.
He noted that Specter had been instrumental in raising the issue of whether the Palestinian Authority was living up to its promises under the Oslo peace agreements. The 1994 Specter-Shelby amendment mandated that the Palestine Liberation Organization amend its charter, which called for Israel's destruction, in order to receive American aid.
The Conference of Presidents of Major American Jewish Organizations, the umbrella body comprised of most national Jewish groups, has also differed with Specter over the years, most notably with the senator's 1990 decision to meet with Saddam Hussein following the Iraqi invasion of Kuwait.
Always a believer in dialogue, Specter made opening relations with North Korea part of his presidential campaign platform. More recently, he met with Venezuelan President Hugo Chavez, who is a bitter critic of American policy.
Regarding Specter's 2002 plan to visit Iran, Malcolm Hoenlein executive vice chairman of the conference, said that the group "felt it sent the wrong message."
But Hoenlein, who grew up in Philadelphia and has known Specter for almost 40 years, described him as one of the most reliably pro-Israel lawmakers.
Smukler recalled that in the 1980s, Specter traveled to the Soviet Union to press for authorities to allow the opening of an Israeli embassy in Kiev.
The Philadelphia Jewish establishment's attachment to Specter was on full display in recent weeks with dozens of names attached to campaign ads in the Jewish Exponent.
Klein said that as the head of a national Jewish organization, he usually refrains from endorsing a candidate. But when asked specifically by Specter to lend his name to an ad put together by Gary Erlbaum -- one that painted Sestak as anti-Israel -- Klein said that he couldn't refuse, though his group's name wasn't used.
Marc Felgoise, a pro-Israel activist from Fort Washington, organized his first Specter fundraiser back in 2004. He's sat with Specter in his "hideaway" -- the smaller office in the Capitol Building used during late nights and adorned with pictures of Specter with world leaders.
But Felgoise decided against signing the ad because he thought that it was too negative, and so decided to put together his own advertisement that focused on Specter's attributes.
"Arlen Specter has always made the time for the Jewish community when it has shown up in Washington," said Felgoise, on the eve of the primary. "Win or lose, the Jewish community owes the respect and appreciation due to Arlen Specter."