Former House Speaker Newt Gingrich said Sunday he has some advice for President-elect Donald Trump's early days in office: "Swing for the fences."

"I've done this twice in my career," Gingrich, one of Trump's closest advisers and potential Cabinet member, told CBS News' "Face the Nation" host John Dickerson.  "I did it with [President Ronald] Reagan in '81. I did it as speaker in '95. This is a city that if you don't shove it as hard as you can when you have momentum, it will surround you. The swamp doesn't want to be drained, and the swamp will just suck you in if you let it. He needs to have a very, very aggressive first year."

Trump won a decisive Electoral College victory, said Gingrich, but even more, Republicans won control of Congress, and the nation's states will have the most Republican governors and state legislators in history, marking a shift toward Republican government on a national scale.

Gingrich said Trump "has to be the salesman" moving forward, and take on agenda items that he can work on in a bipartisan manner such as on infrastructure, "something Trump knows an immense amount about."

"I suspect on tax reform there's a chance to put together a tax reform package which also, as speaker Paul Ryan pointed out today, also means you can handle a lot of the terrific differentials inside the tax code in a way that could be very advantageous to American business," said Gingrich.

Gingrich said he has seen some changes with Trump since the election, as his meetings with President Barack Obama, House Speaker Paul Ryan and Senate Majority Leader Mitch McConnell were also encouraging, as is the fact he's integrating Vice President-elect Mike Pence "so thoroughly, which also happened with [Dick] Cheney."

Trump has also said he'll be open to incorporating some aspects of Obamacare, said Gingrich, and when it comes to immigration, there will be "substantial deportations," even though Ryan said on Sunday there won't be a "deportation force."

"Two million people would be a lot of people to deport," said Gingrich, pointing to a Trump interview airing Sunday night in which he promises "substantial deportations" of criminals. 

"And if it's the same thing, we'll regain control of the border, and if you pass a guest worker program, you would be a long way to dealing with the rest of the folks who are here without legal permission, and I think that by that stage we would accommodate it in some way," said Gingrich.

Obamacare will likely be repealed, Gingrich continued, "but there are certain aspects of it that are widely supported, and you don't want to capriciously take away the right of somebody to be on their parents' insurance until 26. You want to protect the right to have insurance coverage without any kind of precondition."

Trump's election has met with many protests, but Gingrich said it's important to remember that other presidents, such as George W. Bush and Obama were attacked from "day one."

"Interestingly, while we got involved in a fight with Clinton, no one ever attacked the legitimacy of the Clinton presidency or Bush one, or Reagan or Carter for that matter," said Gingrich.

Gingrich also called some of the fears being written about a Trump presidency "garbage."

"The 'Washington Post' had this column that pointed out that we're on the anniversary of Kristallnacht, the night when Nazis attacked Jewish businesses," said Gingrich. "This is crazy. Donald Trump is a mainstream conservative who wants to profoundly take on the left. The left is infuriated that anybody would challenge the legitimacy of their moral superiority. And so the left is hysterical."