"I really think we risk, great risk, at losing everything we gained, and that is the opinion of every military leader I've talked to," McCain said on the "Today" show Thursday.
McCain, the ranking member of the Senate Armed Services Committee, has been highly critical of President Obama's handling of the troop withdrawal from Iraq.
The Arizona senator said that while he supports the pullout from Iraq, he believes a residual force should be kept in place in order to assist in counterintelligence and security measures.
"The president made a promise, a campaign promise that he would bring all the troops out. It's very obvious — I know for a fact because I was involved — that there was very little real discussion with the Iraqis about a residual force being left behind … so we risk losing everything that we gained," McCain said.
Obama marked the end of the nine-year war in Iraq with a visit to Fort Bragg, N.C., on Wednesday.
Obama, who vowed to end the Iraq war as a presidential candidate, noted its “heavy costs,” including more than 30,000 wounded troops and the nearly 4,500 who made what he called “the ultimate sacrifice.”
All troops will be leaving Iraq by Dec. 31, but the U.S. has vowed to continue to assist the country.
McCain told the "Today" show that Iran is filling a void being left by the U.S. departure from Iraq. He also raised concerns about the impact of the withdrawal on successes in Afghanistan.
"I travel extensively in that part of the world. Every leader I meet with believes that the United States is withdrawing, that our influence is clearly on the wane, and I think that that has an effect on the Taliban in Afghanistan," he added.