The actor was a huge supporter of Obama in the run-up to him winning the US presidency election in November 2008 but says he's been disappointed in how he has run the country.
He told America's ELLE magazine: "I've talked to a lot of people who worked for Obama at the grassroots level. One of them said to me, 'Never again. I will never be fooled again by a politician.' You know, a one-term president with some balls who actually got stuff done would have been, in the long run of the country, much better."
However, Matt says he does have some sympathy with Obama because he has a lot to deal with, including ongoing negotiations over the country's debt ceiling.
He said. "I don't know what you do in the face of that kind of intransigence. So, my heart does go out to the president. He is dealing with a lot."
This is not the first time Matt has criticised Obama and the decisions he has made over the past three years.
He recently said: "I really think he misinterpreted his mandate. A friend of mine said to me the other day, I thought it was a great line, 'I no longer hope for audacity.' He's doubled down on a lot of things, going back to education ... the idea that we're testing kids and we're tying teachers' salaries to how kids are performing on tests, that kind of mechanized thinking has nothing to do with higher order. We're training them, not teaching them."