Key said on TV ONE's Close Up tonight getting back to surplus is the first step to growing the economy.
The Government has targeted a wafer-thin surplus by 2014-15 in today's Budget announcement.
"If you keep borrowing money and spend more than you earn, you eventually not only have to pay that back, but you have to pay it back with interest," Key said on Close Up.
Key said the Government had a whole host of other plans for stimulating growth.
"Whether it's reform of the public sector, extend our mining and gas exploration, more farming, more money in other areas ... I don't think we should be despondent about New Zealand."
Labour leader David Shearer questioned the validity of the targets, telling Parliament: "You get sick of hearing these projections that never actually work out."
He claimed the National-led Government had overseen the "worst growth in 50 years". Green Party co-leader Russel Norman claimed the economy was becoming "more imbalanced".
But Key said New Zealand is doing well compared to other OECD countries.
"Our growth rate, according to most of the economic commentators, is likely to be better over the next three years than the euro zone, the United States, Canada and Japan. The only country that's equal with us is Australia."