Gates said he will continue as chairman of the software giant and will also assume the role of "chief software architect."
In addition to his CEO duties, Ballmer will also become a member of Microsoft's board of directors on Jan. 27.
The executive shuffle comes as Microsoft is facing stepped-up pressure on the antitrust front -- namely, reports this week that the U.S. Department of Justice will demand a breakup of the software giant into separate companies as part of an antitrust settlement.
Ballmer responded to those reports Thursday, saying a company split would be "the single greatest disservice that anybody could do."
"It would be absolutely reckless and irresponsible for anyone to try and break up this company -- there is no precedent," he said. "It would be reckless beyond belief."
Gates: Still a full-timer
Gates, meanwhile, said his "commitment to working full-time" for the company remains "100 percent" but that it was time for the management structure to evolve.
Calling the move "a very exciting evolution for me," Gates cited his 25 years as CEO of the company he co-founded in January 1975 as a reason for him to take "the next step."
"I've certainly enjoyed being CEO," he said. "However, what I really enjoy the most, and what I'm getting back to, is working with the product group."
Gates said his new role will involve mostly strategizing. He said he plans to increase his use of Microsoft's (Nasdaq: MSFT) development tools to get a feel for how they could be made easier to use.
He even joked that he may "threaten" to get back into writing code for the company.
Discussing move for years
According to Gates, the decision to step down as CEO was something he had been discussing with Ballmer and the Microsoft board for the past couple of years.
The first step in that direction came in July 1998, when Ballmer replaced Gates as president of the company. At the time, Gates said he wanted to spend more time focusing on developing new ideas and strategies and allow Ballmer to focus on the day-to-day running of the company.
When asked if it was difficult to give up control of the company he has overseen for so many years, Gates did acknowledge a certain sense of nostalgia but added that he was ready for the change.
"It's a nice milestone to look back and say, 'Hey, over last 25 years we really got something done,' " Gates said. "I'm proud of that."
During their joint news conference, Ballmer and Gates alluded to a new "vision" for the company -- similar to the change that occurred five years ago when Microsoft shifted its focus to the Internet.
Company's new 'vision'
Gates said consumers can continue to expect "something new and neat" for Windows every year for the "foreseeable future," and that the company will continue to be focused on software.
He believes, however, that the nature of software is changing, becoming more of an Internet service than a packaged product, he said.
Gates said he thinks the next 10 years will "the decade of software."
Ballmer and Gates promised to present in April a roadmap of Microsoft's plans for the next three years.
