Madison Manor, once used as housing by Drew University, was purchased by investors in March for $3.35 million, and was vacant and undergoing renovations when the fire broke out late Nov. 5.
The Madison Fire Department called for assistance from 10 other departments over the course of the night, and firefighting efforts continued into the next morning.
The owners of the property have said they plan to rebuild.
Much of the three-story building was torn down by Friday morning. By Monday morning, a pile of rubble remained.
Carol Werner of MRY Associates, which owns the project with Normandy Real Estate Partners, said the building needed to be torn down.
"We will rebuild and hope to have the project complete by first quarter 2015 if not sooner," she said Friday.
The property is adjacent to the former Green Village Road School, which Madison officials are working to sell to a redeveloper.
A website for the property at 39 Green Village Road said the 21 units were to be available in the first quarter of 2014. The new one-, two- and three-bedroom units were scheduled to start being shown to prospective tenants the same month the fire occurred.
"On behalf of our partners and everyone involved in the restoration of Madison Manor, we are devastated by the fire that struck the building," the owners said after the blaze. "We are thankful for the Madison Fire Department, police and public safety departments and all of the other fire departments and emergency personnel who responded to the site in the early morning hours of November 6th."
The property website says "Madison Manor combines the best of modern, contemporary living with the old world charm of a magnificent building from the 1920s," located two blocks from downtown and the train station.
The renovation was a venture of Mark Yeager, founder and principal of MRY Associates LLC, and real estate private equity firm Normandy Real Estate Partners, according to the Madison Manor website.
The fire burned away the roof and damaged the third floor. The second and first floors and the basement sustained water damage from the firefighting efforts. No one was injured. Madison Fire Chief Lou DeRosa has said officials did not consider the fire suspicious.