Local resident Minako Akiyama told AP she first heard a deafening noise of something being torn apart.
"There was kaboom! I ran upstairs, then I saw the house just over there on fire, with a tail of the plane sticking out of it," she said.
Five people on the plane, and two of them died.
Another woman was killed in one of the houses, and two others were injured. It has been revealed the 36-year-old pilot, who was killed, was employed at SIP Aviation, a company that trains people to fly small aircraft. In the morning, the Tokyo Fire Department tweeted it was fighting the flames, and attempting a search and rescue in "intense heat."
The plane was going to Izu Oshima Island, about 100 kilometers south of Tokyo in the Pacific Ocean, according to local broadcaster NHK.
Police are currently looking into the cause of the crash.