Japanese media reported parts of as many as nine bodies were found in the apartment of Takahiro Shiraishi in Zama City, Kanagawa Prefecture, southwest of Tokyo.
Mr Shiraishi told police he had killed the nine and hid their bodies, broadcaster NHK reported, citing the Metropolitan Police Department.
A police spokesman could not immediately confirm these reports.
Investigators found the bodies while searching for a 23-year-old woman who had disappeared, the spokesman said.
Police originally found two heads inside a cool box at the entrance to the apartment before making the grisly discovery of the other body parts, according to Jiji Press.
Jiji Press said Shiraishi had told police he chopped up the bodies in a bathroom and a saw was found in his room.
Police inspect the apartment where dismembered bodies were found in coolers in Zama.
Ban on foreign house buyers by early 2018 - but Aussie buyers exempt
She wants to introduce the legislation before Christmas and pass the law early next year, before the TPP is ratified. This means non-residents or non-citizens can not purchase existing residential dwellings.
She had been missing since September 21 and her older brother reported her disappearance to police, according to the Asahi Shimbun.
Police started to investigate on October 24 when another man reported to the Takao Police Station that his younger sister had gone missing. "I'm looking for someone to die with me".
Police also found that security cameras installed at JR Hachioji Station and Odakyu Line's Sobudaimae Station in Zama had captured her walking with a man who was apparently Shiraishi on October 23.
Police believe the body parts were placed inside the coolers between August 22 and October 30.
According to a man in his 40s, who lives in one of the units, Shiraishi moved into his apartment about two months ago.
But he said he had not heard any unusual noise from the apartment.
Police in Japan have arrested a 27-year-old man after he admitted to killing an unidentified person and dismembering the body.