Wikipedia states that the average size of an owned residence in Japan is 121.7 m^2. When you think "average house/ building" you have in mind a two-storey building, which is approx 6.6 m.
This site gives us a volume of 859.1 m^3. Density of concrete is 2400 kg per cubic meter. So 2,061,840 kg = 2,061,840,000 g.
Using the formula Q = m*c*ΔΤ
Using 3 ends of 80%, 85% and 90% hollowness for the mass.
According to Assaltwaffle, concrete's specific heat capacity is 880 J/kg°C.
Change in temperature is from room temperature to concrete's melting point:
Concrete's melting point = 1450 °C (google source)
Room temperature = 20 °C (average of 15 °C and 25 °C)
High end: 80% hollowness: 2,061,840,000 x 0.2 x 880 x (1450 - 20) = 5.189e14 J = 124 kilotons. Large Town level
Mid end: 85% hollowness: 2,061,840,000 x 0.15 x 880 x (1450 - 20) = 3.892e14 J = 93 kilotons. Town level+
Low end: 90% hollowness: 2,061,840,000 x 0.1 x 880 x (1450 - 20) = 2.595e14 J = 62 kilotons. Town level+