Hi Graham this can be a difficult thing to appraise from afar.
Thermal efficiency and energy (cooling) conservation is everything in a cooler box.
I see you have what looks like screws sticking out into the air...these would bleed cooling efficiency somewhat. Also orientating the outside HSF so the fins are vertical may help a little. You may be able to fit a bigger fan to the hot side as well?
I found that with my camera in cooler box it took around 39 minutes to reach ~20 to 25 degree C drop (down to -5C) with a 48W peltier. However this blew out to around just over an hour when I placed the box onto the focuser. I had previously been insulating the box nose piece but placing it all on the the DT really killed system efficiency.
I found if I wrapped as much insulation I could fit around the focuser DT/nose piece helped...but the cooling efficiency was effected and takes around just over an hr to reach the above and at best now cools to ~-1~-2C.
The Camera body is a real issue as you mentioned, but in general I did find the EXIF followed the box temp...with a lag of course...but not as you mentioned. You stated your box dropped 5C in 40 mins? This sounds as if you have efficiency/peltier issues...aka not enough cooling power, efficiency losses etc.
Have a look at my Picasa DSLR cooler box and the
DSLR yahoo mod site by Gary Honis. (although you may have seen this site).
Hope this helps a little.
Brendan