Yes, it will make all the difference. Light being reflected off this will affect the primary and especially the secondary.
The LB's look good in all white (on the outside) but the inside should always be as much flat black as possible.
With too much light light bouncing around the tube, you will have a loss of contrast.