If it were me, I'd be cleaning it. However, that's just me
For peace of mind, you could always let an expert clean your mirror. I know Bintel offers such a service.
If it doesn't affect the views and there's nothing corrosive about the 'deposit' perhaps it's not worth worrying about.
You might need to do some research on what those fumes may or may not be, and how corrosive they are.