I've not used Olympus lenses but have been using Pentax prime lenses on my Canon bodies for years. Works great and there's some realy good old glass out there.
Infact, using old manual lenses on digital bodies is so popular now that the old lenses (still haven't got everything I want) are now going for up to 20 times the price I was finding them for over a decade ago(was collecting for my film body at the time). Oh well
According to the chart
at this page(half way down) the Olympus OM lenses have a 46mm back focus which means you can achieve infinity focus with an EOS body(44mm) with an adaptor.
I say go for it.