Can anyone go in westminster abbey, london just to pray?

Yes you can, and you don't have to pay to enter. It's the same with St Paul's cathedral. I attended Evensong there once, a very moving experience.