But does it make sense to even talk about inside and outside then when talking about the universe?

It makes about as much as uncountable infinite numbers. I can say everything in the universe is within a countably infinite distance of me. Thus, anything an uncountable infinite distance from me is outside the universe.

Not a very tangible idea, but not useless of purpose of reasoning about the affairs of things.

It's a bit of an abuse of notation. Talking about the inside definitely makes sense. (Talking about the outside is a bit like talking about the empty set.)