I've always taken that as the characters not knowing which language is being spoken. If it's been established I expect to see [SPEAKING MANDARIN] or some such.
They don't subtitle it because the audience isn't expected to know what's being said.
But when it's subtitled in the movie itself (which is then covered by the TV's closed-captioning system), the audience is clearly expected to get to know what's being said.
