I'd suggest a logarithmic playback. With the right choice of base you could probably strike the right balance between matching the playback speed without having to wait a linear fraction of twenty minutes when there's a pause in the flow. (With a bit of animation work you might even be able to express how much time is going by.)