Completely agree with the text sources. I think the key question is whether LLMs are actually reasoning about geography or just memorizing coordinate patterns from training data.
Going with the hypothesis that their "geographic knowledge" simply reflects coordinate density in text (more populated areas → more mentions → better land prediction). If true, plotting all Wikipedia coordinates should correlate with the "LLM world map" here.
Here is a very basic attempt at implementation of the method idea in Python, successfully compressing and de-compressing a file without any neural weights available:
I use Fuz for interactively searching my note collection, across a couple hundred text files. It's extremely useful for rapidly finding code-snippets, meeting notes or specific project information. And fast, especially combined with a hotkey for iTerm 2 that pops up a terminal and lets you search within a few keypresses.
As a nice side-effect, I no longer worry about where I store text (e.g. with Obsidian), as I know I'll find it again if it's there. It helps using memorable keywords though.
Going with the hypothesis that their "geographic knowledge" simply reflects coordinate density in text (more populated areas → more mentions → better land prediction). If true, plotting all Wikipedia coordinates should correlate with the "LLM world map" here.
This is exactly what you see here, if plotting all GCS co-ordinates extracted from Wikipedia: https://github.com/Magnushhoie/Wikipedia_Coordinates_Visuali...
This implies that larger models aren't necessarily "smarter" at geography - they just have bigger memorized datasets.