Not as many as later civilizations but there are buildings that likely pre-date the Sumerian civilization like the desert kites. And in Syria and Turkey there are megaliths and ruins which are older than Sumer which builders the Sumerians might have know of from oral history.