I am genuinely interested in finding a good strategy to deal with memory allocation errors
Try allocating less memory and switch to a slower but less memory intensive approach. Try freeing up some memory from other places where it isn't needed right now and deal with the performance hit of reallocating that memory later.
Of course both of these approaches require a fair amount of architectural changes, but neither is unreasonable.
No; the reasonable answer is to terminate and either log the failure or restart the process if it is critical.
Use process isolation to handle recovery from OOM situations if automated recovery is required.
A process that has run out of memory is likely in either of two situations; the process has an unfixed memory leak, or it is working with an input that is too large for the memory resources of the system it is running on, in which case it is likely thrashing. In both situations, the best way out is to terminate; in the former, regular restarts can still keep the system as a whole functional, in the latter, hanging on will just make sure the system keeps swapping.
In situations where it is not to restart, it's better not to have dynamic memory allocation at all. But fault tolerance is generally a better strategy, see e.g. Erlang; systems should be designed so that processes can be restarted.
The architecture cost of this is _huge_; it's only worth doing on reliability-critical systems, where you might consider banning dynamic memory allocation altogether.
I think it's perfectly reasonable for quick tech demo projects to never check anything. fopen() is probably worth a little bit of checking, but not malloc().
Try allocating less memory and switch to a slower but less memory intensive approach. Try freeing up some memory from other places where it isn't needed right now and deal with the performance hit of reallocating that memory later.
Of course both of these approaches require a fair amount of architectural changes, but neither is unreasonable.