This is interesting! Have you considered leveraging Google OR Tools[1] for route optimization? At a previous hyper-local eCommerce startup I worked for, we used it to solve similar problems. Although the setup and integration is not super easy, but the results far outweighed the effort.
I have considered it! I've opted for a more specialized optimization library that deals specifically in the Traveling Salesman Problem (https://github.com/graphhopper/jsprit). I will revisit this though, might come in handy pretty soon - thank you!
1 - https://github.com/google/or-tools