Thanks for sharing the scripts. The instruction is intuitive -- well organized and contains a very detailed description of the virtual disk part, very neat comments. As a long-time user of OS X and homebrew, I must say I become very interested in your solution -- running PostgreSQL natively on OS X, or macOS.
However, it makes me so confused that if it is your intention to run something heavy in a consumer-level Operating System. As far as I know, macOS has a newly-added but slowly-performed file system, named APFS. It might be good enough for single disk request but not good for the concurrent requests (like 4; ; ) -- I am not questioning/blaming about the old port design of either xhyve, OS X, or even hardware architecture SCSI, but that's where we can start now.
Anyway, there is a newly-adopted storage technology named NVME emulation which might help a lot in your case -- not only it handles with lots of disk requests at the same time (65535 queues and 65535 commands per queue), but also the architecture makes it possible to program the controller code in software, especially in user world. I don't know all details of NVMe but it works well in my experience -- both of consumer level and enterprise level -- it brings smooth disk latency in the virtual machine even under a very high bandwidth (50Gib transfer in nearly half a minute, can you believe that?). Do you know if it is possible to use NVMe emulation for faster disk access in xhyve and accelerate the PostgreSQL execution natively?
>However, it makes me so confused that if it is your intention to run something heavy in a consumer-level Operating System
I assume this is to build a development environment. Especially if you're working on a laptop, on a client-server app that needs to talk to a Postgress (or other) database, running the database in a VM so it goes where you go is a pretty common setup.
It is true and that's probably the most unclear part of the original blog. I think the author has given some kind of advice -- use a vnet based on host-bridge -- not sure why he just commented out from the script. In my understanding, if I am working on a laptop, I can just set SNAT rules which allow traffic from a private network to go out to the internet. Given that the number of connection won't be large (no more than 400 connections, or 500, a usual number of open file for one process), it is acceptable that the new connections can wait until a period of time.
However, it makes me so confused that if it is your intention to run something heavy in a consumer-level Operating System. As far as I know, macOS has a newly-added but slowly-performed file system, named APFS. It might be good enough for single disk request but not good for the concurrent requests (like 4; ; ) -- I am not questioning/blaming about the old port design of either xhyve, OS X, or even hardware architecture SCSI, but that's where we can start now.
Anyway, there is a newly-adopted storage technology named NVME emulation which might help a lot in your case -- not only it handles with lots of disk requests at the same time (65535 queues and 65535 commands per queue), but also the architecture makes it possible to program the controller code in software, especially in user world. I don't know all details of NVMe but it works well in my experience -- both of consumer level and enterprise level -- it brings smooth disk latency in the virtual machine even under a very high bandwidth (50Gib transfer in nearly half a minute, can you believe that?). Do you know if it is possible to use NVMe emulation for faster disk access in xhyve and accelerate the PostgreSQL execution natively?
- SCSI: https://en.wikipedia.org/wiki/SCSI - NVME: https://en.wikipedia.org/wiki/NVM_Express