According to the Wikipedia article manchineel actually hasn't killed anyone in modern literature, despite having a description of the taste of it's fruit. My guess is the tree doesn't directly kill as much as massively inflame flesh so it's still "survivable"