Every two years I've looked at Python again and been told that the horrible mess I saw last time is obsolete, there's a good tool now that fixes all the problems. Every time it's just been an even bigger mess with one more tool on top.
Pretty much, but Poetry is slowly emerging as the winner, I think. There is no good solution, but if you're starting a new project, Poetry is probably your best bet.
Its behaviour isn't reproducible unless you've frozen your dependencies, and once you've frozen them it becomes very difficult to upgrade any of them. And if your project is itself a library then freezing is basically not viable, so every developer is testing with a slightly different set of dependency versions.
This combines poorly with the fact that venvs are stateful and easy to get mixed up. If you accidentally run the pip install for project A in the terminal window for project B, then you have quite possibly permanently fucked up that venv - there's no reliable way to roll back to the previous state - and if you regenerate it from scratch then, per the previous point, you'll probably get different versions of your transitive dependencies. If your tests were previously passing and now fail - or, worse, were previously failing and now pass - then you're gonna have to abandon whatever you were previously doing for an unknown length of time and deal with this.
Beyond that, those two commands (already twice as many as I'd like) don't tell you anything about how to run unit tests or package up your application for distribution, so you've got a bunch more commands to memorise, which tend to vary between one project and another because how to do those things also changes every two years. And since venv and pip were only included in quite recent versions of Python, if you've got some projects that are using older versions too then you have to memorise a couple more sets of commands for that stuff as well. The whole thing's such a mess that it almost makes docker seem like a sensible idea.
In order to reproducibly package a project with venv, you need to manage venv, requirements.in, requirements.lock, setup.py, pip-tools, MANIFEST.in, and maybe some other things that I'm forgetting. It's just unmanageable.
Yes, Poetry is best-in-class package manager. I liked the idea of Pipenv, but in practice it just didn't work well. FYI, Poetry is just a fancy tool for managing your virtualenvs
How long will that last for though? If I invest time to learn it and set it up and apply it to all my projects is it going to all be wasted because someone will have a new idea 6 months from now because of clear and obvious problems with poetry?
There's only so many iterations you can have through the cycle before "no this is the last one!" isn't plausible any more.
For packaging a Python project, that tool is Poetry. Pipenv, virtualenv, setup.py, venv, setup.cfg, are all outmoded.