If you live in Paris, you would probably be fine, but outside Paris you may have hard time finding people speaking english (that is, in the street, shops, etc). Sadly, foreign languages are not proper priorities in our education.
I lived in Rennes, and frequently visited Paris. Your post summarizes my experiences exactly. What's more--I learned French in university, but my accent and the native accent were mutually unintelligible for quite a long time.
