I assume this installs itself as a replacement launcher. In which case it'd be the first thing that come up every time the phone boots. I'm pretty sure the "login" step they show is just on initial install.
As others have mentioned, Kyte replaces the default home screen, so whether kids turn off the phone, pull the battery or the app crashes, it'll just restart. The only way to exit is for the parent to put in their password.
The home button sends an intent that this application would just act as the default receiver for (just like the Amazon Fire tablet does for its custom homescreen), though if they are still giving access to the Market, a kid could install another launcher that responds to that intent and switch out of this app.
If the permission system for apps is a whitelist of installed apps the parent has approved, then installing an app from the market wouldn't necessarily allow the newly installed app to run. The new app would just show up on the device-administration page for the parent to see and decide on.
Now, if the kid uses the market to uninstall the Kyte launcher itself... then I'm guessing the stock launcher or a pop-up asking what launcher to use would come up.