I don't think the most tangible benefit of MVC is reuse, but rather code organization. In MVC, if you're looking for something, you know where to find it. That saves a lot of time. It's easy to write an application, but harder to maintain it.

Also, imagine an application with a Web view displaying a data set returned from a controller. Now the client/user wants an OS-native version of the app. That's simple, just build a native (OS-specific) window view and call the same controller for the data set (model). Or maybe an API is needed for external consumption. In that case a web service "view" can be built to return the same data set from the same controller. We're just swapping views here--it's plug-and-play.

It's a clear separation of responsibilities. I believe that used to be called "modular" programming. ;)

It seems like you've run into limitations in the frameworks you've used, not in the MVC pattern itself. In ErlyWeb (which I created, so I'm biased -- but not too biased), components are implmeneted by a controller module and a view module. Any component can include any other component. The controller decides which subcomponents to include, and the view decides where to put them after they've been rendered. This design makes it natural to separate a page into different controllers + views that can be reused in other pages. It also maintains the MVC philosophy that the controller should decide what should be rendered, and the view should decide how to render it.

MVC is an elegant design pattern, but I think an MVC framework must provide a simple way for components to include other components. Otherwise, I can see why you would find it frustrating that your main controller has to do all the work.

Many people accept the MVC dogma as the word of god. In truth, it is a design trade-off just like anything else. You should really look at SeaSide or UncommonWeb. In SeaSide the view and controller are combined into a component in the way you describe as intuitive. In Rails best practice is to create partials (partial views) that can be re-used across views, put code into the model instead of the controller, and use before_filter to trigger re-usable functions.

Rails has filters that can run before, after, or around some or all of your controller actions. This can be done on a per controller basis, or it can be application wide. In this case you'd probably define a "load_widgets" before_filter in your application controller, then set your view to display whichever widgets it had data for.

There are other pieces of this puzzle besides before_filter.

In Rails, you don't necessarily have to have a monolithic view. On more complex sites, you use fragments, called partials. Sidebars can be easily created by having a "shared" view directory full of partials. Further, there is a content_for helper in Rails views. Views render late; you can set an instance variable in the subviews that will affect what gets rendered in the layout. Trying to keep track of that can get hairy, so content_for abstracts that for you. Combined with the partials and the before_filter in the controllers, you can get a view to produce a number of things without having to resort to pulling DB info inside the views -- that reminds me too much of the bad old days of PHP.

In the logged-in users example, in what is normally in the sidebar, you use a render :partial => 'shared/users' if @users; then add a before_filter in the site-wide application layout to pull in the data if someone is logged in. A more complicated example is to render a "sidebar" partial, which renders more partials depending on some array of sidebars you want to display ... then use a before_filter to figure out what you want to populate the sidebars with. Any controller or actions can override or add to that sidebar array.

The main thing that sucks about this kind of organization is having the code scattered in a number of different files. This is where using an editor that knows the relationship between the files is important -- you can hit some key combination and jump between the different fragments without having to hunt them down in the file browser.

I think having that degree of modularity is a great thing (granted you have a good text editor). It makes it much easier to understand what's happening in the code and much easier to maintain it (and get someone new up to speed).

I was helping out on this one site built in PHP a while back where the code wasn't organized this way. It took me about a week to figure out where things were, which was a huge productivity drag for the other guys.

I just found this out five minutes ago -- You can get a plugin called "embedded actions" for Rails (2.0, and probably 1.2.x). The README is here: http://dev.notso.net/svn/rails/plugins/embedded_actions/trun...

A snippet from the README:

Unlike partials, embedded actions also let you define business logic to be performed before the partial is included. That logic is encapsulated in the already well understood metaphor of an action inside a controller.

So a simple call like

<%= embed_action :controller => "songs", :action => "top10" %>

lets you include an html fragment containing the top 10 songs into any of your pages, regardless of which controller, action or view wants to do the including.

----

In other words, you call "embed_action" from within a view which will call the controller-action and embed whatever result is there. You still get the organizational benefit of MVC without the hassle of rolling your own partials.

As an additional note to what I was saying about partials, Rails 2.0 also has a concept of "partial layouts", allowing you to switch out the surrounding content of a partial.

Similarly, Django has context processors for the same purpose.

"For example there could be a list of logged in users to be displayed in the sidebar, or any number of widgets. It seems unnatural to have to gather the data for those in every controller call."

Aren't these things conceptually separate views, each having its own controller?

However, mentioning MVC and web applications gives me an excuse to engage in nostalgia and once more mourn the passing[1] of WebObjects. It was simple in WebObjects to make any reusable part of a page into its own component. Also, MVC has a much bigger payoff when you have development tools designed to support it. WOBuilder allowed you to graphically build interface components, then just drag connections between properties in your controller object and the widget you wanted to display them in. And not just top level properties, but you could drill down through the relationships of any object (e.g. user.shoppingCart.numberOfItems, Google "key value coding" if interested). (And yes, this is just like Cocoa and Interface Builder, came out of the same technology base originally.)

I have yet to see any web development tools[2] that come anywhere close to the productivity enabled by WebObjects. And I didn't even get into the Enterprise Objects Framework...

[1] WebObjects is still very much alive inside Apple (iTunes store, Apple Store, pretty much everything Apple does on the web) but for outside developers it is quite dead. The only life support is ant + Eclipse plugins, but that defeats the whole purpose as the development tools were the whole point of WebObjects. [2] Tools as distinct from programming languages. WebObjects was Java, so nothing special there, but real old timers still miss the Objective C version.

Try Rails and see. It's pretty impressive. Django is supposed to be pretty good too if Python is more your thing.

The trick with MVC is to realize that it's not some immutable magic spell, it gives results by enforcing a separation of concerns: data-modeling (and its manipulation and constraints) in the model, control-flow (and selecting and ordering the data to be presented) in the controller, and rendering it a certain way in the view.

If you, as you suggest, have the view (essentially a glorified HTML template) request the data it needs (e.g. from multiple sources), then your data-selection and visual-presentation become tightly coupled. (The point of a view is that you can have multiple views of the same data). If the original view is handling things like data-selection and complex control flow, then when you want to create an alternate presentation of the same data, you'll end up copy-and-pasting code (c.f. "Don't Repeat Yourself").

So the best solution is to get a really good feel for separation of concerns, code-non-duplication, etc., and then organize your code accordingly. When using MVC, use every bit of it to keep your code simple and decoupled; if you just slap code into units called "models", "views", and "controllers" wherever you like, then you will receive NO BENEFIT (or negative benefit!) from using MVC. But, used properly, it can be a powerful tool.

To answer your question in the title...MVC is a good solution. The way frameworks implement it also varies, but basically the idea is to setup a way to separate things into a few layers. I have found MVC-based systems easiest to implement and modify because of this separation.

Caveat: My MVC experiences have been mostly with Java (Struts) and Python (Django) Then to get into your actual points:

1) The view CAN do the work of determining what will be shown. To quote the Django site: "In our interpretation of MVC, the "view" describes the data that gets presented to the user. It's not necessarily how the data looks, but which data is presented."

2) Anything that is a common widget should be abstracted and included on the page in some sort of "include". Either have some sort of base page that does the work, or on the individual pages do some method of "include". This really doesn't relate to MVC, other than your point about adding the data to your context/session, which should again be done generically and non per-page.

3) The general idea for any MVC system is to have the flow go through one controller. This controller may delegate to sub-controllers, but the idea is everything goes to one spot.

4) What is the best solution? There are many good solutions for writing a web app.

"The view CAN do the work of determining what will be shown"

1) In Django the controller is called the view and the view is called the template, same principle but unless you want to get confused you should think of it as MTV == MVC (if you want to equate the two, Django seems to be perfectly happy making up its own interpretation of MVC for no apparent reason). Really the controller (in the MVC sense) is doing the work here.

Sorry I think its just the wording that is confusing here.

The controller is doing the work to determine what will be shown, yes. I meant once the controller determines which view should be used, the view then can manage the specific data being shown.

Django does separate the view into a 'view' and a 'template'. One does your work, and the other does the displaying.

The Best Solution for what? Client side mashups? Greenfield development? Mixed-platform interop? Speed and threading? Flexibility with changing requirements?

Not trying to be a jerk, it's just a complicated question. I guess you mean greenfield web applications? Even then I'd have to ask things like how heavy you were client-side, what languages you knew -- most importantly, where's all the data? If you're delivering data basically from the server to the client and back, does most of the processing take place on the client or the server? Some MVC paradigms are crazy to push into Javascript. Some are crazy to use for simple projects.

Lately for Web 2.0 Applications, I'm using a framework that has a lightweight MVC (very lightweight!) in Javascript and pulls information using JSON from various sources. But I wrote it myself, so I'm biased.

MVC is a paradigm that can be applied a lot of ways, in a lot of situations. Most frameworks emphasize the controller, when the model is really what you're after. Your controller classes can "dumb down" in many scenarios to be so lightweight as to not require a separate class. Hard to do that with a pre-canned framework. I guess I would be extremely careful in confusing the framework with the architectural conceit it uses -- two different concepts.

The biggest benefit of MVC, in my view, is the amount of reuse it encourages. When a view has no knowledge of what it has to do, you can reuse it an infinite amount of times. A table view is a table view, it shouldn't need to know what its displaying. A button shouldn't need to know anything about your app, because its just a button. This is what makes Cocoa/IB so powerful. In some instances, you don't even need to write a controller, you can just use a stock controller, like Cocoa's array controller.

I would also say that this is one of the biggest drawbacks of current client side web frameworks. Reusing views is much more difficult than it should be, especially if you want to do anything remotely custom or complex, since subclassing is essentially not possible and delegates are very infrequently used.

I tried using Struts for a project a few years back and after I realised how much easier it was to maintain the codebase with a proper enfored separation, I never looked back.

Though I found myself, in many cases, writing more code than I would have without a framework, I found the maintenance benefit more than compensates.

Things have got more complex in recent years with more and more complex interaction logic embedded in webpages with Javascript and i'm not sure i've seen many frameworks address this.

Admittedly, there are now better frameworks out there than Struts (this was Struts 1) and I think i'll have to do a bit of exploring for my new project's website but we'll see.

I think MVC suitability is dependent on the language used. MVC works well with imperative languages but isn't necessarily desirable if developing for example in a functional language.

Java also has a good component oriented framework: Wicket. I think MVC it's a nice and simple solution, it does the job. However, IMHO component oriented frameworks are conceptually better and reuse is much easier.