I'm a big advocate of Agile but at the same time I'm leery of joining teams that say they do Agile. Why? Because they usually mean, "We do standups and burndown charts and that's it." It's cargo-cult Agile, at best, and micromanagement by story point, at worst.
Football teams don't do standups saying "I will run X yards". Each and every player on the team
-knows precisely what the objective is
-knows precisely what his/her role is and is capable of fulfilling it
-knows what s/he needs to communicate to other teammates for them to fulfill their roles and does so promptly
-is capable of assessing what needs to be done next given the current situation on the field without having to be told and takes on that task on their own initiative
-doesn't have anyone breathing down their neck saying "A point must be scored by X:XX on the clock or else!" regardless of whether the tactical situation makes it possible
Together, such a team functions like a well-oiled machine. _That_ is what Agile is trying to teach. But more often than not, few, if any of the personnel or business conditions are met to make it possible and so it's a painful, miserable charade for all involved.
I have seen Agile work exactly once in my career; it's absolutely beautiful when it does.
If teams can't consistently implement "Agile", then maybe it's not really that great of a method. Or perhaps it's not much of a method at all, just a bunch of pseudo-philosophical nonsense, measuring things that aren't relevant and using methods whose correlation with actual results is near random.
I mean, look. We _know_ there is no single project management method that produces good results across the board. That is a fundamental result of undecidability (and related no free lunch theorems). There is no single method, there never has been, and there never will be. The teams that work well do so because they are good at what they do, they have a specific skillset for what they are working on at that time, they work well together and adapt to the circumstances, and frankly, are a bit lucky. That is a moment in time. Most, if not all, teams cannot reproduce that result consistently year after year, nevermind among different teams. It is a fleeting attribute, not a systemic byproduct of _process_, unless the field is so narrowed so far that optimization is in fact possible (which is exceedingly rare, and certainly doesn't manifest itself in general software development).
I think people look at Agile from the wrong angle.
Agile is old enough that many people don't know what pre-Agile development was like. Often teams would pretend everything is fine until months after projects should have shipped. Pretending things are great while looking for a job or jumping projects was really common.
Even cargo cult Agile tends to make it clear things are going to fail ahead of time. Which is huge.
Sure, that does not mean the project is good or profitable, because on it's own it's a long way from enough. But, nobody cares about process when things are great, it's when things are failing and you want to get horribly broken to break even where even cargo cult Agile can help.
> Agile is old enough that many people don't know what pre-Agile development was like. Often teams would pretend everything is fine until months after projects should have shipped. Pretending things are great while looking for a job or jumping projects was really common.
That's not true, Agile didn't save the software industry, it's just a different methodology with different ideas.
The "software crisis " still exists today the same as it did then, the only thing that's changed is young people who don't have the necessary perspective to understand that.
A big part of the point of "agile" is that process is something that you hack/adjust/tune. If you're trying to do agile by dogmatically following a process, expect to fail.
When you can say, "Let's tweak our process in this way for the next two sprints, and see if it works better than what we've been doing", that's agile.
But you're right, it's fragile. The reason it's fragile is because, as you go up the management tree, eventually agile runs into big-process management. And big-process management wants their reports so that "they know what is going on". Real agile looks sloppy, out of control, even though it delivers more value to the business faster than big-process development does. So real agile scares big-process management, and eventually gets killed by a layer of management that can't deal with it.
"When you can say, "Let's tweak our process in this way for the next two sprints, and see if it works better than what we've been doing", that's agile."
We did that in our company for about 3 years. Then some guy at the top decided (or maybe that's how he got misunderstood in the middle) what we are doing is not true agile, or whatever, and we are back at square one. So he basically decided to waste all the time we spent in retrospectives and all what we learned.
It's like when people say that true Christianity is about forgiveness and other values, which sounds great, until you run into some fundamentalist with the holy book.
"Agile" has become a meaningless term I feel like. It used to denote the four values of the manifesto for agile software development and devolved into an entire industry of gimmicky practices sold by mediocre engineers who can't even make the simplest of CRUD apps.
I'm still very much in favor of these four values. But in and of themselves they require a lot of thought and analysis in a recursive feedback loop to be used properly.
What we have instead are mindlessly rigid processes for the sake of having a methodology and a huge track record of epic failures in Agile projects. It has become like a drug - people chase that initial high they will never, ever get back.
Good results require a good team, so you can still easily fail even with a perfect process if your team isn't good. The fundamental tactical process we follow is to make sure every task has clearly defined entrance and exit criteria. Work doesn't start until entrance criteria are met, and it isn't done until exit criteria are met. Each task is the smallest possible feature that delivers incremental value and tasks are listed in priority order. That's the whole deal. Scrum or Kanban layer some more controls on top of that just to handle volume or big organizations. Most attempts at measurement are completely in vain.
Those are exactly the things I've been frustrated by. Attempting to break everything down into small chunks with well defined entry and exit criteria can easily be more time consuming and lead to more analysis paralysis than just giving an experienced and trusted team member a goal and letting them work toward a solution, with lots of communication as they go. I suppose YMMV applies, as usual with this sort of thing!
If teams can't consistently implement "Agile", then maybe it's not really that great of a method.
Alternative explanation: There are a lot of shitty teams out there. Agile requires a culture shift, not just adoption of a handful of practices from a book. From my limited experience, it works very, very well if, and only if the culture shift accompanies the process shift. If the process is the only thing that changes.. it's painful.
Agile process is pointless (and probably actively harmful) if people are so close minded they won't adopt the associated culture.
A method that enables a great (at some random dimension) team to reliably get great results is a great method. It does not matter that other teams can not make it work.
But then, the Agile Manifesto was all about "stop talking about this or that method being great and just do what works on your environment!!!"
That was the thing that worked everywhere, and all there is to it. Needless to say that Scrum, or any other named process does not fit it.
Yeah. I was involved in the movement before the term "Agile" existed, and I'm deeply disappointed with what the word has come to mean.
Everybody meant well, and everybody was very smart. In retrospect, though, I think the big mistake we made was thinking that if our hazy principles (as expressed in the manifesto) collided with typical American managerialist business culture, our principles would win out.
In practice, we ended up with what I think of as "mini-Waterfall with Agile labels". Release cadence is faster than with traditional Waterfall. But it is still generally a top-down, push-oriented, document-heavy system. Line workers still seem just as miserable and disempowered as before.
I personally have used Kent Beck and Mary Poppendieck's work to create some extraordinary teams and some great working environments. So I think the principles and the original insights have deep merit. But I recognize little of that in the typical "Agile" environment.
Usually, the more a team advertise itself as Agile, the less agility it actually has.
I myself have never seen Agile work well, not even close. I saw plenty of make-believe and head-in-the-sand and as mentioned in earlier comments, insane amounts of cargo culting.
These days this is the only methodology I follow and it works wonders, anything more I feel is just noise. At the end of the day nothing replaces actual engineering and understanding what you're doing.
>> I myself have never seen Agile work well, not even close.
My main gripe is most organizations I've worked in were usually in the transition between waterfall and some form of agile. The #1 issue was the people who sold the corporate execs on agile being faster and more efficient than waterfall have set unrealistic expectations as to deadlines and release dates.
Suddenly taking 12 months to build that app you were thinking of developing via waterfall, just got cut in half because everybody thought agile is so much faster, when in reality, it really isn't that much faster at all - just a different methodology, which a lot of people forget.
I have yet to work at a place that has released anything any faster than how it could be done using traditional waterfall methodology.
My last point about agile is that I've found it doesn't scale well. When I used agile in smaller teams (10-15 people, 3-5 devs) it was very effective. As you scale up (50+ people, 15+ devs) things slow down, inefficiencies are created because instead of several devs working on larger chunks, you have several dozen devs working on really small parts that get coded completely differently, then sometimes don't work together and it causes multiple iterations of design that wouldn't happen in waterfall. There is an advantage with us devs seeing the larger picture and coding to that, instead of this little component that gets coded 10 different ways.
I think all methodologies slow down when the teams get larger.
In the XP teams I've been on, everybody rotates between stories and swap pairs daily, just to avoid the individual code ownership and fragmented coding styles you mention. Works really well.
> unrealistic expectations as to deadlines and release dates.
Also, one of the core agile ideas in my world is that there are no deadlines or release dates for specific features.
Thats usually not possible. We're not doing pure science, we're doing engineering - which happens to mix science AND economics.
This means the "no deadlines" wet dream is usually a recipe for "oh shit, we ran out of money" for most projects. Also programmers are usually big babies; give them unlimited time and they'll give you unlimited bloat. Other than doing R&D it usually is impossible to have no deadlines.
Most people are not doing R&D, but when you really do your government usually has programs to finance it. That's precisely how you can afford to not have deadlines because your company is paid to advance the state of the art. And even then you usually get paid after the project is over and after you've filled a bunch of forms to clearly state the problem, your solution as well as existing work in the area and why it did not apply to your case.
Or if you're doing servicing work you get paid by shipping at predetermined milestones. Building software is a LOT more complex than just writing down the code and too many developers are clueless when it comes to politics, economics or management.
I used to think the same when I was still new to the software industry. It took me years to understand theres a whole world around programming required for programming to happen in the first place.
People can use "we have no deadlines" as an excuse to never do anything useful.
This is why one part of the Agile Manifesto that isn't talked about as much as it maybe should be is about how you need to build trust with your customer by constantly delivering value. Demanding that trust from strangers is hard sell, and it probably should be.
The big problem with deadlines is that it's impossible to predict how long something will take, so you end up in a game of lies and bluffs to create and take away buffers to account for the uncertainty, which leads to horrible relations between the dev team and it's customers. Also, it falls apart, when priorities change.
This only applies to servicing contracts, when you're the customer constant delivery is very well overrated. Even with servicing contracts shipping every week or two is a definite recipe for mediocrity. You're wasting all that upfront analysis for the sake of early delivery and then suffer from all the bad technical decisions made to build it in a rush. Expect insane amounts of overtime to reach the last milestone on time for such projects.
Deadlines are not that hard when you properly plan to reach them and give yourself a large buffer of time for debugging and QA. You just need to properly analyse the requirements, the existing solutions in your problem's domain, what is known and unknown (keeping meta-ignorance in mind - you dont know what you dont know) and then re-analysing it all every time the requirements change, even slightly.
This means you're going to spend a lot of time upfront before coding starts, and the first deliverable will be months away. Solid foundations yield by far the most agility to any project I've ever seen, and continuous delivery prevents these foundations from being built in the first place. You definitely won't see any proper abstractions at such a pace, what you're most likely to see is an endless sea of indirections.
Same for testing, I've seen many teams starting to write tests for requirements that weren't well understood yet. Then they start piling code up until the tests passes, and finally wonder why there's tons of new bugs in Jira every week. All the tests passed! The type checker passed! But now there's no time left for debugging and the project gets over time and over budget.
The smoothest servicing projects I've ever done were with customers understanding the nature of our work and not minding a long time window before the first delivery, not interfering with the work. The worst servicing projects were having two weeks sprints where the managers would bend over to every single customer demand for fear of losing the contract at the cost of burning out the team and having no quality whatsoever in the finished product.
You get trust with new customers based on your own track record of successfully shipped projects, not continuous delivery over arbitrary milestones. And then you can manage the customer, instead of being managed by them. They pay you because they can't do the work in the first place after all.
Thanks, but I understand the economics of software reasonably well.
Note that we are discussing "no deadlines or release dates for specific features", not just a magical world without consequences.
If you are looking for how this works, Poppendieck's books on Lean Software Development go into both the theory and practice. But the basic notion is that you control costs via scope control, waste reduction, minimizing WIP, and continuous improvement. One still delivers things quickly and efficiently; it's just that deadlines are not used as a control mechanism.
Yeah, I definitely agree that a popular reason for adoption was "go faster", which was poisonous to the actual attempt.
Honestly, I think you eventually can get substantial speed gains. But you mostly get them by cutting out waste. Doing that takes a fair bit of time and work. And not just work, but commitment to personal and cultural change. People don't do that under strong time pressure; instead they are inclined to revert to previous behavior.
I find if you define the way things must integrate(REST) with defined versioned interfaces. Then you can have small teams working on different services, without that much overhead.
There are way, way, way more uses of programming than simple CRUD services. Operating systems, game engines and browsers are only 3 examples where that doesn't apply, at all :)
Besides, what's to say there wont be overlap in these newly separated services that will generate work in double because teams dont talk to each other? Or maybe you'll generate two average solutions instead of a better unified one.
Even with multiple teams working on different aspects of a project, you still need yet another team to maintain the global vision of the thing. To make sure everything is consistent, to avoid work duplication, to consolidate solutions and abstractions and whatnot.
Its fair to say the larger a development team is, the harder it will be to manage. Its even more true with large teams of average developers, which really is the norm when you think of it.
> Usually, the more a team advertise itself as Agile, the less agility it actually has.
Bingo. Usually when you ask them what agile processes they are following they just mumble something about daily stand ups (which I'd argue are not agile at all).
I've seen agile work. But it's more about automating builds, having continuous (or near continuous) deployments, having database migrations to easily integrate others changes, etc.
My evaluation of how agile a company consists of "When a trivial bug is reported, how long until the fix is in production?". If the answer is greater than a day then they aren't agile.
I feel like "agile processes" is an oxymoron. Real, actual agility requires a lot of thinking; something usually prevented by having processes in the first place.
A better question to ask an "Agile" team would be: "what problems are you trying to solve?" - most of the time they haven't taken the time to properly analyze the problem in the first place, even when the project is near completion. They just run with dumb user-stories and start piling up Jira tasks with no clear vision.
At this point I don't believe Agile can work whatsoever. A success rate of one in a thousand projects, which usually happens to be a dead-simple CRUD app anyways, means absolutely nothing, other than it has failed in a spectacular way. If software development was an actual science instead of a pop culture, the scientific method would've thrown Agile out the window decades ago.
Your last two sentences almost exclusively applies to web services, which is a tiny fraction of what software engineering can do.
For me real agility is how much effort you're willing to spend upfront to properly analyze problems before attacking them, how quickly do you adapt your solution when new facts become known.
Its dead-easy to have agility at the beginning of a project, what matters is still having it 2 years down the road and more. Continuous deployments, database migrations, automated builds and whatnot do absolutely nothing to help you reason about the simplicity of your codebase - which is what yields true agility.
The last two sentences don't only apply to web based projects, they can apply to any application development from web to desktop to mobile. I think that would cover the vast majority of software developers.
> Continuous deployments, database migrations, automated builds and whatnot do absolutely nothing to help you reason about the simplicity of your codebase - which is what yields true agility.
The help in the sense that you can make and deploy changes seamlessly. For understanding code the unit tests that go along with them also help tremendously, if they are well written unit tests anyway.
Oh yeah, but you could be spending 10 times the required amount of efforts because of accidental complexity and none of these tool will point that out for you. I've seen it happen way too often to trust a team relying heavily on processes and tools anymore.
Besides, you set them up once and then forget about them, happy with your day to day use. But actually fighting complexity and making sure you're still solving the problem at hand and not bloating things up requires near constant critical thinking, something no tools or processes will ever be a replacement for.
> Continuous deployments, database migrations, automated builds and whatnot do absolutely nothing to help you reason about the simplicity of your codebase
I pretty strongly disagree with this. Simplicity in agile projects is achieve through continuous improvement of the code. All of those things make refactoring cheaper and easier, which means you can keep your code base in good shape as domain and business needs change.
Without those supporting practices, people get scared to change anything. And once they get scared, bit rot sets in.
> Simplicity in agile projects is achieve through continuous improvement of the code.
This is also called "emergent architecture" and its a big lie. You'll at best reach a fraction of the quality you'd have by thinking about said architecture upfront with twice the development time - it works for CRUD projects simply because such projects are dead-simple to begin with.
How are these practices helping us reason about the simplicity of the codebase? How are the people dependent on your code going to handle all of your frequent refactors? What does continuous deployment even mean outside of the web? How do you refactor something as big as a game engine with 10+ projects built on top of it?
More importantly, how are you going to add proper abstractions to an existing project? No refactoring tool will ever do that, it cant be automated and the more advanced a project becomes the harder they become to introduce in the codebase. I've seen plenty of "best practices" projects that were just huge hairy balls of indirections. Yet everyone on the team felt good about it because none knew any better.
You can't judge practices based on how they work for one particular area of software development. Especially when said area is one of the simplest out there.
> You can't judge practices based on how they work for one particular area of software development. Especially when said area is one of the simplest out there.
You know exactly nothing about what I've based my opinion on. Because all you've done is jump in and rant about your beliefs.
> thinking about said architecture upfront with twice the development time
Design quality is limited by the information you have, and no amount of thinking creates information. The beginning of the project is when you have the least information about everything: domain, users, user needs, business needs, technology, team. Everything.
You mostly get information by trying things in the world. And actually trying things in the world is the only real test of ideas that come from thinking.
If your methodology requires understanding everything up front, then it's basically useless in the real world. Even if you could perfectly understand your intended market and use case, that will change over time. But you can't, because your competitors are always plotting to beat you, and they will not share their plans with you.
Amazon, for example, has been in operation 20 years, and could easily be going for decades more. If in the early days they had tried to write code capable of dealing with their current needs, let alone their needs from 2050, they would have been idiots. Pure up-front design is an unworkable fantasy.
> More importantly, how are you going to add proper abstractions to an existing project? No refactoring tool will ever do that [...]
Refactoring existed before automated tools, and mostly exists beyond it. Tools help, but it's mostly a human activity. Try reading Fowler's book on this, and his subsequent writing. You can't effectively criticize something you don't even understand.
Perhaps in other aspects of how they develop. How do they do releases(Continuous?). Code review? Automated Testing? Do they test individual features as they are completed or do they test releases as big chunk at the end? Do they have big up front planning, or do they have a plan little bit - implement a little bit-> deploy a little bit -> learn loop. How much ownership does developer have over a feature? Does he ability to say "No" if it isn't ready.
I personally would look for specific practices and specific results.
For example, I think a focus on quality is necessary to enable real agility, so I'd look for unit tests, acceptance tests, continuous integration, continuous deployment, pair programming (or fast-cycle code reviews).
And for looking at results, I look for things that are hard to fake, like release cadence, amount of work in process, and unit of work size. So I'll ask, "When was your last release? And how about the release before that?" And I'll want the difference to be small. (My favorite place to work released multiple times a day.) I'll ask other questions like, "What are you working on? Who's working on that? When did that start?"
Another thing I'll measure teams by is the extent to which they're focused on problems vs features. Good teams ship things that solve problems (and make sure that they actually do). Bad teams ship features because that's what the spec says or what the boss wants.
Yeah, I think most job postings are so poorly written that there are too many false positives and negatives to use them for much filtering. The main thing I'd eliminate first off is anybody requiring Agile certifications; organizations that lean on that have basically given up on thoughtful hiring.
Good luck with your search! If you're in the Bay Area, feel free to drop me a line when you go back and I can recommend some places.
> the big mistake we made was thinking that if our hazy principles (as expressed in the manifesto) collided with typical American managerialist business culture, our principles would win out.
Agile helped. Many programmers appear to find unions distasteful, so they needed some other way to try to protect their jobs from outsourcing. Agile claims "face-to-face" is most efficient.
some other way to try to protect their jobs from outsourcing
But if the team is a "black box" with their only contact with the rest of the firm being mediated by the Product Owner, then that team could be anywhere. I always saw Agile as a sly trick to make outsourcing easier by packaging the team into a self-contained unit.
How would an Agile team be any different in that respect from any other kind of software team?
Also, that black box team is by definition not "Agile" because the engineers are not face-to-face with the users, nor communicating with the users daily during development.
Teams communicating daily with users - i.e. people who will use the software, as opposed to some manager - are some kind of unicorns; I've heard they may exist somewhere, but I've never ever seen one. Especially not the ones communicating daily.
And if your "daily" communication with "users" involves specifications and talking to managers, it doesn't matter whether you're in-house or outsourced.
I have done that repeatedly. If you're doing in-house software and have a brisk release cadence, it's not particularly hard. And it's not just an Agile thing. I was doing it for financial trading systems in the 90s, and my dad was doing it for manufacturing systems in the 70s and 80s. He'd watch people using the software, see an issue, talk with users about the issue, make a modest change, an then see if it was better.
It looks different for modern consumer products, because your software is typically a small part of someone's life. They don't want to talk to you daily. Instead you have a product manager acting as proxy for the users. But if you have good instrumentation, you're still getting user feedback daily.
Even in the consumer context, you can talk to real users frequently. At my last startup, we had user testing every Tuesday afternoon. We'd book 4-5 sample users every week, and we'd always have some question we wanted to answer. And plenty of places have small beta tester groups that provide fast feedback on new features.
That all depends on a fast release cadence and a tight relationship between product management and developers. But once you have it, it's addictive for product managers. Oursourcing basically becomes impossible, because their turnaround time for answering a question drops from seconds to hours, and the turnaround time for shipping a small change goes from hours to days. This forces batch size back up, which is not only vastly less fun, but also reduces ability to explore the product space and find high-value business wins.
Every time I've been asked about outsourcing, I've asked, "Will you send your product managers there too?" 99% of the time the answer is no, because they can see how that would impair the information flow. The trick is getting people to extend that desire for connection and fast iteration past the managerial tier.
Agile is properly a meta-methodology, a methodology for helping you build the proper methodology for your team. If you're not trying things out every so often, keeping what works, and discarding what doesn't (even if it's officially part of some named methodology!), you're not doing Agile Manifesto-Agile.
Mistaking a meta-methodology for a methodology is a major category error. It pretty much guarantees failure.
I'm at least reasonably agile. Looking back over the last five years, someone who thinks Agile is a methodology would think I'm very inconsistent and all over the map, because my methodologies keep changing. But that's because my needs, responsibilities, task types, and indeed even customer type have shifted over that time. What would be weird is if I did the exact same thing the whole time, because "Agile".
What would be weird is if I did the exact same thing the whole time, because "Agile".
It wouldn't just be weird; it would be very bad.
I like your definition of it as a meta-methodology. I think 90% of the blogs I read about Agile completely miss this -- even those, like this one, that rail against Cargo Cult Agile.
I think what people get so messed up about is the mix of discipline and performance. If your team decides that in your situation you need daily standups and burndown charts? Then you got stuff to rigorously do every day. You make your own rules, sure, but whatever they are, they're meaningless unless you have the discipline to stick with them.
Some folks naturally want to focus on the "make your own rules" part. Some folks naturally want to focus on the discipline part. But focusing on either is a terrible mistake. They're equally important. (Insert long discussion here a bout Shu Ha Ri)
I'm willing to wager that 95% of teams don't have a choice when it comes to a lot of this stuff. The burndowns and standups are mandated by management. So a lot of this "If your Agile isn't working for you, just change it!" doesn't really apply.
To some degree, yes. I've worked in tons of multi-team environments. Being able to walk into a team room and ask simple questions like "How much work do you guys think you have?" or "About that feature. Have any idea when it might be done", or "Anything I can help you with?" is a reasonable thing to ask if you're burning the kind of money businesses burn.
The counter to what you say is that none of this crap should take more than 10-15 minutes a day. I find that if I establish that upfront, SMs and managers don't end up creating Agile Death Marches. Instead they come back and ask how that can possibly work.
Because Agile is meta, it really should disappear. Aside from setting up things and tweaking them, it shouldn't be a topic of discussion. If you're talking about it too much, you're probably doing it wrong.
But you are correct. It is not like that in most places. Sadly.
>"If you're trying things out every so often, keeping what works, and discarding what doesn't (even if it's officially part of some named methodology!), you're not doing Agile Manifesto-Agile."
I feel the exact same way. And I've had the same exact experience: only one place in my decade long career has done agile well.
The Project Manager was completely on point with gathering information from stakeholders and direction from executives. The scrum master was a perfect neutral voice who really did an amazing job mediating. Our team was full of smart, capable, focused multidisciplinary engineers. It was absolutely beautiful.
I've also never ever seen it again. I've been trying to figure out what was different about that org than everywhere else. Because I've always worked with smart, social SWEs.
I think the problems arising from cargo cult agile are more fundamental than the methodology. In order for agile to work well for SWEs, the organization's infrastructure and expectations must be a fit. There are key things that I saw at the place where it worked and were missing everywhere else.
* Strong product ownership. The PM is the captain of the ship. They must interface with the execs, with the customers, with the technologists, with the customer service. They must synthesize the stories and effectively outline clear exit criteria for them. And they must accept input from engineering to prioritize tasks that must be done but do not necessarily directly provide customer value. But the buck stops with them. Engineering should not be subject to department infighting for prioritization of pet features. The direction must be clear, or you're foundering.
* A neutral voice in the planning meetings to keep things on track and ensure everyone who has a question / input is heard.
* Demo days. It's so important for the morale of everyone - on the team and off - for the exposition of the completed work for a given sprint.
* Respectful retrospection.
* Trust and verification. Micro management is toxic. The only time other than sprint planning that the PM should be involved with the daily work of engineers is when a story requirement needs clarification.
And modifying the scope of an in progress sprint should be relegated _only_ to emergencies. This cannot happen if there are _always_ emergencies, though. Which is why it's important for a PM to understand and allocate time for maintenance / infrastructure improvement.
Here are a few problems with your football team analogy:
1. The football team that has the knowledge and judgment you described are all A-players. A-players tend to not need a lot of process. One of the reasons we have project management and software methodologies and all that process is to help the team's B- and C- players perform predictably and consistently, if not close to as good as the A-players. If you actually found that mythical group of software people where each and every one of them knew the business in as much depth as your football players do, then yes, they probably wouldn't need as much structure. My experience has been these teams are extremely rare.
2. A football game is physically observable at all times by all stakeholders. Status needs no reporting. The score is known to all. The position of the ball is known to all. This is not true for software projects. Part of project management is providing visibility to stakeholders and leadership so that risk can be managed and informed strategic decisions can be made. This is why "Agile" emphasizes delivering working software as milestones. Different project management schemes use different methods to track and report status, but the end goal is the same.
3. A football game is already time boxed due to the structure of the game. There is a start time and a duration, it's separated into quarters, team activities are structured into downs, which aids the team in making tactical and strategic decisions. How would all these smart players "know precisely what to do" if they didn't know what quarter it was or how much time was left on the clock? Software does not naturally have any of this structure, so various methodologies seek to frame project with structure to aid decision making.
4. You really don't think any football team's leadership ever yells "We gotta get at least 10 more by halftime go make it happen!" You have to have intermediate milestones. Every sport I've ever played has had a coach or lead player barking the current goals at me. Guess what: That's part of the machine's oil.
Actually, in my mind you helped strengthen the football team analogy.
1. I only like to work with A players as well. Only hire A players. If you have B and C players, let them go and keep looking.
2. Yes, working software is observable to all. When you have working software, everybody knows what the score is and where the ball is. If you software sux, the ball is in the wrong end of the field.
3. Many software projects have the same time constraint. A conference it needs to be ready by, or a point in time where there is no money left.
5. Football is a zero-sum PvP game, programming is a PvE race-the-clock game. In football, you don't have to be good, you just have to be not worse than other team (+/- luck factor). In programming, you have random obstacles (requirements, management decisions) thrown at you, the effort it takes to scale those obstacles is mostly unpredictable, and you're being pushed to do it before deadline "or else".
6. Timescale. American football match takes an hour (1.5 hour for soccer, + break). That's how much time the team works together until the next deliverable. In programming, it's weeks or months. Sustaining teamwork on such a long scale is a completely different task. If you want to compare something to football, look at hackatons - and then you'll see people who worked together before often excel at getting things done.
Re: #1, that kind of depends. Are the B- and C-players at that level because they are incapable or because they've never been shown how to be A-players? If the latter, a well-bonded team can motivate and teach them how to become an A-player as well.
> knows precisely what his/her role is and is capable of fulfilling it
I also like Agile when it works, but I'm of the opinion that not following it is a symptom and not a disease. If you're fortunate enough to be on a team of people who know precisely what their role is and how to fulfill it, they'll tend toward Agile by virtue of being able to work quickly and having time to help each other. If you're not that fortunate, then it no longer becomes feasible to divide the work into semi-predictable chunks. The problem is in thinking that Agile is a suitable substitute for skill coverage in a team.
Strong agree. I have worked at several places where "Agile" meant "Create a lot of tickets in Jira and then knock them out fast" or "Create a lot of points in PivotalTracker and then knock them out fast."
I've also been viewed as a trouble maker if I email the development team and ask them to read this:
Football (and any pro sports) teams run retrospectives after every game. They watch tons of video, seeking to learn what they were doing, what challenges they were struggling with, and how to improve.
IT's just like anything in IT. "serverless", "nosql","cloud", "Agile","REST" . These become buzzwords because everybody sees what he wants to in these words. But I'd also argue that if a concept is hard to implement or every implementations are somehow broken because there is no formal definition, then the problem is the concept at first place.
Methodology doesn't replace good management. Most often I see the trappings of agile used as tool to conceal a lack of planning and means to micromanage and give the illusion of progress.
I'm fixing a project now where the PM is throwing a list of thousands of tasks at me to show me how much work is done, but haven't delivered anything and cannot commit to ever delivering anything.
I love your description. That said,that is more "well-oiled" than agile. Agile have been taken over by standard and process gurus that want the certainty of a process rather than the ambiguities of people.
Do you have personal football team experience? And is this american football or European futball?
Because most american people I know in football, know their EXACT role. Mr. Offensive lineman - I smash whoever is in front of me. If the play is to the left, I smash left. If the play is to the right, I smash right. They know nothing about defense. They know nothing about receiver routes. They don't care. They stand up and smash people.
The QUARTERBACK knows a bunch. But he also has a headset on, with 3 people feeding him tips for each play. It would be like a PM coming by every 5 minutes and telling you that you should use a Dictionary to store the current data, and then on the next problem you should maybe look into an ArrayList.
Basically.. football is a terrible analogy to software development.
This is the sort of tangent that we don't mark off topic, because going into the intricacies of football as an analogy for software development is whimsically specific (as opposed to generically ideological). Whimsical specifics are interesting!
Perhaps you got downvoted because you resorted to the "Do you have" trope to insinuate that the other person is ignorant. This is a variant of "Did you even read the article", a form of incivility we specifically ask people to avoid here: https://news.ycombinator.com/newsguidelines.html. We also ask people not to use uppercase for emphasis. If you edited those things out of your comment it would be much better.
Offensive lineman have to know a ton about the play being run, whether they should pull left or right, who he's responsible for blocking, if he should drive block the man up or chip and work to get in front of a linebacker, etc.
Receivers have to be on the same page as the qb as to what is getting done and how they'll adjust based on coverage.
Running backs usually don't have to do a lot of thinking, they're more reactive, initial responsibilities and steps set but changes quickly as the play develops.
Thats not to say anything about the above except that FB is in many ways the most "team" of team sports with vastly different body types, roles and responsibilities at play. If you can't make an analogy related to business work within a gridiron (american) football context, you probably can't make it work :)
> Because most american people I know in football, know their EXACT role.
The comment you're replying to says this specifically.
> Mr. Offensive lineman - I smash whoever is in front of me. If the play is to the left, I smash left.
This is an oversimplification and if this is all the offensive lineman does, he'll be rapidly out of a job. The linemen are constantly adjusting to deal with the defense. They'll hold to avoid a big sack. They'll leap on the ball when there's a fumble. They'll block someone else if needed. It's not smash left and done.
> It would be like a PM coming by every 5 minutes and telling you that you should use a Dictionary to store the current data, and then on the next problem you should maybe look into an ArrayList.
I think it would be a lot more like management saying "we need you to build product X" and then expecting the QB to deliver. The QB isn't just running plays by rote.
>Because most american people I know in football, know their EXACT role. Mr. Offensive lineman - I smash whoever is in front of me. If the play is to the left, I smash left. If the play is to the right, I smash right. They know nothing about defense. They know nothing about receiver routes. They don't care. They stand up and smash people.
That's a very oversimplification of american football. For a basic example even the "dumb" offensive lineman has to play differently whether it's a run or a pass. A more complicated play like a screen will require a lot of coordination with his other teammates to perform correctly. American football is actually a pretty good analogy for a the typical development team and the different roles an offense has. You have people in the trenches, people doing flashy stuff and a field general trying to tie it all together.
I think that a lot of what you are saying is only true if you don't understand American football all that well. Blocking schemes are far more complex than just "smash whoever is in front". WR's run much more complex routes than just a single route as well (option routes depending upon the coverage and coverage technique).
His analogy is limited (as all analogies are, at some level), but it works for its intended purpose, IMO.
But if it's a pass, and it's a three-step drop throwing a quick out, the tackle (offensive lineman on the outside) wants to guide the rusher to the inside in order to keep him out of the path of the ball (which will be a fairly flat throw, and therefore perhaps within reach of the defensive lineman). So in that case, yes, the offensive lineman has to know what the receiver route is, and to react accordingly.
If is about it still works pretty well. Stupid top-down deadlines are different than the effective communication the QB gets. And a dev team does need effective communication.
As another poster said here, I've seen Agile work well one time out of many organizations. I think the worst is when individual teams don't choose to use Agile and it instead comes from somewhere high up on the organization: "We are now Agile".
I've not been a part of but I've heard of horrors like SAFe. I've seen the diagrams [0] and I can't believe someone was sold on that.
I guess BigCorp needs to use something as a software delivery methodology. If they don't use Agile what else would they use?
>I've not been a part of but I've heard of horrors like SAFe. I've seen the diagrams [0] and I can't believe someone was sold on that.
the diagram is an eye candy for a mid-level executive. Upon seeing it, they start dripping saliva and become weak in the knees. The diagram has Lean, Agile and while it is missing the word Scrum it obviously is depicted there too. I've been through couple of years of the process looking extremely similar to that diagram (though word "SAFe" wasn't there) implemented across whole R&D (only super important projects got exception) of a pretty BigCo. Well, it was beyond horror. The product development was basically stalled completely. The gears were churning with lightning speed, people were constantly overworked, rushing to meet multitude of deadlines, .... all that while product just wasn't moving forward. No meaningful features could be delivered. It is hard to believe until you see it with your own eyes.
Then, one nice morning, all the mentioning of it just disappeared, like it never happened, and we're back to happily cranking out according to current release plans with weekly team meetings.
The irony is that "Agile" methods were developed in part to protect developers by poor management. Yet it's the same poor management that relies on off-the-shelf processes.
I've been in so many meeting where a manager tried to espouse the benefits of "being agile": Faster releases, tracking work more accurately, ability for constant review.
But I've never been in a meeting where somebody tries to talk about the benefits to developers: Your goals can't change day to day, your boss doesn't get to strictly tell you what to do or how to do it, and no more arbitrary deadlines!
You really have to listen to how somebody is selling a process to understand their interpretation of it.
What specifically do you think is wrong with SAFe? It's overkill for small teams, but in my experience it works well when you have multiple agile teams working together to deliver frequent releases on a large, complex product.
> Agile teams find their ways within clear, yet very general rules.
And reiterate on those rules.
We implemented Scrum at the company I work for. It could have just as easily been any other sort of development methodology---Kanban was under consideration for less structure, for example. What we were looking for was something that works for us, and that would get us out of the huge mess that we were in. Scrum addressed many of the issues (especially management) we were having, so we figured we'll give it a shot and slowly start implementing it.
We don't stick to the books. The core concepts of Scrum are implemented, but are guiding. We constantly iterate on the process itself---every sprint we review how we did, identify problems, and put something in each sprint to improve upon them. We introduce concepts gently. If we find that something doesn't work, it gets adjusted or replaced. If it works, we apply it even more stringently. If making it more stringent introduces problems, relax it.
We do nothing because we "have" to under Scrum. Everything we do is analyzed, rationalized, and adjusted. And it has been wonderful---it saved a department that was falling apart. Are there problems? Sure. But we've identified them, and they're being addressed.
Concepts like TDD and pair programming---those aren't things that can be forced. The team needs to be on board, and if it's not working, then changes need to be made to hear and address their concerns, or get them to understand why we do things the way that we do.
At a higher management level, how projects were managed couldn't be forced. It was a gradual change, and it took a couple years to completely change how the entire company approached projects.
as DevOps Borat says - "agile waterfall". Agile causes deadlines to be tighter because agile causes huge inefficencies, especially when it comes to inter-team/project/module interactions and dependencies, and pile of additional unnecessary stuff to be done that takes additional energy/resources/time to burn to deal with and which gets in the way of the stuff what is really need to be done.
A snake oil - be it agile or communism - is easy to identify by the fervency with which its peddlers preach no true Scotsman.
Agile doesn't cause more inter-team interactions and dependencies; that's an orthogonal issue. In general dependencies can be minimized by organizing around feature teams instead of component teams. But you can do the same thing with a waterfall process.
>Agile doesn't cause more inter-team interactions and dependencies;
It doesn't cause more of them, it makes them fail.
> In general dependencies can be minimized by organizing around feature teams instead of component teams.
You can't include common core upstream components into downstream branching features. Basic topology of a tree. Until of course you just have only one team doing the whole project.
Even when components with a dependency between them are in the same team the dependency still exists and agile makes it much more harder to handle. Until i guess it is one person doing both components - that by transitive closure would mean one person doing all the project himself (until the project component graph contains disconnected islands)
You're missing the point. The agile manifesto — and the various agile methodologies in common use today — don't mandate any particular practices around reuse of common core upstream components.
If you want to divide the teams up by component rather than by feature you can do that, but it's usually a bad idea. Most projects can achieve higher velocity and quality by having teams take on vertical slices of functionality through the full software stack; this minimizes misunderstandings and delays when handing off work between teams. But either way it's essential to have stable APIs and clear design contracts before building any dependencies on top of common core upstream components. Again this is all orthogonal to agile versus waterfall.
Also, a real agile development cycle started from OO design, you need to design the code well enough that each object is relatively isolate from each other before we can talk about Agile.
nowadays, every organization talk about agile development cycle, but few business ppl really spend time to understand what it is, and how much it is involved.
like one of the poster here said, "Agile waterfall"
Well that is one approach and it sometimes works. But usually it fails or least forces excessive rework because much of what you thought you knew at the beginning of the project later turns out to be incorrect. Another more agile approach is to avoid big up front design. Treat design as an emergent property and keep the internal OO design clean through constant refactoring.
>But either way it's essential to have stable APIs and clear design contracts before building any dependencies on top of common core upstream components. Again this is all orthogonal to agile versus waterfall.
in the ideal world you'd have that stable API, contracts, and delivered components satisfying them from day 0. In the real world you have back and forth iterations which are just fine under waterfall and which are heavily impeded to the point of impossibility by the sprint cycling in agile. On practice in waterfall the upstream and downstream components are developed almost in parallel, and they have active feedback cycle between them. On practice in agile you can't really start consuming a component until it reaches the stage of deep maturity (and reaching such stage under waterfall is significantly faster due to almost immediate feedback cycle from downstream components as mentioned above) and that makes agile process into super-slow component dependency serialized process, a "component-wise waterfall".
There is nothing in agile which prevents upstream / downstream teams from working in parallel and iterating on components. Nor does agile require that components reach the stage of deep maturity before they are consumed; in reality it's completely the opposite. I can't fathom where these misunderstandings are coming from.
Again, if your project is organized in component teams then you're likely to have those problems regardless of methodology. The optimal solution in most cases is to reorganize around feature teams, plus have a small cross-team group of technical experts for each major component who can provide stewardship and review major changes.
>There is nothing in agile which prevents upstream / downstream teams from working in parallel and iterating on components.
a task (for example acting on a feedback, fixing a bug, etc.) can't be taken in mid-sprint because it would affect the sprint's deliveries and thus a replanning mid-sprint would be needed which thus makes all this ritualistic sprint belly-dancing pointless. So, the downstream has to wait until next sprint for any actions by the upstream wrt. the feedback while that downstream is still responsible for meeting its current sprint deliveries which it has hard time to satisfy given the unsatisfactory delivery state from the upstream. After being burnt this way several times the downstream team finally learns from experience and refuses (or fights pretty heavy against management pressure) to take on in a given sprint any tasks which depend on components which haven't reached deep maturity by the sprint's start or any maturity at all and hell no to any promises that that upstream is coming down mid that sprint (any deliveries from upstream, even if scheduled for mid-sprint actually almost always come at the end of the sprint). And if it is the first version or a major change you can't even seriously plan on working with it next sprint - see feedback sprint cycle above - the best you can hope is the end of the sprint next after the next... That makes even idea of iterative parallel process pretty much impossible, a bad joke slow as toothache.
>Again, if your project is organized in component teams then you're likely to have those problems regardless of methodology.
Agile worsens these issues couple orders of magnitude compare to traditional relaxed [i.e. we aren't coding F-35] waterfall.
No that's not the correct way to do agile project management. As I have repeatedly pointed out, you can prevent most upstream / downstream iteration delays by organizing the project around feature teams, not component teams. But if you persist in using component teams for some reason then obviously both the producer and consumer teams must build in sufficient time into their story point estimates for iterating on the interface. As for defects that come up mid-sprint, either every team should keep a percentage of their capacity in reserve, or on larger projects you should dedicate a full Kanban team to maintenance.
Sure there are some managers out there who implement agile incorrectly by failing to follow best practices. But all of the problems you described are self-inflicted and easily avoided on agile projects with competent leadership.
Agile to me has always been the way for managers to get an almost daily status of what developers are working on. All the metrics, tools, charts and meetings have been for the benefit of management not devs.
The more jobs I have the more I think collective code ownership is the problem. I've never worked at a place where that hasn't resulted in a sprawling mess. No methodology will save it.
I know this is going against the grain here but, have you ever worked somewhere that followed the Waterfall methodology?
For all its problems, the one thing I've found it works well for is cutting down on technical debt. Developers get requirements up front, and documentation can be planned out alongside development. You do need skilled business analysts/project managers for it to work well, but it definitely cuts down on the sprawling mess I've seen in Agile. I'm sure Agile can work well, but from my limited experience too much emphasis is placed on moving fast, resulting in considerable technical debt (not a problem if a codebase is small enough that a team can entertain the idea of a rewrite, but it's disastrous for large codebases).
> I know this is going against the grain here but, have you ever worked somewhere that followed the Waterfall methodology?
I haven't. I feel like the simple waterfall model is too rigid, but one of the modified versions would work well.
I am trying to design more of my code upfront these days, and spend less time coding directionlessly. Trying to become more prolific. Waterfall seems to encourage that.
That's actually a good thing. One thing I really hate about, say, Scrum, is that they renamed everything. That way, decades of experience with project management was lost to young people, who fail to see the connection. I think it was entirely intentional, though, because how else could you sell snake-oil?
It's not a rename. It's completely different. Similar responsibilities are split between the Product owner(Planning what features to include, interacting with users and some roadmapping), scrum master(Normally ex developer, coaching developers on how to improve the process, TDD, Code review) and the team itself(Estimates, final say about much goes in a sprint).
I think most people prefer kanban or scrum over traditional PM style management.
However they have failed deliver on all promises.
When people complain about agile, they aren't harken back to traditional pms
And the reason for this reorganization of responsibility is what, exactly? See, that's the problem. If you rename it, people won't compare it. So it gets easier to sell, because if people compare it, they might get a clue and find some flaw in it.
I am not against some shuffling of responsibility. But ideally, you should know why you do it and best if it's backed up by some studies. You can't do a study if there is no way of comparison.
It's not just people though, it's also stories, sprints, iterations, epics,... you name it. Also, everything that came before was retroactively (and derogatorily) renamed "waterfall", despite the fact that many of the advantages of agile were actively discussed since 1960s.
And, yes, I believe well managed project can be better than agile, at least in some cases. We have Agile in our organization, ostensibly, but in effect it's lot more paperwork than it ever was under so-called waterfall. At least that is my experience, of course some other people can have different one. Maybe you just didn't get in the contact with the right snake-oil salesmen - good for you!
I admit I don't know much about project management. So it's quite possible you're right, and most of the project management work is put on the team (for better or worse, I think it's for the worse, anyway).
But, what I think is crucial, is (at least for someone) to know the history. So if the scrum master is to successfully improve the process, he should have a clue about project management and what was tried with what result. Otherwise he will just rediscover Brooks (Fred, although Mel is probably also a possibility). And that's why it's bad to say things like you are saying, that scrum master is not project manager, because then you will get people who really have no clue.
But a scrum master really isn't a project manager. That's like saying it's bad to say that a stakeholder isn't a project manager. It's a completely different project role.
I recently applied to a very well-known agile consulting shop and was rejected because of certain issues with the take-home exercise. For example, in the exercise I used a priority queue, which I had implemented as a single class (just the queue, which is to say a heap - nothing else). But these guys thought it should be broken into several classes, because a single class must not have more than 150 lines of code and the implementation was slightly more.
After that, I checked implementations that are available online. Every single one was implemented as a single class, including the ones implemented by Microsoft, Oracle, etc. And why not? It does not make much sense to break down this structure further.
But these best practises have taken on a life of their own.
Agile makes more sense if you remember it was literally an attempt to port the then-newly-hyped open source development method (Cathedral and the Bazaar had just hit) to a corporate context.
So this is entirely correct: it's an attitude, not a tick box procedure.
This is a special case of a general principle. Engineering and coding are crafts, not assembly line work. A craft is a field where the individual human being matters a lot. No "methodology" will make less skilled or experienced craftspeople perform as well as better ones, though the inverse is true so good management does matter.
Culture fit, maybe? As an IT contractor / problem solver, I have often met teams more trying to follow a protocol than actually working for value. From their perspective, they were just complying with the rules set from their employer, so all fine (apart of the fundamental reason they had to call an outisder in, of course). Mileage may vary with country and industry, though.
You also need patience, understanding and forgiveness. Many rationalists don't understand these three principles are necessary for the other principles. Without them people are always guarded, because they are vulnerable. And many business environment fire first and ask questions later. That sort of business environment is antithetical to agile.
People have to accept that a lot of software development is an art and not an assembly line with predictable steps. The agile manifesto is great but Scrum gives management the illusion that software development is a totally predictable, quantifiable process.
In the end, it's still experience and judgement that counts.
Agile is (in part) based on small iterations of tasks to eventually accomplish user stories, basically, right?
Can someone explain how this has anything to do with either administration work or building a physical appliance to sell to a customer who will be using it offline? Because while both have user stories, both are things that require you to not use the agile methodology.
When you're doing system/network/database/etc administration, you have a specific request or task to accomplish. But there is no iteration. There is no, let's do a little work and see how it evolves. If you're practiced, you know exactly what to do and how long it takes. So you just do it. Spending time updating boards and stories and tasks is a complete waste. Yet I've seen Agile shoved down the throats of teams who used to simply get shit done as fast as they could, and now go through some mandatory red tape to justify what they've been doing on short staff and no time for years.
When you have to sell someone a physical product, I just can't see Agile being very useful, mainly because of the iteration aspect. Yes, you could go through all the motions and provide full features over time and watch them grow with agile, but there's very few times I've seen a product designed from the beginning reach its target goals with any sort of a redesign. In my experience, what I have seen is the Agile method wasting everyone else's time, eventually requiring a redesign.
Story time: A meniacal team lead who's trying to make a name for himself in the company sees a way to save the company tens of millions by spending only one or two million on a product. But first we need to customize the product to work for our specific needs, which means writing middleware and customizing the interfaces to give our teams what they need to use the product. Basically, turn a kit car into an actual Ferrari with the 12 cylinder engine and everything. We take six months and actually mostly accomplish it - and then the maniac decides it's time for an upgrade, while teams are trying to learn how to drive this thing. The upgrade requires massive changes, but of course we're Agile, so we iterate and iterate until the Ferrari is now a Lamborghini. It's now been a year, and we're back where we were six months ago, only with a different looking car. Only one team managed to drive the Chimera in that time.
The fallacy of iterating on user stories is that the user knows what they want, the product lead knows how to get there without it becoming obsolete, and the developers will magically replace whole systems by iterating over features that they already know will not be changing before the product is done.
Personally, I think anyone could come up with any methodology at all and they would have exactly the same success as with any other methodology. The only thing that makes it all work are teams who get their shit done and have the bigger picture in mind, and help each other. But you don't even need a methodology at that point.
Agile just seems like so many pie charts for managers, and a buzzword to repeat to gain relevance.
There are a variety of agile methodologies. Most product feature development teams find that Scrum works pretty well. But for sysadmin or devops work usually Kanban is a better fit.
Football teams don't do standups saying "I will run X yards". Each and every player on the team
-knows precisely what the objective is
-knows precisely what his/her role is and is capable of fulfilling it
-knows what s/he needs to communicate to other teammates for them to fulfill their roles and does so promptly
-is capable of assessing what needs to be done next given the current situation on the field without having to be told and takes on that task on their own initiative
-doesn't have anyone breathing down their neck saying "A point must be scored by X:XX on the clock or else!" regardless of whether the tactical situation makes it possible
Together, such a team functions like a well-oiled machine. _That_ is what Agile is trying to teach. But more often than not, few, if any of the personnel or business conditions are met to make it possible and so it's a painful, miserable charade for all involved.
I have seen Agile work exactly once in my career; it's absolutely beautiful when it does.