Usually when some tool, technique or methodology inspires an almost religious or fanatical following it's a "bad smell" for me that makes me approach things with a heightened sense of skepticism (probably from all those years of training to be a scientist/engineer)
Agile methods such as XP are definitely in a "camp" that inspires such a following and is probably second only to Ruby or Spring in that regard. Also recently a few blogs have had a bit of back-and-forth on Agile methods (for example
this and
this)
However funnily enough despite my skepticism I think Agile methods have really added quite a lot to the toolset of architects and developers.
In this blog entry I will focus on
Extreme Programming (XP) as it's what I know (so some of this may or may not apply to
Scrum,
Crystal etc.)
The Gems1) Unit Testing
Thank the heavens for JUnit. Unit testing is great for the obvious reason of testing your code to make sure it works like you think it should (duh!). It's also a great "artifact" that helps to make up a part of a regression test that can be run at anytime.
But most of all, like a good design document, unit testing helps you think from the outside-in (black box) of what you are creating and what the interface should be and how it should behave to the caller. It also helps capture in code small pieces of the user requirements and design that may not be expressed elsewhere. Invaluable!
(Note, however, as is discussed later a set of unit tests does not replace a good architecture/design document)
2) Frequent, Small ReleasesMy favorite paradigm for thinking about software projects surrounds risk, and many of the biggest risks such as user dissatisfaction, harder/longer integration, discovery of and fixing performance issues etc. are mitigated by smaller and more frequent releases. The longer you go without releasing the more chance you'll "drift" from the path to working code and user satisfaction.
The other positive outcome of this is increased team morale. There's nothing like a bunch of working releases under your belt to make your customers happy and keep you and your team positive and happy! In doing so, your customers / users become more trusting and are often willing to give you a little more slack. Conversely several missed releases is a morale and motivation killer and often introduces a lot of tension with your users/customers - exactly what you don't need!
3) Shorter ("stand-up") meetingsNow I'm not saying no to any meetings - just less of them and shorter. I've had projects where there were two 1-hour whole-dev-team meetings in the morning. Usually they ran over and by the time they were done it was practically lunch! Lunch-time and not much really achieved as a team.
Too often most of the 8-15 of us were debating minutiae that could and should have been resolved between 2-3 people.
I like quick update meetings where everyone gives a quick status - what they are working on, how its going, any surprises, anything blocking etc. It takes about 15-20 minutes. If there's something that needs resolving I make sure the relevant people meet at some point that day and during the day the project manager and I help people work on the blockers.
The best managers I have ever worked with were enablers who focused on removing roadblocks that prevented or impeded our success.
4) No overtimeThere's no "backup" for your brain - once you;re toast you're done! Full-on design, development and testing is hard-going day-in day-out. Beyond the usual 40-45 hours a week, burn-out is inevitable, at least for us mere mortals :-)
5) Frequent integrationI love this for the same reason I love frequent, small release - risk reduction and increased morale.
6) Refactoring
We very rarely write our best code the first time, whether due to lack of experience, tight deadlines or the build-up of too many patches on the same old worn-out code, refactoring is a great practice. Small classes and small methods that "do one thing and do it well" are a beautiful sight to behold. There are many great refactorings out there.
Again - frequent, small refactorings combined with frequent unit testing, integration and release help keep the code working and maintainable.
7) Constant Communication (with users and with developers)Again it's about risk reduction - unit testing is great because it helps make sure your class/method works like
YOU think it should. But is that behavior really what the requirements / design dictate? Perhaps what the requirements say isn't really what the user needs (yes that happens too!). Constant communication helps mitigate that risk.
The "Duds"
1) Pair Programming
Now I've never done much pair programming so I'm not dissing it based on it's inherent merits or any real-world experience. It's just that XP tends to get "shot down" by management the second they hear about "pair programming". I think XP would get a lot more traction if they played down this aspect. I've found XP valuable in so many other respects but can't really say so to managers for this issue primarily.
2) De-emphasis of architecture and design
I think XP, in rejecting heavy-weight methods swung the pendulum too far the other way in de-emphasizing (and in many ways rejecting) technical design. Many XP big-wigs have tried to address this apparent misconception but from everything I read about XP - a traditional design is irrelevant - whether it's big or not.
Check out good articles by
Martin Fowler and
Scott AmblerFrankly I think this can only really work when you have a really awesome, high-powered and very experienced team of managers and developers. Few of us are lucky enough to be in that position.
Clearly I have skin in the game - being an Architect. But I can't imagine how an "emergent" design can be nearly as good as a well thought-through and planned one. And I'd rather read a 20-page architectural spec (with lots of diagrams) than trawl through thousands of lines of unit test code.
That said, I do agree with XP that "Big Design Up Front" is risky
but unless you do
some serious design up front, you often don't know some of the biggest and nastiest technical risks.
However some big risks only come to light in code so a careful balance needs to be struck.
I guess the point is, I don't think you can ever truly know how much architecture/design is the right amount up front but that doesn't mean you should just throw up your hands and say "the design is emergent - OK let's start coding".
In the same way that
Eisenhower said "Plans are worthless, but planning is everything" perhaps "Designs are worthless, but designing is everything". The end product of the design will ultimately change before coding is complete but in the act of designing you uncover so many things!
3) Rejecting other traditional methods when "Waterfall" is a fact of life.Let me start by saying that by "
waterfall" I mean the simple process of sequential steps: requirements --> design --> coding --> (user) testing --> installation & maintenance.
Here's my issue with rejecting waterfall - it's like rejecting Gravity - that's all well and good if you live in a parallel universe where the laws of physics don't apply :-)
There's two issues with rejecting Waterfall and traditional methods
i) It is impossible to test (completely) without working codeUnit testing is an important part of testing but *FAR FROM* the whole picture. The hardest testing phases are Integration, System-wide and user-testing (oh yeah and performance testing). The
process diagrams from XP seem to indicate there is unit testing, and "acceptance" testing and nothing in between (see
here also). Frankly I would *NEVER* go from unit testing to user testing without something more substantial in between.
Also as anyone who has written lots unit tests know- it's very hard to map from a user test-case to a discrete set of unit tests. You would end up writing so much code it's not really worth it - why? Because of all things user requirements are the most volatile of all! Now you've written so much extra code that must also be thrown away.
So IMHO the largest part of all testing comes *AFTER* coding and unit testing. It simply can be no other way.
ii) You cannot skip designWhether you create a design or not - the second you write a line of code you are realizing a design - it may be in *your* head but it's still a design. If you are on a team of more than about 7-8 developers then to really "scale" communication a written and agreed upon design is a very helpful (I would say 'necessary') task on the path to to success.
Personally I've also found that from writing out a design I've found errors and holes in my own thinking. I've uncovered serious and major risks just below the "architectural" level - deep in a component design. Sometimes this risk will make or break a project! The brain's limitations on
perception, rationality, logic and
short-term memory are well known (at least by statisticians and scientists anyway) and also argue against the mid- to long-term viability of an "oral/aural" design/architecture.
So in essence the tasks *
HAVE* to be in the following generic "Waterfall" order
requirements --> design --> code --> testingbut that's not to say there is no interplay / iterations between and across phases but at a high level that's all it is.
As I've said before Agile has taught me that at many times "less is more" - so I tend to write smaller design documents with lots more pictures and try to always keep them under 20-30 pages or so. From there on, 1-on-1 and team meetings can help get the details out. Also you can farm out individual component designs to individual developers - each creating a small 5-10 page document. Beyond that you do get the "glaze over" effect in people's eyes.
OverallIMHO agile methods are probably most valuable not for the methodology itself but for challenging the status quo (e.g. CMM) with it's huge tomes of documentation, certification
paths and the need for so many dedicated people in "small" roles - it's no wonder consulting firms love it as they can bill for so many people!
Personally I've added several of the practices to my "toolset" especially
1) Unit Testing
2) Frequent, Small releases
3) Frequent Integration
4) Refactoring
5) Short, Stand-up meetings
But getting back to my initial point of the fervor on both sides - as with many of these things I don't see what all the fuss is about. My attitude is to take a look, stay informed, experiment with it and make a choice - take it (or pieces of it) or leave it. I've found XP to be very useful but within the context of a very "normal" waterfall model.
I look forward to hearing your collective experiences - what works for you and what does not. I'm sure I'll get some flames but I look forward to your (firestorm of) comments! :-)
As one commentator rightly pointed out I've really only discussed XP and not other agile methods (such as Crystal and Scrum) so many of my points don't necessarily apply to them. So I guess the blog entry title is a bit inaccurate :-(
other agile methods. Any recommended papers/articles/links?
