7/28/11

The Technology Hype Cycle and why you should bet your Software Career on the Cloud

You'd have to be living under a rock not to be well aware of the hype cycle in technology. Here is a list of hype cycles I've personally lived through and been affected by. I've tried to identify the year where expectations were either rising the fastest or reached their peak

1999 - Y2K
2000 - Y2K (Yes it was still pretty hot)
2001 - Java / J2EE
2002 - Linux (also Enterprise Application Integration was hot here too)
2004 - Open Source & Web Services & BPEL
2007 - SaaS
2008 - Web 2.0 and Mashups

I'm probably off by a year or two in some cases but as you can see each and every year has something new.

Of all of those hyped technologies very few of them delivered on the (marketing) promise. However I think some of them really were a net positive. On a scale of "mostly helpful" to "mostly hype" here are the highlights:

1) Open Source and Linux definitely helped reduced a small chunk of development costs. The problem is most of the costs in development are people - that said any reduction was useful. As a (Java) developer my first preference in most technology choices is Open Source because most of these products "just work" and there is enough documentation and forum help to dig you out of most holes.

2) Java was probably a net drag initially (especially J2EE in its first few incarnations) but after JavaEE 5 things are getting better.

3) SOA was a slight evolution (XML + HTTP?) however if not for the advent of REST, SOA would reside more towards hype than helpful as REST dispelled the myth (pushed by consultants and software firms) that we needed a bunch of heavyweight WS-* protocols to make SOA viable.

4) Virtualization was definitely a net positive for IT organizations and data centers - definitely more helpful than hype saving organizations lots of money.

So what about the cloud, cloud is getting hyped a lot lately. A lot of people don't fully understand it so this may surprise people: the advent of the Cloud is, in my opinion, the biggest event in software development since Tim Berners-Lee proposed the world wide web .

Take that in for a minute. I am willing to bet my public reputation that the Cloud is going to be a HUGE sea change for IT organizations and for developers. Bigger than anything in the past 12+ years.

As a developer if you are not getting on board with this you are gonna be left behind. This is like being a N-Tier C++ programmer in 1994 as the web was starting to gain traction. Get in with the cloud NOW!

Why? Cloud solutions (computing, storage etc.) allows companies (especially large ones) to reduce and perhaps eliminate many of their data centers and supporting IT staff. For smaller companies the "Pay as you go" model (converting CapEx costs to OpEx) and the "elasticity" support to allow you to scale-on-demand are exciting. Venture Capital companies are now demanding the companies they invest in to use the cloud - it helps to reduce costs now and scale more later without large outlays. So who is NOT going to get on board this trend? I can't think of any organization that cares about costs / growth who will not ultimately want to adopt this model.

Also as the number of devices per person (Tablets, laptops, smartphones) continues to accelerate the "elasticity" demands will become ever more prevalent for large enterprises as well as Web 2.0 companies. True you can't use more than one device at a time but devices are getting smarter and creating work themselves (to keep your email, rss feeds, facebook status all up to date etc.).

This is like when Electricity distribution took off with the advent of AC (alternating current) it was critical for economic growth. True the Cloud may take 3 or 4 years to REALLY take off (1994 in the web wasn't nearly as exciting as 1999) but when it does . . . .

And when stuff is hot (like cloud is, or is going to be) the pay goes up up up and the firms that drive value in this space are going to be worth billions.

There are a number of Cloud providers and it's growing almost daily - but if you want to get on board I suggest you start taking a look at Amazon EC2 and Amazon's Elastic Beanstalk (for Java folks). Back in the nineties they used to say "No one was ever fired for choosing IBM" - not that IBM was flawless but it was hard to argue they weren't top of their game for Enterprise software (back then). Today I think the same is true of Amazon - not perfect but compared to the other providers they are clearly on top of their game.

Getting knowledge of AWS and understanding the very different architecture, design and coding strategies (e.g. security, privacy, failover concerns and NoSQL, Hadoop etc.) will help you transfer more quickly over to whatever is "next" when invariably Amazon gets surpassed by someone with a better mousetrap.

I still have a few doubts. Invariably there will some mini-Bubble in cloud company valuations - in the hype cycle there will be a "Peak of expectations" and a "Trough of disillusionment", but in the end the "Plateau of productivity" is going to be a very good (but very different) place to be. And when a large wave of change is coming - it's better to be ready with a surfboard and paddling ahead of the wave, than sitting on the beach waiting for it to come ashore.

7/13/11

Scrum Purity Test

OK so just for fun (malicious fun!) I decided to create a Scrum purity test. The reason is I am pretty sure a LOT of people out there SAY they are using Scrum but aren't really - or they aren't using all the tools and techniques involved. Fair to say, I have never fully utilized Scrum in any place I have worked - mostly for political reasons - usually because of lack of buy-in from Senior Management. That said, we've made it work and I'm sure others have too.

But let me say that I don't think you HAVE to adopt Scrum 100% to get value from it. Small iterations with defined deliverables and constant communication and accountability can go a long way regardless of what you call it.

Anyway here is my take on a "Scrum Purity Test". There is another take here.

FIRST AND FOREMOST
Q: Do you have Sprints at the end of which you deliver (roughly!) shippable functionality?
- No - Then don't do this test - you've already lost. I don't think you can consider yourself agile without small iterations.
- Yes, but the Sprints are 4 weeks long and the code has lots of bugs - zero points
- Yes, the Sprints are under 3 weeks and the code has some bugs - 5 points
- Yes, the Sprints are under 2 weeks and the code is tested (but has some small issues) - 10 points
- Yes, the Sprints are under 2 weeks and we release to production - 20 points

MEETINGS
Q: Do you have a Daily Scrum Standup meeting?
- No - zero points
- Yes but it goes over 15 minutes - 5 points
- Yes, it's under 15 minutes and we answer the three questions - 10 points

Q: Do you have a Sprint Planning meeting?
- No - zero points
- Yes kind of, it's kind of ad hoc - 5 points
- Yes Ken Schwaber would be proud - 10 points

Q: Do you have a Sprint review meeting at the end of a Sprint?
- No - zero points
- Yes - 5 points

Q: Do you have a Sprint Retrospective to identify what can be improved?
- No - zero points
- Yes - 5 points

ARTIFACTS
Q: Does your team maintain a Product backlog?
- No - zero points
- Yes, kind of, it's ad hoc and it lacks priority- 5 points
- Yes its prioritized, updated regularly, visible to all - 10 points

Q: Does your team maintain a Sprint Backlog?
- No - zero points
- Yes, but features aren't broken into tasks - 5 points
- Yes, features are broken down and well understood before beginning - 10 points

Q: Does your team maintain a Burndown chart
- No - zero points
- Yes, kind of, we track it roughly in an ad hoc fashion - 5 points
- Yes, we use some tools such as Greenhopper or Excel and its visible to all 24 x 7 - 10 points


ROLES
Q: Does your team have a Scrummaster?
- No and we could use one - zero points
- No we don't need the protection / impediment removal (e.g. in a startup) - 5 points
- Yes we have one and we also report to her (e.g. a Project Manager) - 5 points
- Yes we have one and it's not a person we report to - 10 points

Q: Does your team have a Product Owner?
- No - zero points
- Yes but they really don't understand the user - 5 points
- Yes and they know our user base inside and out - 10 points


SCRUM-BUTS
Q: Does your team do Estimates?
- No - zero points
- Yes and we estimate using hours and days using MS Project - 1 point
- Yes and we use planning poker and our estimates are within 15-30% - 5 points
- Yes, we use planning poker and our estimates are within 10% - 10 points

Q: What are your Specifications like?
- Big requirements docs - zero points
- Good user stories - 5 points
- Just in time requirements / stories - 10 points

Q: How is testing performed?
- We have No dedicated testers on team - zero points
- Developers Unit test only - points
- The feature is tested soon after Sprint is complete - 3 points
- Software passes acceptance testing as Sprint ends - 8 points
- Software was tested and is deployed to Production - 10 points


Now what is your total - out of 130 points? What grade would you give yourself on the following scale?

110 - 130: A - you are doing a great job following Scrum (just remember to make sure your users and stake holders are happy)
90 - 110: B
70 - 90 : C
50 - 70 : D
< 50: F

I have to say I thought we were doing pretty good but when I scored my team we got a 59 a D! Yeouch! That said we ship on time and everyone is pretty happy but clearly we are not doing great following Scrum.

What do you think? Are there any critical aspects of Scrum that I missed? Any items that deserve more points (or less)? I am being unfair?

POSTSCRIPT
Some folks have been kind enough to get me searching for "the Nokia test" which has thrown up the following