Can this be called agile?

Background: corporate I.T. talks about wanting to become agile. It has already stopped using the traditional waterfall methodology (requirements, design, development, test, implementation...). High-level requirements are documented. Some development groups document use cases, others document screen shots, data flows...; often the developers simply meet with and email back and forth with customers to understand what's wanted. Most development groups next include the customer when they have questions, or when they are ready to show functionality to the customer. Note, though, the following:

• I.T. follows a standard, hierarchical organizational structure. There is no use of self-organizing teams.
• None of the development groups use pair programming.
• Some groups have developers on the other side of the world. Of these groups, some may have daily scrums or stand-ups via teleconference - though it should be noted that customer representatives do not attend these daily meetings. It should also be noted that there is little overlap in working hours with the remote development staff.
  
• None of the groups have their customers sitting near them during development.
• No one has the title of "project manager" or "scrum master". Customer representatives act as project managers for some projects, and senior developers take on that role for other projects.
  
• Some groups support single applications. Others support many applications. Of the ones that support single applications, some release enhancements on regular schedules (1 - 2 months).
• There is talk about standardizing on using use cases or feature cards and working with customers to prioritize these.
• Large projects (over 100 hours of work) have to go through an approval process, requiring specification of effort, risks, ROI, ...
• Continuous integration, automated testing, and test driven development are NOT being used.

While I.T. has definitely moved far from old-line development methodologies, it seems to me there is a lot lacking with respect to abiding by the agile philosophy.

What do you think?

SharePoint for Internal Collaboration? Fail!

There's a lot of buzz about SharePoint in the corporate world. It seems to have come out of nowhere, but I believe SharePoint has hit its stride because it provides a safe answer for IT management when asked about their strategy for web 2.0 in the enterprise. It's a clear continuation of the IT management thought process that's been around since the '70's - what started with the statement: "you can't go wrong if you buy from IBM." The other cya vendors - IBM, Oracle, CA,... Haven't had a good story here, so Microsoft has won by default.

The only problem is that SharePoint sucks. Ok, maybe that's a bit harsh. SharePoint is ok as a document repository. But really, it does suck at facilitating collaboration in the enterprise. If you've followed my writing, you know that I don't normally use harsh adjectives or make overly bold statements. But this one is easy. SharePoint is just not in the same league as so many other tools. Let's go through some of SharePoint's disadvantages.

Web 2.0 - the philosophy

The premise behind web 2.0 is that:

• People are more than executors of commands - they have important and relevant knowledge, and they are sources of good ideas.
• Significant value is gained by allowing people to collaborate, work out problems on the open stage, and make that knowledge available and easily findable. Better decisions can be made (people will have better information with which to make them), decisions and tasks can be completed more quickly (they will spend less time finding relevant information), and the organization will be less brittle (knowledge will be stored centrally, rather than solely in people's heads).
  
• People - on the whole - are smart and ethical.
  

A change in management culture is often required of organizations that want to take part in this. Historically, organizations were based on "command and control", with jobs and tasks specified from the top down; and information was shared "on a need to know basis." But Web 2.0 in the enterprise requires trust, a willingness to listen to the ideas of people at every level of the organization (and to act and allow action on good ideas), and acceptance of a more open, transparent, and collaborative environment.

SharePoint - Web 2.0 or not Web 2.0?

With roots in document management, SharePoint provides for significant security configuration. While this should be a good thing, in practice it makes it too easy to secure content from general access. Wikis and blogs generally default to open access, and wikis generally default to open editing. This latter approach is better aligned with the philosophy of Web 2.0. You can do this with SharePoint, but you may have a more difficult fight with those who want to stick with access "on a need to know basis."

SharePoint TCO

Proliferation of SharePoint sites is also a common problem in organizations. It's easy to find yourself with dozens - or hundreds - of sites. And unless you know about them or they are referenced elsewhere, they remain hidden from view. Microsoft's response to this? Put in place a governance framework. Microsoft's web site has a whole set of documentation on governance for SharePoint. Of course with this you get added overhead and bureaucracy. Oh, and there's 3rd party commercial software available to help with this.

Which leads to the next issue with SharePoint: it is definitely NOT a "light" solution. Technically, Microsoft recommends development, test, and production environments; you'll need Active Directory, IIS, the .Net Framework, SQL Server databases, and antivirus software for the server. Microsoft kindly provides a sample deployment project plan for deploying SharePoint - 381 tasks requiring 129 days and people in 9 different roles. Organizationally, Microsoft recommends maintaining an ongoing staff of people for the roles of business analyst, creative designer, trainer, infrastructure specialist, developer, and architect.

SharePoint as Antagonist to the Flexible Enterprise

This may be obvious, yet I feel it still needs to be said: SharePoint only runs on Windows. In contrast, many other collaboration tools will run on Windows, Solaris, or Linux. Oh, and if you want integration with Microsoft Office, your users will need to be running Windows (and Microsoft Office).

Productivity Enhancer? Or Drain?

SharePoint's roots are as a document management system - a place to store your Word, Excel, Powerpoint files. Microsoft knows it has a cash cow in Microsoft Office, and it's not going to allow the product to be be put out to pasture without a fight. And Microsoft knows nothing better than how to fight. In this game, Microsoft is building hooks between SharePoint and Office - Microsoft makes it easy to save a file to a SharePoint site, to open and edit a file stored in SharePoint, and to save changes back to SharePoint; and your calendar, tasks, and contacts can be synchronized between Outlook and SharePoint (of course, if you want this latter functionality, you will also need an Exchange Server and Active Directory). Microsoft hopes this keeps you captive. If this were a good solution, so be it. But it's not.

When you click on a link, you expect to see content relating to the text of the link. But with Microsoft's solution, you get a popup window asking if you want to open or save the file that contains the content related to the link. Failure #4. Now I don't know about your experience with browsers, but I've found that opening documents in my browser (IE, Firefox, Opera, ...) is hit or miss. Sometimes it works. Sometimes it causes my browser to grab the CPU and not let go, sometimes it makes my browser greedy for RAM, sometimes my browser crashes. This has trained me to save all downloadable files to my hard disk, and then open them in their native apps. Failure #5. Of course when I'm done looking at the file and I move on, the file is still there, taking up disk space on my PC. I have to periodically empty the folder I've set up for these files - failure #6. And until I do this, these files increase the bandwidth, time, and storage space required by our corporate backup solution - failure #7.

Now let's compare SharePoint with wikis with respect to versioning. With SharePoint your content is in a file. If you want to make a change, you download the file, make the change, and upload it. If you want - and if you know how - you can have MS Word store a record of changes in Word documents. Excel has a change tracking feature, though it's somewhat crippled (e.g. it can't track formatting changes, when you use it some other features become unavailable...). PowerPoint doesn't have a change tracking feature. Additionally, when you save a file back to SharePoint, it doesn't delete the old version. So you could conceivably compare the versions side by side.

Using wiki applications, you write content directly on the page. If you want to make a change, you click an "edit" button, make the change, and click the "save" button. Most wiki applications keep both the old and new versions, and they make it easy for you to see what has been changed.

So it's not that you can't view content changes with SharePoint, it's just that it's so much more difficult than with wikis. Failure #8.

SharePoint as Virus Vector

One more issue to chew on: Microsoft Office documents can be carriers of malicious code, whereas wiki pages have much less risk. Microsoft includes "Forefront Security" with SharePoint, but why should you have to increase your risk, and install - and administer - even more software for your solution?

Bottom Line

If all you want is to enhance communication and collaboration in your enterprise, there is a plethora of tools available that could satisfy your requirements and protect you from all of the problems with SharePoint (for example, check out Atlassian Confluence or Mindtouch Deki). Don't make the mistake I've seen in other organizations - choosing SharePoint without understanding what they were getting into and what alternatives were available.

Consider yourself warned!

IT and organizational strategy

Yesterday I attended the InformationWeek500 Virtual Event. In one presentation, Rob Preston, Editor-in-Chief, discussed survey findings regarding the status of CIO's in their organizations. Predictably, the survey reveals that IT is seen as a cost center rather than a provider of strategic capabilities, and CIO's are regarded less as equal members of the executive team than as managers of utility services.

The problem with this common belief is that it is self-fulfilling. IT is not included in strategy determination, and it is not funded in such a way as to be able to provide strategic capabilities, thereby proving the original belief. This is harmful to organizations, as they are missing out on opportunities to differentiate themselves, their products and services from their competition's; to make significant improvements in their processes and productivity; and to provide increased value to their customers and shareholders. And missing these opportunities is undoubtedly financially harming these organizations.

I asked Rob how we could educate CXO's on a larger scale than individually. He stated that he sees no other way; and that if the organization doesn't see the value of IT, move to another organization. Though frustrated by his answer, I know I can't expect an easy answer to this problem, given its prevalence and longevity.

However, I don't think we should simply throw up our hands and ignore the issue. All this would do is serve to let this less-than-optimal situation continue unabated. So here's my thought: we need to step out of our IT thinking, and try to think like marketers. We need to stop complaining amongst ourselves and start educating our business partners. But in my experience, simply providing these people with a vision of the value we could provide doesn't convince. Rather, I think we need to start compiling real-world examples of the value IT can provide - as well as real-world examples of the downside of ignoring IT. Here's a start:

• Harrah's Entertainment has measurably increased sales and customer satisfaction by gathering and using detailed information about its customers and their transactions.
• Dell became a leading pc vendor by using IT to create a super-efficient and flexible supply chain. It is now using IT to host a web site allowing its customers to have a say in the products it offers.
• Amazon.com was able to become one of the world's the largest book sellers and major retailer because of the IT infrastructure it put in place to take orders, suggest related products, and quickly fulfill those orders. Now it uses that infrastructure to enter new markets, such as providing ecommerce hosting and offering computing services.
  Wal-Mart first connected its stores together in the 1960's(!), implemented its own satellite network in 1987, and continued to use IT to track inventory, stock shelves, and develop an efficient supply chain. Wal-Mart is currently a significant influence in the adoption of rfid for tracking product.
• GM has leveraged IT to "cut the delivery time on new vehicles from 70 days to 30, and saved the company millions per year in crash testing by moving to digital simulation."

While it's possible rattling off examples such as these may have an effect on our business partners, wouldn't it be great if the big IT consulting companies such as IBM, HP, and EDS would create marketing campaigns highlighting the importance of IT?

agile implementation versus releases

I've noticed this conversation coming up more frequently lately - business customers and even IT managers stating their desire to have IT development groups schedule software releases every so many months. They state this as a preference over what I call "implementation as completed" (similar to agile's short development cycles, but more flexible). Note that I am talking about development and enhancement of internal applications, not software for sale.

I've stated my disagreement with this direction. I've won some people over, some not. It seems so clear to me, but either I'm not communicating my argument very well, or it is a difficult concept to grasp. Maybe putting my argument in writing will help. So here goes:

I have 3 arguments for implementing functionality as it is completed:

1. You have more flexibility in adapting to changing priorities. The people I'm talking about want to have an up-front process for selecting functionality to be included in the next release; and then have that functionality made available on the release date. The problem with this is that it locks you into that specification. Sure, you can change the scope after the work has begun, but that requires negotiations, reestimation of the work and time required, and possibly re-work. In contrast, my preferred method is to maintain a prioritized list of desired functionality, and to work on these requirements in prioritized order. If priorities change, you either lose only the time put into the current task, or you only have to wait for the current task to complete before the new highest-priority task is begun.


2. Using the release (or "big-bang") method, the greatest-needed functionality is made available at the same time as the least-needed functionality.


3. My last argument is similar to the previous one, yet subtly different. Using the big-bang method, you get all of your functionality on the release date. Even if all of the functions have the same level of need, you still end up losing value. I've created a very short and simple spreadsheet to exemplify this. A screenshot of the spreadsheet is below. For the sake of this example, let's say you want to have 6 functions implemented, and each provides a value to you of $10/month. Let's also say that the 6 functions each require a month of work. Using the big-bang method, you get no value until the release at the end of the 6 months. In contrast, the spreadsheet below shows that, by the end of the 6th month, you've received $50 of value from the implementation of function 1 at the end of the 1st month, $40 of value from the implementation of function 2 at the end of the 2nd month, and so on. So whereas in the first 6 months you get $0 of value using the big-bang method, you get $150 of value using the implementation as completed method.
   
   Lastly, note that the example assumes that each function provides the same level of value. In reality, the implementation as completed method compares even more favorably because the greatest-value functions will be implemented first, providing an even greater value difference. For example, if the first function below had a value of $20/month, you would receive $100 of value from that function alone during the time you would be otherwise be waiting for the release.

If you have arguments for the big-bang method that you believe override my arguments above, I'd love to hear them!

Innotech - Day 2

Only got to the morning sessions on Thursday - family emergency kept me away from some afternoon sessions that looked like they'd be good...

Agile Project Experiences
So many discussions and articles about the superiority of the agile methods lack the depth and details I am looking for - and this panel discussion did nothing to satisfy my thirst. Am I the only one feeling frustrated? Like other discussions I've had and presentations I've seen, these panelists talked about how they could do things using agile methods they couldn't do otherwise, but they didn't specify what those things are; they said their velocity increased, but they admitted that they couldn't measure this; and they said the quality of their solutions increased (in terms of bug rates), but they didn't have hard numbers. Regarding that last issue, Arlo stated that they pretty much just don't find bugs any more, but I'm more than a little skeptical about that.

My second issue is that agile proponents always compare these methodologies with the strict waterfall methodology. But I haven't seen a strict waterfall project in over 20 years. Since those early projects (at GE), every project has had constant communication with customers and other stakeholders, iterative prototyping or development, functionality prioritized and built in order of importance or risk, and/or intra-project negotiations with respect to change requests, scope changes, etc. So it seems to me the real question is one of asking about the benefits of a strict agile methodology (which is what I always hear profferred) versus the more amorphous, flexible development progression that seems common in real life.

And now my last issue: so far I haven't heard a satisfactory answer for how, using agile methods, to provide management with the information they want. At this session the panelists said management will be happy when they see the increased velocity and the list of functions completed. Is my experience with management unusual? I've never been involved with management that would accept that. In my experience management wants to be able to plan future work and the resources needed for it (i.e., project portfolio management). That means they want estimates for the length, value (e.g., ROI), and resource requirements of every project. And they want projects managed to those estimates.

Maybe my issues exist because of the space I work in - corporate, internal development. I can see how agile methodologies could work well in commercial or open source software development. For example, Microsoft provides general guidance as to the functionality they will include in their next operating system versions, but as development progresses, they drop capabilities to help reach their desired ship date. And these methods seem even more natural for open source software development. For example, I can see how Linux development is best done using agile methods.

There's so much more I could write. But I've rambled enough.

Wiki Then and Now
Ward Cunningham gave a two-part presentation. The first was a historical perspective of his invention of the wiki and his involvement in the development of agile methodologies. Fascinating and impressive.

The second half was a discussion about aboutus.org, the company founded by Ray King and for whom he currently works. This company troubles me. They scan domain registrations and create a wiki page for each domain. And they allow anyone to edit those pages. In his presentation Cunningham said "you've been drug out [into the public], and we're here to help you." But they're the ones who have done the "drugging"! I may rile some feathers, but I just don't see value in their site for companies:

• To be listed, you have to have a domain. But if you have a domain, you most likely already have a web site, so you don't need AboutUs to advertise your existence.
• AboutUs might advertise the value of allowing your customers to comment on your companies services. But if you want, you can provide this capability on your own web site. And you'd have control over the publication of these comments too.
• People who search for your company may choose AboutUs' links rather than yours - do you see any value in that?

And companies' customers may not see much value either. AboutUs might argue that you can see comments - both positive and negative. But smart companies will lock their AboutUs pages (Cunningham said you can do that, but their web site contradicts this statement) - or have them deleted (Cunningham said they would do that for companies who request it - though the site contradicts this statement).

Given all this, I highly recommend that, at minimum, companies actively monitor their page on AboutUs.