Making Claims
Feb 11th, 2007 by abbot
“First they ignore you, then they laugh at you, then they fight you, then you win.”
–Mahatma Gandhi
Do you remember the flying car? I know the people from my generation grew up with dreams of one day having such a fantastic automobile. Over on Technorama, one of their regular contributors, Bruce Barr, points out an article by Julia Laton, “Are We on the Brink of the Flying Car?” According to the article, an Israeli company names Urban Aeronautics is claiming they will have a flying car on the market by 2012. The craft is designed to fly for up to two hours on one tank of gas, at up to 155 miles per hour (250 kph) and 12,000 feet (3,700 meters). It will cost $1.5 million. Here is the interesting thing, currently it can only hovered just 3 feet (1 meter) above the ground.
Mike Rothman, author of the Pragmotic CSO, and blogger of Daily Incite had an interesting point in his Pragmatic CSO Weekly posting. He, like many security professionals, have spent this week attending the RSA Conference. Mike writes:
Which goes to the topic of this week’s pep talk - don’t believe everything you hear. For those of you familiar with my research at Security Incite - you know I’m pretty cynical about pretty much everything. I’ll admit I was born cynical and sarcastic, but being in the security and networking business for the past 15 years hasn’t really helped soften my edge.
That was very apparent on the show floor, where vendors were resorting to all sorts of tricks (including of all horrors, booth babes) to gain the attention of potential buyers. And once they have your attention, their objective is to keep it. And sometimes they make claims on the show floor that don’t necessarily hold up in the lab. Empty claims don’t help you to do your job any better.
I would also add to Mike’s statement, do not include facts that you cannot backup in a presentation. This came up this week. A gentleman was preparing a presentation and wanted some facts on the cost savings of ITIL. Wouldn’t you know, he got a response from the ITIL expert within his company quoting itSMF, “Up to 70% reduction in downtime, 1000% return on investment, and time savings of 50%.”
If I was in the audience during this presentation, upon hearing such I quote, I would stop believing the presenter. The use of the word, “up to” makes any claim possibly true while making the statement meaningless. You could have “up to” 99% reduction in downtime and a trillion percent return on investment. Chances are real good that you won’t. When I go into a presentation, the last thing I want is to be caught off guard. There is always someone who has read other numbers/statements and they will want your response. If you cannot respond to that person, you will lose the rest of the audience. The 70% reduction in downtime and 1000% return on investment, are such amazing numbers, it rings of hype. If the audience thinks your presentation is full of hype, the credibility of the presentation suffers.
I have mentioned this site before, but I have to point to it again. The IT Skeptic site goes after the hype around ITIL. Concerning the claims around ITIL, the IT Skeptic wrote an interesting posting, “The Emperor has no clothes. Where is the evidence for ITIL?” There is even a podcast.
I am not saying one should believe without question what the IT Skeptic posts. Don’t believe the IT Skeptic, the itSMF, the folks at RSA, or someone telling you the flying car is just around the corner. Just dig a little before quoting numbers. When I pointed to the IT Skeptic I got the comment back about him being a “ghostwriter.” Focus on the message, not the messenger.
Imagine if we could invent a tablet that one could take to the RSA conference. It is capable of speech recognition and everything someone said is translated immediately into written words. Then the tablet can add links to the subject areas all the way back to the sources. As the salespeople talk, you could checks your table to see where all these numbers and ideas originated. That is better then a lie detector. I would trade in my flying car for such a device. One of the great things about blogs is that they can include links which one can easily follow to the source. Alot of salespeople would be out of work if we could do the same thing to the spoken word. Some fast talking executives would be demoted back to the mail room. In the meantime, do your homework and check those numbers. If it sounds too good to be true, it is.
There are alot of areas in security where I can argue both sides of an issue. I am using ITIL only as an example. Any area of business where you are changing the fundamental way you do business might prove difficult to quantify ROI. A discussion of metrics involving SOA can be found on Dana Gardner’s BriefingsDirect titled, “Panel of IT analysts look to the movie business to explain SOA’s relevance and ROI.” One of the great things about Dana Gardner podcasts is that he makes full transcripts of the shows available. The panel starts off with a discussion on a statements that Verizon had come out that with a stable of 500 services that they were expecting to yield $20 million in savings over two years.
It is a valid argument to point out that when you are changing the fundamental way of doing business, it complicates how you might determine ROI. In relation to SOA, Steve Gorone raises the question, “How do you actually quantify what your ROI is, given the advantages of using an SOA approach? I’ve listed the main reasons why people would want to do SOA, in terms of the advantages, and they basically break down to four major areas.” The four reasons are:
- The reuse of IT assets
- Reduce the expense associated with doing the application integration test they normally would have to do
- Meeting compliance requirements
- The issue of how agile do you make your business
Of course, my main interest in SOA comes from the last two claims of compliance and agility. Another key point that interest me is the idea that Tony Baer points out:
If there is one benefit that SOA delivers, it’s that the value becomes the service rather than the plumbing. If you think about the way we’ve traditionally developed functionality or integrated systems, we’ve had to spend inordinate amounts of time in the plumbing and maintaining it. SOA theoretically, if it’s done right, standardizes the plumbing, makes everything declarative, so you take out the guess work. The result is that if you look at outsourcing, SOA separates the plumbing from the service. Therefore, what is probably ideal for outsourcing would be the plumbing, because that’s where the value is and that’s not where IT organizations should be spinning their wheels.
That quote from a security point of view is very interesting. Alot of our efforts have been on the plumbing. While there is a great need to secure the plumbing, what is being done to secure the services?
The point is, IT is not the same as manufacturing. Metrics generally are not as simply as the replacement of one machine with another that can produce more widgets at less power consumption. For example, how does one measure the value of agility? If you fail to be able to adapt and provide one of the latest web 3.0 services, how many customers will you lose? How do you measure customer satisfaction because it was easy for the customer to get to you? Present information and selling points, but also be aware of the arguments both ways. This is the only way to insure that you won’t be taken off guard. Plus, it shows that you are not just a salesperson. You truly do know the subject matter.