When writing the book, “Software and Vendors and Requirements, Oh My!” (http://www.lupinepartners.com/books.html), I interviewed a dozen or so software sales professionals.

I thought you might find their thoughts on references interesting. I agree with some of their comments but not all. More quotes in later blogs…David

On evaluating the vendor…

“References can be very deceiving. In the product areas that I work in, people are currently using six different versions of our product. So when you talk to a customer and that customer says, ‘Oh we hate software company X,’ well they may be on a release that is six years old and they haven’t upgraded. Or if you talk to a client that is like, ‘We love it,’ well it turns out that their accounting staff has the latest and greatest version. And so I think you have to be very careful about using references because the value is limited at best and my opinion it really doesn’t gain you very much.”

“I think a lot of our prospects fail to ask for the current product notes because current release notes will tell you what enhancements have been fixed, what workarounds are there, and what currently doesn’t work in that release of the product. Everybody knows it exists, everybody does. If you own software today, you know it is out there. But what they want to see if this future product that we keep striving to hit, and we never hit it.”

“If I was buying software from our company I would tell my sales rep, ‘I am going to make the decision on your currently released product and I want to get as many report books, manuals and release notes as possible. I don’t care about your future releases because hopefully you are going to be moving in the same direction as me, but I need to find a product that I can implement today.’”

“Be very careful about asking for references. And also be very careful about asking the users of the software how it works. I’ve had mixed results with this. A lot of times as a salesperson you never really know what a customer is going to give you as a reference. So in our market—this may not be true in all industries—my experience is that everyone pretty much knows everyone else. There are exclusive and very private companies that don’t share and don’t associate with association meetings but in general, we all know our competitors.”

“You are very much buying a relationship with the vendor. You are very much buying a relationship with the software company. The software could be great, but also evaluate the software company for who we are, the kind of people we send out, the way we talk to you and treat you on the telephone, both pre-sales and post-sales. And that should be a part of the questioning when you call my references. How do you feel that they value you? Is it just the dollar they value? Did they love you before the sale and after the sale you are just a number? And this does matter?”

Somebody asked me recently about the most outrageous software demonstration I had ever participated in. I have two…couldn’t choose between them.

The first…was my first. Way back in 1990. This particular software company came to my employer’s office with two sales professionals. One was the local sales rep – the other guy was brand new to the software company. So new, that this was his first day on the job…a bit of a gunslinger was he.

These two guys were so out of sync with each other….the low-point came when they actually wrestled the keyboard from each other trying to make a point. Very funny. Needless to say we killed them…The demonstration was so bad that it worked to their favor. I contacted the president of the software company and explained what happened. He came out and did the demo with another sales professional…and they eventually won the sale.

Interestingly in retrospect…this whole little drama was the first step in me forming Lupine Partners. More on this another time.

Lesson: If you are going to be bad…be REAL bad.

Second really bad software demonstration: As more and more technology has become Internet based, it is not unusual for software vendor salespersons to request Internet connections to be made available during the demonstrations. This particular vendor made a big deal about having the connection; processing over the Internet was a large consideration for this particular client.

The demonstration started, and very quickly we began to go through the scripts. The software vendor salesperson was talking about the speed of the processing. I looked up at the screen and noticed that the whole demonstration was running on c:\localhost, which for the technically challenged means that the entire demonstration was running locally on the vendor’s laptop. In other words, the entire Internet connection “hook-in” was a sham. I held my tongue to see if the representative would correct his misdirection. He didn’t. He proceeded through the whole day talking about the speed of the vendor’s internet product. When I held the debriefing that night, I asked the team about the Internet speed. They were very impressed. I then asked how many of them knew that the entire demonstration was local, processed on the software vendor salesperson’s laptop. The IT director raised his hand. The other people in the room were shocked—and then they were angry. We had a long discussion about product and company—good product, bad company.

The really interesting turn on this story was that three weeks later I was going through demonstrations with another client with this same vendor. The same thing happened again. It was an entirely different sales team; in fact, these were senior sales people. I held my tongue and then did the same thing at night: revealed what had been done to the project team. Same result, same anger.

All of this was avoidable. All the sales representative had to do was disclose upfront that he was going to do the bulk of the demonstration locally in the interest of speed and timing. That’s what everybody else does. But they were insecure about the functionality of their Internet offering that they misled.

The really interesting point is that the software company in all 3 examples…from 1990 to 2004…is the same company.

There are scoring systems to gauge the financial soundness of software vendors. They ask for years of financial statements and from these statements try to determine the companies’ financial shape. The determination of financial soundness is in itself a subjective determination. Does having a large cash balance mean that you are financially sound or does it mean that you have not been investing in new technologies or reinvesting back into the company? Does having no debt mean that you are financially sound or does it mean that you are missing the boat by not borrowing to take advantage of business opportunities? If you had a bad year last year, does it mean that this year is doomed? And so on.

The only thing I look at on the financial statements is the notes section—if there are audited statements from a CPA. At a minimum, ask the following five questions in the RFP:

· How much is your recurring revenue? In other words, if you do not make a new software sale this year, how much revenue would you have coming in from your existing customer base?

· How many deals do you have in your pipeline and what is your success ratio of demos to sales?

· How many sales have you made this year?

· How many sales have you made that have not yet started implementation?

· How many implementations do you have in process?

To me, the past is past. What you want to know is whether the vendor is viable at the moment and how big its customer base and recurring revenue stream is. If you want to go into a full-blown analysis of the financial condition of a software vendor, you need to determine in advance what the criteria will be for financial soundness. I have been a CPA since 1982 and I can tell you that this is a slippery slope and not easily done unless you get audited (not a compilation and not a review) financial statements from a CPA. The audited financial statements should be given an unqualified opinion.

Last week I spent quite a bit of time explaining budgeting to a client as it relates to software evaluation and implementation…wasn’t the first time I had been through this with them.

The problem was when I spoke with them in February, I used a plural and they only heard a singular.

Here’s what I said: You need to make sure and prepare your software budget(s) before you start the evaluation process.

They heard ‘budget’ – singular…and because of this, things have gotten a little gummed up.

There are actually two budgets that need to be created.

The first budget is the software evaluation and acquisition project budget. This is how much you are going to invest in fact finding and evaluation.

The second budget is the actual software acquisition and implementation budget - this includes all costs (hardware, software, consultants, present value of license fees, etc.). This second budget often is wrapped into a formal return on investment calculation. The expected return can drive the investment amount, which in turn drives the maximum amount the company is willing to invest in software.

It is within the realm of possibility that after you get preliminary numbers from software vendors (this is the vendor long list process discussed in my book), that the acquisition costs are sufficiently high to NOT warrant acquiring new software. The numbers don’t make sense – in other words, the expected returns are not high enough relative to the cost of acquisition.

My client prepared the project budget…but never went through the process of trying to quantify the benefits that would accrue with a new system.

They are still going through with their software effort…but from a corporate standpoint, they don’t know why they are, and what they expect to gain.

Was an uncomfortable conversation, but one that was necessary and educational.

Does human intuition have any place in evaluating software?

Uh, that would be ‘yes’.

What did your intuition tell you about the people who were demonstrating the product?

You just spent quite a few hours and possibly days (weeks?) with the vendor’s representatives, which may include senior executives, and maybe the firm’s founder and president.

Were they straightforward? Did you feel like they were honestly listening to you? Did they ever say something was their fault? You developed some sort of vibe about the company, and one of the things you need to take into consideration is whether you had a group vibe about the sort of company the software vendor is. How do you evaluate this vibe? You don’t. But you discuss it as a group and make sure it is one of your X-factors at the end of the evaluation effort. In 1990, I led my first evaluation effort.  One of the software vendor’s demonstrations was horrible.  So horrible that the two sales reps actually got into a fight over who was going to continue to lead the disaster…culminating in them each putting their hands on the keyboard and doing the push-pull thing.  Each trying to wrestle control of the keyboard (this was before the ‘mouse’). Our collective vibe told us that we needed to give them a second shot – with different sales professionals.  The scoring for this vendor was a zero. 

Long story, short.  The vendor did come back – brought the president of the software company with them, and did in fact win the account.  A large part of this was due to the damage control that they did…but in a small part due to the project team using their eyes and ears to do the right thing, and not JUST depending on the evaluation methodology.

Earlier in the week I was talking about a client of mine who was insistent of getting two years of financial statements from software vendors…even though they admitted they were not going to do anything with the information…

There are scoring systems that have been created that (attempt to) gauge the financial soundness of software vendors. They ask for years of financial statements and from these statements try to determine the companies’ financial shape.

The determination of financial soundness is in itself a subjective determination.

Does having a large cash balance mean that you are financially sound or does it mean that you have not been investing in new technologies or reinvesting back into the company? Does having no debt mean that you are financially sound or does it mean that you are missing the boat by not borrowing to take advantage of business opportunities? If you had a bad year last year, does it mean that this year is doomed? And so on.

The only thing I look at on the financial statements is the notes section—if there are audited statements from a CPA. At a minimum, ask the following five questions in the RFP:

• How much is your recurring revenue? In other words, if you do not make a new software sale this year, how much revenue would you have coming in from your existing customer base?
• How many deals do you have in your pipeline and what is your success ratio of demos to sales?
• How many sales have you made this year?
• How many sales have you made that have not yet started implementation?
• How many implementations do you have in process?

To me, the past is past. What you want to know is whether the vendor is viable at the moment and how big its customer base and recurring revenue stream is. If you want to go into a full-blown analysis of the financial condition of a software vendor, you need to determine in advance what the criteria will be for financial soundness.

I have been a CPA since 1982 and I can tell you that this is a slippery slope and not easily done.

From time to time I have been asked why you need a consultant, an outsider…or much less, a methodology to evaluate and implement software solutions for organizations.

I used to give detailed explanations as to why having a methodology is so important. (You don’t need a consultant or an outsider.) Now I can just tell them to read the newspaper. This little nugget passed my desk yesterday:

“State of Colorado officials have temporarily shut down a new computer system for motor-vehicle titles and registrations after discovering a glitch. The state Department of Revenue is putting off plans to roll out the $13.2 million Colorado State Titling and Registration System, called CSTARS, because it sometimes provides incorrect information to law enforcement officials.

“When we start to hear from folks that their vehicles are being impounded, we look back at the records and say, ‘Yikes,”‘ said Maren Rubino, operations director for the revenue department’s titles and registration section.

The crash of CSTARS is the latest multimillion-dollar computer bug to dog state government, and it has prompted Gov. Bill Ritter’s administration to conduct a full-scale review of the state’s process for updating its computer systems.

Last year, the federal government notified state officials that Colorado may have to repay as much as $11 million for welfare benefits that were distributed because of errors caused by the $200 million Colorado Benefits Management System. The Colorado Department of Labor and Employment is assessing whether it can salvage an unemployment benefits program, Genesis. That project, originally costing $44.8 million, was stopped in December 2005. The agency has received $2.3 million to analyze whether the program could be saved.

Ritter spokesman Evan Dreyer said two other computing systems - a payroll program at the Department of Transportation and a voter-registration system for the secretary of state – have provided big-dollar woes to the state.

The Revenue Department will work with the state Office of Information Technology, a division of the governor’s office, to hire a consultant to review the system, said Roxy Huber, executive director of the department of revenue. The review could take two to three months.”

This is ridiculous.

If I lived in Colorado, I would be screaming bloody murder about this state of affairs. Lots of ‘millions’ being tossed around in that article. Because it’s ‘computers’…people (both bureaucrats and taxpayers) just shrug their shoulders with sort of ‘well, what are you going to do – computers, can’t live with them, can’t live without them’ mentality.

I think I may call old Roxy up and see if I can’t get some of those millions coming my way…

Good grief.

About 7 years ago a very interesting evaluation occurred with a client of mine in the Pacific Northwest.

This client was very well organized and was led by a project manager who was top notch, the best I have ever worked with. Everything was done by the book; in fact, some of their practices I incorporated into my professional practice and accordingly have been included into my software evaluation book.

Was just a real joyful professional project.

We were down to final selections. This was the only engagement where I was requested to have an actual vote in the process. Usually, I just facilitate, but in this instance I had one vote and there were four other people on the evaluation team. The team voted. The result was four votes for vendor A and one vote for Vendor B.

Vendor B won.

All through the process, the understanding was that the project manager/sponsor had the final say-so but would take the advice of the committee into account while making his decision. Between the two vendors — it was not a functional tie; it was close, but it was not a tie.

Thus, the team’s vote (including mine) to go with Vendor A.

What we didn’t know was the business relationship my client had with another organization that was already invested in and using Vendor B. The project manager/sponsor used the process to prove that the functionality was good enough with Vendor B to warrant building a connection between their two systems—using Vendor B’s software.

And this was how he went about it.

If he had let this be known from the start, he would not have gotten the buy-in that he did. It was a straight-up software selection effort, but the goal was a little bit different from normal. The “winner” was chosen in advance—for legitimate strategic reasons—and was deemed the winner by coming in a close second place.

Six years later, in preparation for my book, I visited that project manager and asked him if he regretted the decision or the approach. He did not. They are still using the product and are overall happy with the result and the decision.

So, there you go.

This is going to bug some people…but I have seen this enough times over the years to know that it is true more often than not.

The fact of the matter is that the motivation of the IT department during a software evaluation is different than the CEO owner…owners want to make money, increase market share, and win. They view technology as a means to an end…it’s a tool. And there a lot of tools out there…thus the need to evaluate them.

In many cases, this job is given to the IT department of an organization.

What is the worse thing that can happen during all of this?…Answer: all the things that an owner fears. You spend a lot of money and you don’t increase market share, you don’t make incrementally more money, and you don’t win. Or worse, your company comes to a grinding halt because you can’t issue customer reports, process transactions, or track marketing efforts.

So what you have are competing forces and competing fears. One coming from the standpoint of employer/owner, and the other coming from an employee.

If the software evaluation and/or implementation goes bad…whose ass is on the line? Yep. You know and he knows it.

The sub-conscious agenda that begins to happen is that the employee who is toting the load of the software project is looking for the best, but safest, choice. They will look at revenue, number of employees, maybe price…but they probably not looking at it from the owner’s eye of increasing market share – in other words, from a business perspective.

Of the two parties in this ‘transaction’…who is most at fault?

Wrong.

It’s the owner/CEO.

And it gets back to his fears…his fear of technology and the inclination to abdicate on all matters related to technology…to a person or committee that is coming from a different place. The owner/CEO has the mandate of communicating the purpose of the software eval…and has the responsibility of either learning an evaluation/implementation methodology himself, or insuring that the project team is sufficiently educated on the process.

This missive is a continuation of my series on the role that fear plays in the software evaluation and implementation process. Today I am going to continue my examination of business owners and CEOs…and their unstated fears in all of this. What does the fear of making the wrong decision equate to? You guessed it – paralysis. The owner with the fears (and by the way it is not an irrational fear – for many companies this is the largest capital expenditure that will be making in a long time…should give you a tad to think about…) will often get seduced into study after study – either using his own staff’s time or worse, paying an outsider for it. I have clients who have spent 5 years doing this….I know because they have hired my firm to handle the implementation…and they tell me. With an eye roll and the shake of a head. This all boils down to being overly careful…because they don’t have an evaluation methodology that they trust, or because they don’t trust their staff. There are 3 ways to look at these processes:

1. Ready, aim, fire (ideal)
2. Fire (not so good)
3. Ready, ready, ready, ready….(paralysis – fear of making the wrong decision)
    

If you see yourself in any of this…there are some easy steps to move your software train down the road.