Many times, am asked to respond to loose questions like which CMS/Product would be the best – if you say can’t say , next question is which sells most, which gets selected more after structured evaluation.Every situation is unique. several times everything falls in place, finally budget is not there. Start all over again.Other occasions, budget is too tight -so choices get minimised..Tony Byrne points to Technology Selection Pitfalls. Excerpts with edits:
Pitfall #1: Creating a 1-dimensional selection team
The traditional divide between IT and Marketing - or IT and "Business": Business has the budget and picks a tool without adequately consulting IT, or IT is given sole responsibility for finding the right vendor and doesn't adequately involve Business, or Business simply won't participate. The best software selection teams are multidisciplinary. Depending on the size of your organization, that team should include:
System / Database Administrator ,Developer ,Enterprise Architect ,Content Contributors (ideally more than one) ,Editor ,MarCom Manager etc.
Pitfall #2:. Becoming slaves to spreadsheets :Spreadsheets can be useful tools product selections - as long as you employ them judiciously. Over reliance on long spreadsheets can lead you down the wrong path. This typically happens in two places - RFPs and vendor scoring. In RFPs, it is tempting to try to challenge vendors with long lists of desired features and attributes. Here's a secret: vendors have seen all your requirements before, and I assure you that they have very attractive answers.
One of the rows might say, "Supports the Firefox browser." Nearly every vendor will check the box. Many selection teams also use spreadsheets to determine finalists by applying weights to criteria, scoring how bidders perform, then adding up the math. Some government procurements require this type of approach. Try to be more flexible. For one thing, selection team members will emerge from the competition with favorites, and they frequently reverse-engineer the equations before entering scores to give their preferred vendor a high total.
Pitfall #3: Failing to test the proposed solutions -Buying software without trying it out is frequently compared unfavorably to purchasing automobiles. "Would you buy a car without test-driving it first?" Indeed, technology selection teams often assume they have to make a decision based on written proposals and inevitably thin, generic demos. The fear of making a bad choice with incomplete information creates tremendous stress. At its best, the test phase becomes a cooperative search for understanding how a particular tool fits your environment. At its worst, it will uncover the difficulty of working with outside suppliers under the press of a deadline.
Pitfall #4: Focusing too much attention on the product - and not enough on the vendor – The relationship with the winning vendor will have a greater influence on your long-term success.. Software companies (and open-source projects) have strong personalities. One vendor might excel at core software engineering while the other focuses on customer-driven feature development. One vendor promotes a large and active consulting arm; another works primarily through local systems integrators .. Good read