Sunday, 1 February 2009

26 ways to evaluate a software package

(By Anthony Miller). Another public sector procurement caught my eye last week. Aberystwyth University ran an electronic auction for the supply and installation of an Enquiry & Enrolment Management System. The evaluation criteria made interesting reading. There were two main parts – technical (70% weight) and price (30% weight). Here’s how the technical specs read:

Technical criteria: Weighting: 70 % overall: (and I will forgive you if you lose the will to live part way through this): 1a. Complex question filtering. Weighting: 5 %. 1b. Response quality monitoring. Weighting: 5 %. 1c. FAQ functionality. Weighting: 5 %. 1d. Communication timetabling. Weighting: 5 %. 1e. Contact segmentation functionality. Weighting: 5 %. 1f. Enhanced enquiry service. Weighting: 5 %. 1g. Personalised web pages. Weighting: 5 %. 1h. HTML email functionality. Weighting: 5 %. 1i. Email management. Weighting: 5 %. 1j. SMS messaging. Weighting: 1 %. 1k. Reporting & surveys. Weighting: 1 %. 1l. Event management. Weighting: 1 %. 1m. Archiving communications. Weighting: 1 %. 1n. Data auto import considerations. Weighting: 10 %. 1o. User friendliness. Weighting: 2 %. 1p. Technical knowledge requirements. Weighting: 5 %. 1q. Call campaign management. Weighting: 2 %. 1r. Integrated management information. Weighting: 2 %. 1s. Enquiry source tracking. Weighting: 5 %. 1t. Actions/communications logging. Weighting: 5 %. 1u. Online prospectus requesting. Weighting: 2 %. 1v. Online phone call registration. Weighting: 4 %. 1w. Current infrastructure integration considerations. Weighting: 5 %. 1x. Terms and conditions of contract. Weighting: 4 %. 1y. Quick answer system. Weighting: 5 %.

They got bids between £79K and £289K and awarded the contract to one Hobsons plc for around £200K (I’d love to see the spreadsheet they used to compare bids!). I guess it’s good to see the award didn’t go to the lowest bidder, but I don’t know whether the Uni’s users will like the Hobson package – ‘User Friendliness’ was only deemed worth a 2% weighting!

1 comment:

Mike Unwalla, TechScribe said...

The assignment of a 2% weighting to user-friendliness appears to be utter stupidity. (I write 'appears to be', not 'is', because I do not know all the facts.)

If the usability is poor, important tasks can be frustratingly difficult, and complex tasks can become impossible for the average person. See 'Usability, testing, and documentation' on