When Market Research and the Computing Industry came together in the mid 1960s, their first child was Batch Tabulation [BT] software, a fast and reliable replacement for counter-sorters and electronic adding machines in the production of sometimes vast volumes of cross-tabulations. Nowadays, computers are used in every part of the MR process from designing questionnaires through to publishing reports on the Internet, but by far the greatest growth has come in the area of data collection. Looking at the ASC’s software catalogue, or browsing Tim Macer’s site, you might conclude that the chief purpose of MR today was to fill data warehouses with information that someone might want to look at later. And listening to the leading MR software supplier, you might infer that in the future the researcher will carry out any odd bits of data analysis that might still be required by eccentric clients, and that the professional survey analyst was at least an endangered species – although it’s not yet clear that any reputable zoo or wildlife park is ready to attempt a breeding programme. Having been involved in BT for more than 35 years, I feel I am as qualified as anyone to speak its epitaph, but I hope that this essentially historical paper may suggest that its day is far from over.
The history of BT really starts with AGSP, designed by a team led by John Williamson at ICT in 1965. It was a high-level programming language; it was used only at ICT (later ICL) and ran only on Atlas computers, of which there were only three in the world. There were two support programmers and five user programmers. In addition to the standard HL language components such as variable declarations and assignments and flow-control statements, AGSP had specific “table definition” statements and its own macro language, making possible the generation of hundreds of tables in a few simple statements. It took a novice several months to become fully productive. Although AGSP passed out of use when the last Atlas was de-commissioned in 1973, the same year saw the first appearance of the program which John Williamson was later to say was “AGSP’s natural successor” – Merlin. Merlin is not only still around today, but sales (I’m pleased, as Merlin’s author, to be able to tell you) were actually up last year. Like AGSP, it’s a high-level language, it requires significant training and aptitude, and it’s aimed at the purely professional survey analyst. In 199x [reference to be checked] it won an ASC “shoot out” as the most powerful package on show – but it’s not for part time users. It’s used in analysing passenger surveys for the Tyne & Wear Public Transport Authority, freight shipment surveys for DHL, and Royal Mail and British Telecom customer surveys: wherever data structure and analysis requirements are of a high order of complexity, there’s a Merlin programmer at hand.
The mid 1970s saw the emergence of what has become the best-known and most widely-used BT program, Quantum. Designed and mostly written by Ed Ross, it was the first such program ever to be sold to an end-user – the Post Office, as it happened – and therein lies the story of its success. Despite programming purists’ objections to its mixture of procedural and declarative language, relative to AGSP or Merlin it is exceedingly simple to use and it can be picked up very easily. It made “in house” survey analysis practical and gave rise to the term “specwriter” to describe someone who analysed surveys (although it should be said in fairness that some practitioners regard this as a management manoeuvre to pay them less than programmers). And the choice of C as the Quantum source code language meant that the program was portable to a wide range of computers and platforms, significantly opening up a considerable market in the days before we learned that Windows Is The Only Operating System. In 1997, Ed sold Quantum to SPSS, and development has slowed as announcements about a brand new system to replace it and their other MR software products, Dimensions, have multiplied. But it’s not yet clear whether BT will be one of Dimensions’ many wonders: SPSS’s view seems to be that MR analysis in the future will be ad hoc and researcher driven. They may be right about that and they may be wrong, but a bigger concern for SPSS must be how, in what seems to be a time of significantly reduced IT investment, they will persuade customers to switch from a system with a proven track record and a huge human resource base. I’ll return to that point.
And if SPSS are aiming at the researchers as their new users, they will come up against the third product we shall look at today, Snap. Snap was developed by Peter Wills and Steve Jenkins for Mercator in the 1980s and now has so many users they’ve given up holding User Group meetings as Peter is unhappy with the acoustics at the only possible venue left to them, the Royal Albert Hall. [Review of Snap’s approach and functionality here – Mercator are happy to help with this bit.] It’s clear that Snap certainly does away with the need for specwriters: if you can use Word or Excel, you can use Snap. But it’s also clear that Snap doesn’t compete with Merlin or Quantum; either an organisation feels it doesn’t need the power of Merlin or Quantum for its analysis needs, or it wants Snap for researchers in addition to Merlin or Quantum for its specwriters.
| Back to: Top | Programme | Page last updated on 31 August, 2003 |