Wat is wetenschap en hoe doe je wetenschappelijk onderzoek? Bij de beantwoording van deze vragen lijken sociale wetenschappers slechts twee smaken te kennen.
Vooral bij economie krijgen studenten voornamelijk onderwezen hoe je wiskundige modellen moet maken, interpreteren en ‘testen’; alsof daarmee de kous af is. Elders in het gamma-domein krijgt methodologie juist teveel aandacht, zodat de studenten door de bomen het bos niet meer zien. En zijn veel methodologen scherpslijpers en haarklovers geworden die elkaar verketteren in plaats van anderen te leren, en zo nalaten studenten beter van dienst te zijn?
In dit artikel heb ik voorrang gegeven aan beknoptheid en ‘de gemene deler’: zaken waarover in hoofdlijnen weinig verschil van mening lijkt te bestaan. Excuses voor mijn Engels, dit stuk heb ik oorspronkelijk geschreven voor de studenten die ik bij hun scriptie begeleid. Het bouwt voort op mijn artikel over Clayton Christensen.
S. de Beter (13 maart 2020)
Descriptive research is basic
A scientist is supposed to make solid and reliable claims about the world. Or at least about some part or aspect of reality. Business scholars, for instance, only make claims about the business world. He/she may answer three kinds of questions: 1) what is going on? (descriptive research), 2) why is it going on? (explanatory research) and 3) what can be done about it (evaluative or prescriptive research). Note that social scientists (including economists) should not only study human behavior ‘as we see it from the outside’ but also the way people think and feel about their own and other people’s behavior (choosing the right combination of observing the ‘objective’ and the ‘subjective’ world is crucial)
Should research always starts with the first question? Of course! Without description there is nothing to explain or prescribe, and a bad or incomplete description means that you are looking for the ‘wrong’ explanation or solution. So we shouldn’t dismiss the first stage as ‘mere description’ (the title of a good article about descriptive research). However, most ambitious scientists want to enter the explanatory and prescriptive stage as soon as possible, even if the descriptive stage doesn’t deliver enough relevant and reliable outcomes. For instance, replication studies have been very uncommon and unpopular until now, especially in economics (also in business studies?)
What are the basic steps in descriptive research? First of all, you need data about the phenomena you want to investigate. For his research on disrupting innovation Clayton Christensen collected technical and business data between 1976 and 1992 on all the products and producers in the market of floppy’s and other digital storage media. Next step is categorization, or classification as some scholars call it. Basically, you look for relevant similarities and differences in your data (as always, relevance is a practical matter). Christensen, for instance, used technical data to make a distinction between sustaining and disruptive innovation and business data to distinguish between entrants (new firms on the market) and incumbents (active on the market for some time).Third step is looking for a pattern. Suppose you observe, like Christensen did, that some incumbents stay in the market if innovation is mainly sustaining and loose their market share if there is disruptive innovation. Then you want to know whether this is coincidence or a more general pattern. In other words, you have developed a hypothesis and you have to collect data to test this hypothesis. If they are confirmed you have found a solid pattern – without knowing how long this pattern will exist and how this pattern can or may change.
Having found a consistent pattern, again and again, are you able to make predictions? This is the ultimate goal of science according to some scholars (like economist Milton Friedman). Of course you can, but only by assuming that “the pattern we have found in the past, will also be present in the future (but I don’t know why)”. We need to discover the causal mechanism(s) if we want to make solid predictions. Even more if we want to give unambiguous prescriptions, which is the highest goal for (business) science, Christensen says.
If your descriptive hypotheses are confirmed you may go to the explanatory stage. Some people call this step abduction: finding the most likely (and most simple) explanation of the pattern found in the descriptive research. Christensen made this step by looking for anomalies: individual cases that didn’t match the general pattern he had found earlier. Like IBM, one of the very few incumbents not perishing when disruptive technologies entered the market for computers. The reason: IBM founded a new business unit that was not obliged to comply to the general business model of IBM. On the contrary, the people working at that new business unit were free to develop a new business model, adopted to the possibilities and constraints of the new disruptive technology.
By finding similar anomalies Christensen discovered a general explanatory pattern why incumbents lose from entrants when a disruptive technology comes in: in the short term, the current business model of the incumbents makes it attractive to get high-end customers (with high margins); and doesn’t make it financially attractive to pay so much attention to low-end customers, who are not prepared to pay for all the advanced features the incumbents can offer. As a consequence they know a lot about the high-end clients (to serve them better) and almost nothing about the low-end and about potential clients who prefer the simplicity and lower prices entrants can offer with their disruptive innovation (a combination of disruptive technology and a corresponding business model)
Also in explanatory research, categorization is very important, not of attributes (as in descriptive research) but of circumstances the (business) actors may find themselves. Because different circumstances give different results and outcomes. So a scientific advice should always be conditional: “if you are in the circumstances described as ……….., you should take action X to realize goal Y”.
Both in descriptive and explanatory research, science is an ongoing process with successive stages (an empirical cycle). Most of the time it starts with induction: you want to discover a pattern by observing a phenomenon. In other words: you go from specific cases to a pattern that seems general. Afterwards you ‘close the curtains’ to think about potential consequences of the pattern you think to have found, by deriving hypotheses (deduction). Then you test these hypotheses by using ‘new’ data (induction).
Even if your hypotheses are confirmed, you are not ready, Christensen and other philosophers of science say. Maybe your hypotheses will be refuted in a replication study, and by using another database. Maybe you should revise your hypotheses or even your theory, to get better results. By doing the empirical cycle again you find out.
Especially if you use advanced statistical tools to test your hypotheses, you should distinguish between descriptive statistics (like a regression line as a ‘data condenser’) and explanatory statistics, to solve the problem of potential confounders (X has an effect on Y but both are influenced by Z). Besides, most statistical tests only say something about the average of the sample, after the outliers are removed. But the outliers may be more interesting than the average, because it means that some firms (or other actors) are an exemption to the rule (anomalies). Those firms maybe in other circumstances than the average firm, and consequently need another prescription. And they are also very useful if you wants to make your theory better.
Getting a better description, explanation, prediction or prescription requires that the empirical should be repeated over and over again, for different sub-populations. It also requires a thorough assessment of the studies conducted by other scholars, asking the most important questions: “what exactly are they claiming (correlation, causation, prediction or prescription?), about which parts or aspects of the world, and what are their claims worth, especially for predicting the near future and for giving advice to specific categories of actors on how to act in specific circumstances to reach specific goals?”