Valuing publicly sponsored research projects: Risks, scenario adjustments, and inattention

Survey-based choice scenarios used to value non-market public goods typically preclude any risk that the benefits described may not be delivered. Our survey specifies explicit risks of (a) outright program failure and (b) program redundancy due to possible private sector substitutes. Additionally, most analyses assume that survey subjects fully accept these scenarios and that all provided information receives their complete attention. Our discounted expected utility model of choice accommodates both these objective risks and the possibility of subjective scenario adjustment or selective inattention by respondents. We then counterfactually simulate willingness-to-pay in the absence of these distortions.

1. Risks of failure and substitution are not typically included as attributes of the good under study. Additional uncertainty concerns how severe changes in climate may turn out to be. This point is addressed below.

2. Krupnick and Adamowicz (2007) note that a “surprisingly large number of [stated choice] surveys do not use debriefing questions” (p. 53). They advise against the use of debriefing questions which force respondents to answer either yes or no, and suggest that it is preferable to offer a range of responses. This information can then be “used in validity regressions.” However, they suggest that such variables “are best left out of the final regression, however, because of endogeneity concerns. If such responses are not too numerous they can be dropped” (p. 54).

3. We note that while we have information about the amount of time each respondent spent on this one page of our internet survey, respondents may have different cognitive abilities, so that the same amount of “attention” could map into different amounts of survey time for different respondents.

4. The data used here were collected in the Fall of 2001 and consist of 14,071 choices by 2,003 respondents. The survey is still accessible for use as a pedagogical tool (but does not record choices).

5. We vary the manner in which the question is posed to the respondent because stated choice experiments are often subject to the criticism that the choices elicited from respondents are merely an artifact of the format and framing of questions. By randomizing the format and framing of choices in our survey, we are able to assess the sensitivity of our results to these different scenario design decisions. In many earlier surveys, just one design would be selected, which precludes this type of sensitivity analysis.

6. If we were to repeat this survey experiment, we would certainly expand this choice scenario to be explicit about the provision mechanism. Given the data that we have to work with, we can only assume that respondents made their choices with an understanding that the government-sponsored research program would be put into action if aggregate “check-off” amounts reached some goal and that if the total was not sufficient, there would be no program (and the “checked-off” amount would be rebated as a tax credit).

7. Amato et al. (2005), Hor et al. (2005), Mansur et al. (2005), and Sailor (2001) examine air conditioner technologies generally. Curriero et al. (2002), Davis et al. (2002), Davis et al. (2003), McGeehin and Mirabelli (2001), and Yoganathan and Rom (2001) focus on the role air conditioners play in mitigating the effects of extreme weather events.

8. Bonte (2004), Griliches (1992), Jaffe (2002), Jones and Williams (1998), Mamuneas (1999), and Popp (2006) examine the spillovers associated with research and development projects. Griffith et al. (2001) consider tax credits to firms to promote R&D, rather than withholding tax credits to households in order to sponsor R&D.

9. See for example Clark et al. (2003), Goeree et al. (2002), and Nunes and Schokkaert (2003).

10. The online survey was programmed in Perl and made liberal use of the FastTemplate utility. Table O-1, in the online Appendix O, contains detailed summary statistics for these randomized choice framing differences. Appendix O can be found at http://www.uoregon.edu/~cameron/JRU/BCG_Appendix.pdf

11. Subjective risks have been considered in other contexts (e.g. DeShazo and Cameron 2005, and Viscusi and Huber 2006). Unfortunately, we are not able to conduct this interesting comparison with these data.

12. This was a highly complex survey and we are grateful for the huge amount of time, individually and in the aggregate, that participating individuals contributed to this research program.

13. For this formulation we do not consider more flexible discounting formulae because benefits will accrue, at the earliest, in 5 years. See Frederick, Loewenstein and O’Donoghue (2002) and Laibson (1997).

14. It is possible that the effort dedicated to similar research in the private sector may depend on whether a government-sponsored research program is under way. We abstract from any such dependence in this analysis. It proved too difficult to attempt to explain more-complex probabilities to survey respondents.

15. There is also some chance that respondents may overreact by assuming that the probability of failure is greater than advertised. Our estimating specification does not preclude this possible outcome.

16. This product cannot, technically, be interpreted as a subjective probability. If this was our aim, then these “subjectively adjusted” probabilities would have to be constrained as valid probabilities (i.e. lie between zero and one). However, we do not put such constraints on these parameters theoretically or during estimation. To attempt a formal Bayesian updating model, of course, we would need additional information about respondents’ prior probabilities.

17. The government program can thus be viewed as a measure designed to increase the probability of a desirable outcome, namely, the technology being developed one way or the other.

18. We have also explored specifications in which utility parameters vary systematically with observable respondent characteristics with no change in the qualitative results that are the focus of this paper.

19. Note that these designations are completely arbitrary and have no effect on our results.

20. Other researchers have called these types of data a “multiple-bounded” elicitation format (e.g. see Welsh and Poe 1998). Different interpretations of these responses are possible, but we use the ordered-logit interpretation employed in Cameron et al. (2002).

21. Since standard packaged algorithms are not available for models that aggregate ordered data across sub-samples with differing numbers of answer categories, this likelihood function must be programmed in MATLAB (or some other generic function-optimizing software).

22. In future research, we may consider in more depth the effects of allowing the randomized differences in survey formats also to shift each of the three basic utility parameters, not just the error dispersion parameter (which is the model we report here). If we simultaneously generalize all the utility and dispersion parameters to allow for these format effects, many individually statistically significant effects are obscured by this over-parameterization. It is clear that this heterogeneity matters, but we have not yet undertaken rigorous non-nested tests of just where it is best to accommodate it. The answer may differ across applications, so the question may be complex enough to warrant an entirely separate analysis.

23. For example, individuals may be paternalistically altruistic or anticipate spillovers from the new technologies.

24. Table O-1 in the online Appendix O details the distributions for the independent dynamically generated choice scenario attributes. For simplicity, we round these approximate means.

25. This valuable connection was suggested by a referee. Also, in the CV literature, Carson et al. (1997) characterized ancillary conditions as “circumstances of choice.”

The data for this study were collected with funding from the National Science Foundation (SES-9818875). The research was supported in part by a Kleinsorge Summer Fellowship and by the endowment of the R.F. Mikesell Chair in Environmental and Resource Economics at the University of Oregon. We are grateful to Vilija Gulbinas for assistance with survey development and implementation at UCLA, to Paul Jakus and participants at the 2006 W1133 and OREE meetings, Nino Sitchinava, and our referees for helpful comments, and to Ian McConnaha and Tatiana Raterman for research assistance at the University of Oregon. We are also grateful for the very generous cooperation of 114 instructors at 92 different colleges and universities in the US and Canada who announced our survey to their classes and encouraged participation. The opinions expressed in this paper are those of the authors and do not necessarily reflect the opinions of the Federal Reserve Board of Governors or its staff. Any remaining errors are our own.

Authors and Affiliations

1. Department of Economics, University of Oregon Eugene, 435 PLC 1285, Eugene, OR, 97403-1285, USA

   Daniel R. Burghart & Trudy Ann Cameron

2. Federal Reserve Board of Governors, Washington, DC, USA

   Geoffrey R. Gerdes

Corresponding author

Correspondence to Trudy Ann Cameron.

Appendix

1.1 Log-likelihood function

Each different answer format offered on the survey implies a different set of ordered-logit probability formulas. These probabilities, by number of answer levels, are as follows:

2-level: \( \begin{array}{*{20}l} {{{\text{P2}}Y_{i} = \frac{1} {{1 + \exp {\left[ {{{\left( {\alpha _{{20}} - \Delta V_{i} } \right)}} \mathord{\left/ {\vphantom {{{\left( {\alpha _{{20}} - \Delta V_{i} } \right)}} {\exp {\left( {\kappa _{i} } \right)}}}} \right. \kern-\nulldelimiterspace} {\exp {\left( {\kappa _{i} } \right)}}} \right]}}}\quad {\text{YES}}} \hfill} \\ {{{\text{P2}}N_{i} = \frac{{\exp {\left[ {{{\left( {\alpha _{{20}} - \Delta V} \right)}_{i} } \mathord{\left/ {\vphantom {{{\left( {\alpha _{{20}} - \Delta V} \right)}_{i} } {\exp {\left( {\kappa _{i} } \right)}}}} \right. \kern-\nulldelimiterspace} {\exp {\left( {\kappa _{i} } \right)}}} \right]}}} {{1 + \exp {\left[ {{{\left( {\alpha _{{20}} - \Delta V_{i} } \right)}} \mathord{\left/ {\vphantom {{{\left( {\alpha _{{20}} - \Delta V_{i} } \right)}} {\exp {\left( {\kappa _{i} } \right)}}}} \right. \kern-\nulldelimiterspace} {\exp {\left( {\kappa _{i} } \right)}}} \right]}}}\quad {\text{NO}}} \hfill} \\ \end{array} \)

3-level: \( \begin{array}{*{20}c} {{\text{P3}}Y_{i} = \frac{1} {{1 + \exp {\left[ {{{\left( {\alpha _{{31}} - \Delta V_{i} } \right)}} \mathord{\left/ {\vphantom {{{\left( {\alpha _{{31}} - \Delta V_{i} } \right)}} {\exp {\left( {\kappa _{i} } \right)}}}} \right. \kern-\nulldelimiterspace} {\exp {\left( {\kappa _{i} } \right)}}} \right]}}}}{YES} \\ {{\text{P3}}M_{i} = {\left( {\frac{{\exp {\left[ {{{\left( {\alpha _{{31}} - \Delta V_{i} } \right)}} \mathord{\left/ {\vphantom {{{\left( {\alpha _{{31}} - \Delta V_{i} } \right)}} {\exp {\left( {\kappa _{i} } \right)}}}} \right. \kern-\nulldelimiterspace} {\exp {\left( {\kappa _{i} } \right)}}} \right]}}} {{1 + \exp {\left[ {{{\left( {\alpha _{{31}} - \Delta V_{i} } \right)}} \mathord{\left/ {\vphantom {{{\left( {\alpha _{{31}} - \Delta V_{i} } \right)}} {\exp {\left( {\kappa _{i} } \right)}}}} \right. \kern-\nulldelimiterspace} {\exp {\left( {\kappa _{i} } \right)}}} \right]}}}} \right)} - {\left( {\frac{{\exp {\left[ {{{\left( {\alpha _{{30}} - \Delta V_{i} } \right)}} \mathord{\left/ {\vphantom {{{\left( {\alpha _{{30}} - \Delta V_{i} } \right)}} {\exp {\left( {\kappa _{i} } \right)}}}} \right. \kern-\nulldelimiterspace} {\exp {\left( {\kappa _{i} } \right)}}} \right]}}} {{1 + \exp {\left[ {{{\left( {\alpha _{{30}} - \Delta V_{i} } \right)}} \mathord{\left/ {\vphantom {{{\left( {\alpha _{{30}} - \Delta V_{i} } \right)}} {\exp {\left( {\kappa _{i} } \right)}}}} \right. \kern-\nulldelimiterspace} {\exp {\left( {\kappa _{i} } \right)}}} \right]}}}} \right)}}{NOT\;SURE} \\ {{\text{P3}}N_{i} = \frac{{\exp {\left[ {{{\left( {\alpha _{{30}} - \Delta V_{i} } \right)}} \mathord{\left/ {\vphantom {{{\left( {\alpha _{{30}} - \Delta V_{i} } \right)}} {\exp {\left( {\kappa _{i} } \right)}}}} \right. \kern-\nulldelimiterspace} {\exp {\left( {\kappa _{i} } \right)}}} \right]}}} {{1 + \exp {\left[ {{{\left( {\alpha _{{30}} - \Delta V_{i} } \right)}} \mathord{\left/ {\vphantom {{{\left( {\alpha _{{30}} - \Delta V_{i} } \right)}} {\exp {\left( {\kappa _{i} } \right)}}}} \right. \kern-\nulldelimiterspace} {\exp {\left( {\kappa _{i} } \right)}}} \right]}}}}{NO} \\ \end{array} \)

4-level: \( \begin{array}{*{20}l} {{{\text{P}}4{\text{D}}Y_{i} = \frac{1} {{1 + \exp {\left[ {{{\left( {\alpha _{{42}} - \Delta V_{i} } \right)}} \mathord{\left/ {\vphantom {{{\left( {\alpha _{{42}} - \Delta V_{i} } \right)}} {\exp {\left( {\kappa _{i} } \right)}}}} \right. \kern-\nulldelimiterspace} {\exp {\left( {\kappa _{i} } \right)}}} \right]}}}\quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad {\text{Def}}{\text{.}}\;{\text{YES}}} \hfill} \\ {{{\text{P}}4{\text{P}}Y_{i} = {\left( {\frac{{\exp {\left[ {{{\left( {\alpha _{{42}} - \Delta V_{i} } \right)}} \mathord{\left/ {\vphantom {{{\left( {\alpha _{{42}} - \Delta V_{i} } \right)}} {\exp {\left( {\kappa _{i} } \right)}}}} \right. \kern-\nulldelimiterspace} {\exp {\left( {\kappa _{i} } \right)}}} \right]}}} {{1 + \exp {\left[ {{{\left( {\alpha _{{42}} - \Delta V_{i} } \right)}} \mathord{\left/ {\vphantom {{{\left( {\alpha _{{42}} - \Delta V_{i} } \right)}} {\exp {\left( {\kappa _{i} } \right)}}}} \right. \kern-\nulldelimiterspace} {\exp {\left( {\kappa _{i} } \right)}}} \right]}}}} \right)} - {\left( {\frac{{\exp {\left[ {{{\left( {\alpha _{{41}} - \Delta V_{i} } \right)}} \mathord{\left/ {\vphantom {{{\left( {\alpha _{{41}} - \Delta V_{i} } \right)}} {\exp {\left( {\kappa _{i} } \right)}}}} \right. \kern-\nulldelimiterspace} {\exp {\left( {\kappa _{i} } \right)}}} \right]}}} {{1 + \exp {\left[ {{{\left( {\alpha _{{41}} - \Delta V_{i} } \right)}} \mathord{\left/ {\vphantom {{{\left( {\alpha _{{41}} - \Delta V_{i} } \right)}} {\exp {\left( {\kappa _{i} } \right)}}}} \right. \kern-\nulldelimiterspace} {\exp {\left( {\kappa _{i} } \right)}}} \right]}}}} \right)}\quad \quad \Pr {\text{ob}}{\text{. YES}}} \hfill} \\ {{{\text{P}}4{\text{P}}N_{i} = {\left( {\frac{{\exp {\left[ {{{\left( {\alpha _{{41}} - \Delta V_{i} } \right)}} \mathord{\left/ {\vphantom {{{\left( {\alpha _{{41}} - \Delta V_{i} } \right)}} {\exp {\left( {\kappa _{i} } \right)}}}} \right. \kern-\nulldelimiterspace} {\exp {\left( {\kappa _{i} } \right)}}} \right]}}} {{1 + \exp {\left[ {{{\left( {\alpha _{{41}} - \Delta V_{i} } \right)}} \mathord{\left/ {\vphantom {{{\left( {\alpha _{{41}} - \Delta V_{i} } \right)}} {\exp {\left( {\kappa _{i} } \right)}}}} \right. \kern-\nulldelimiterspace} {\exp {\left( {\kappa _{i} } \right)}}} \right]}}}} \right)} - {\left( {\frac{{\exp {\left[ {{{\left( {\alpha _{{40}} - \Delta V_{i} } \right)}} \mathord{\left/ {\vphantom {{{\left( {\alpha _{{40}} - \Delta V_{i} } \right)}} {\exp {\left( {\kappa _{i} } \right)}}}} \right. \kern-\nulldelimiterspace} {\exp {\left( {\kappa _{i} } \right)}}} \right]}}} {{1 + \exp {\left[ {{{\left( {\alpha _{{40}} - \Delta V_{i} } \right)}} \mathord{\left/ {\vphantom {{{\left( {\alpha _{{40}} - \Delta V_{i} } \right)}} {\exp {\left( {\kappa _{i} } \right)}}}} \right. \kern-\nulldelimiterspace} {\exp {\left( {\kappa _{i} } \right)}}} \right]}}}} \right)}\quad \quad \Pr {\text{ob}}{\text{.}}\;{\text{NO}}} \hfill} \\ {{P4DN = \frac{{\exp {\left[ {{{\left( {\alpha _{{40}} - \Delta V_{i} } \right)}} \mathord{\left/ {\vphantom {{{\left( {\alpha _{{40}} - \Delta V_{i} } \right)}} {\exp {\left( {\kappa _{i} } \right)}}}} \right. \kern-\nulldelimiterspace} {\exp {\left( {\kappa _{i} } \right)}}} \right]}}} {{1 + \exp {\left[ {{{\left( {\alpha _{{40}} - \Delta V_{i} } \right)}} \mathord{\left/ {\vphantom {{{\left( {\alpha _{{40}} - \Delta V_{i} } \right)}} {\exp {\left( {\kappa _{i} } \right)}}}} \right. \kern-\nulldelimiterspace} {\exp {\left( {\kappa _{i} } \right)}}} \right]}}}\quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad {\text{Def}}{\text{.}}\;{\text{NO}}} \hfill} \\ \end{array} \)

5-level: \( \begin{array}{*{20}l} {{{\text{P5D}}Y_{i} = \frac{1} {{1 + \exp {\left[ {{{\left( {\alpha _{{53}} - \Delta V_{1} } \right)}} \mathord{\left/ {\vphantom {{{\left( {\alpha _{{53}} - \Delta V_{1} } \right)}} {\exp {\left( {\kappa _{i} } \right)}}}} \right. \kern-\nulldelimiterspace} {\exp {\left( {\kappa _{i} } \right)}}} \right]}}}} \hfill} & {{{\text{Def}}{\text{.}}\;{\text{YES}}} \hfill} \\ {{{\text{P5P}}Y_{i} = {\left( {\frac{{\exp {\left[ {{{\left( {\alpha _{{53}} - \Delta V_{i} } \right)}} \mathord{\left/ {\vphantom {{{\left( {\alpha _{{53}} - \Delta V_{i} } \right)}} {\exp {\left( {\kappa _{i} } \right)}}}} \right. \kern-\nulldelimiterspace} {\exp {\left( {\kappa _{i} } \right)}}} \right]}}} {{1 + \exp {\left[ {{{\left( {\alpha _{{53}} - \Delta V_{i} } \right)}} \mathord{\left/ {\vphantom {{{\left( {\alpha _{{53}} - \Delta V_{i} } \right)}} {\exp {\left( {\kappa _{i} } \right)}}}} \right. \kern-\nulldelimiterspace} {\exp {\left( {\kappa _{i} } \right)}}} \right]}}}} \right)} - {\left( {\frac{{\exp {\left[ {{{\left( {\alpha _{{52}} - \Delta V_{i} } \right)}} \mathord{\left/ {\vphantom {{{\left( {\alpha _{{52}} - \Delta V_{i} } \right)}} {\exp {\left( {\kappa _{i} } \right)}}}} \right. \kern-\nulldelimiterspace} {\exp {\left( {\kappa _{i} } \right)}}} \right]}}} {{1 + \exp {\left[ {{{\left( {\alpha _{{52}} - \Delta V_{i} } \right)}} \mathord{\left/ {\vphantom {{{\left( {\alpha _{{52}} - \Delta V_{i} } \right)}} {\exp {\left( {\kappa _{i} } \right)}}}} \right. \kern-\nulldelimiterspace} {\exp {\left( {\kappa _{i} } \right)}}} \right]}}}} \right)}} \hfill} & {{\Pr {\text{ob}}{\text{.}}\;{\text{YES}}} \hfill} \\ {{{\text{P}}5M_{i} = {\left( {\frac{{\exp {\left[ {{{\left( {\alpha _{{52}} - \Delta V_{i} } \right)}} \mathord{\left/ {\vphantom {{{\left( {\alpha _{{52}} - \Delta V_{i} } \right)}} {\exp {\left( {\kappa _{i} } \right)}}}} \right. \kern-\nulldelimiterspace} {\exp {\left( {\kappa _{i} } \right)}}} \right]}}} {{1 + \exp {\left[ {{{\left( {\alpha _{{52}} - \Delta V_{i} } \right)}} \mathord{\left/ {\vphantom {{{\left( {\alpha _{{52}} - \Delta V_{i} } \right)}} {\exp {\left( {\kappa _{i} } \right)}}}} \right. \kern-\nulldelimiterspace} {\exp {\left( {\kappa _{i} } \right)}}} \right]}}}} \right)} - {\left( {\frac{{\exp {\left[ {{{\left( {\alpha _{{51}} - \Delta V_{i} } \right)}} \mathord{\left/ {\vphantom {{{\left( {\alpha _{{51}} - \Delta V_{i} } \right)}} {\exp {\left( {\kappa _{i} } \right)}}}} \right. \kern-\nulldelimiterspace} {\exp {\left( {\kappa _{i} } \right)}}} \right]}}} {{1 + \exp {\left[ {{{\left( {\alpha _{{51}} - \Delta V_{i} } \right)}} \mathord{\left/ {\vphantom {{{\left( {\alpha _{{51}} - \Delta V_{i} } \right)}} {\exp {\left( {\kappa _{i} } \right)}}}} \right. \kern-\nulldelimiterspace} {\exp {\left( {\kappa _{i} } \right)}}} \right]}}}} \right)}} \hfill} & {{{\text{NOT}}\;{\text{SURE}}} \hfill} \\ {{P5PN_{i} = {\left( {\frac{{\exp {\left[ {{{\left( {\alpha _{{51}} - \Delta V_{i} } \right)}} \mathord{\left/ {\vphantom {{{\left( {\alpha _{{51}} - \Delta V_{i} } \right)}} {\exp {\left( {\kappa _{i} } \right)}}}} \right. \kern-\nulldelimiterspace} {\exp {\left( {\kappa _{i} } \right)}}} \right]}}} {{1 + \exp {\left[ {{{\left( {\alpha _{{51}} - \Delta V_{i} } \right)}} \mathord{\left/ {\vphantom {{{\left( {\alpha _{{51}} - \Delta V_{i} } \right)}} {\exp {\left( {\kappa _{i} } \right)}}}} \right. \kern-\nulldelimiterspace} {\exp {\left( {\kappa _{i} } \right)}}} \right]}}}} \right)} - {\left( {\frac{{\exp {\left[ {{{\left( {\alpha _{{50}} - \Delta V_{i} } \right)}} \mathord{\left/ {\vphantom {{{\left( {\alpha _{{50}} - \Delta V_{i} } \right)}} {\exp {\left( {\kappa _{i} } \right)}}}} \right. \kern-\nulldelimiterspace} {\exp {\left( {\kappa _{i} } \right)}}} \right]}}} {{1 + \exp {\left[ {{{\left( {\alpha _{{50}} - \Delta V_{i} } \right)}} \mathord{\left/ {\vphantom {{{\left( {\alpha _{{50}} - \Delta V_{i} } \right)}} {\exp {\left( {\kappa _{i} } \right)}}}} \right. \kern-\nulldelimiterspace} {\exp {\left( {\kappa _{i} } \right)}}} \right]}}}} \right)}} \hfill} & {{{\text{Prob}}{\text{.}}\;{\text{NO}}} \hfill} \\ {{P5DN_{i} = {\left( {\frac{{\exp {\left[ {{{\left( {\alpha _{{50}} - \Delta V_{i} } \right)}} \mathord{\left/ {\vphantom {{{\left( {\alpha _{{50}} - \Delta V_{i} } \right)}} {\exp {\left( {\kappa _{i} } \right)}}}} \right. \kern-\nulldelimiterspace} {\exp {\left( {\kappa _{i} } \right)}}} \right]}}} {{1 + \exp {\left[ {{{\left( {\alpha _{{50}} - \Delta V_{i} } \right)}} \mathord{\left/ {\vphantom {{{\left( {\alpha _{{50}} - \Delta V_{i} } \right)}} {\exp {\left( {\kappa _{i} } \right)}}}} \right. \kern-\nulldelimiterspace} {\exp {\left( {\kappa _{i} } \right)}}} \right]}}}} \right)}} \hfill} & {{{\text{Def}}{\text{.}}\;{\text{NO}}} \hfill} \\ \end{array} \)

A little extra intuition is involved when pooling data from two-, three-, four-, and five-alternative simple or ordered logit specification. In a two-outcome model, it is natural to normalize the threshold level of the latent variable, α ₂₀ above, to zero. When pooling all four types of data, it makes sense to normalize the distinction between a “yes” answer and a “no” answer to zero. For the three- and five-outcome cases, however, there is no bright line between “yes” and “no,” due to the presence of the “not sure” option. Thus we normalize α ₂₀ and α ₄₁ to zero, but allow all of the other thresholds (α ₃₀, α ₃₁, α ₄₀, α ₄₂, and α ₅₀, α ₅₁, α ₅₂, and α ₅₃) to take on whatever values the data seem to dictate, with the expectation that α ₃₀, α ₄₀, α ₅₀, and α ₅₁ should be negative and the others should be positive.

1.2 Log-likelihood function

Denote the number of observations in each sub-sample with 2, 3, 4, and 5 answer options as N2, N3, N4 and N5. The formula for the log-likelihood function, as always, involves discrete indicators for each answer option. All of the following binary indicators are “zero otherwise”:

• C2Y _i , C3Y _i = 1 if respondent chooses “yes” in 2- or 3- alternative cases

• C2N _i , C3N _i =1 if respondent chooses “no” in 2- or 3-alternative cases

• C3M _i , C5M _i =1 if respondent chooses “not sure” in 3- or 5-alternative cases

• C4DY _i , C5DY _i =1 if respondent chooses “definitely yes” in 4- or 5-alternative cases

• C4PY _i , C5PY _i =1 if respondent chooses “probably yes” in 4- or 5-alternative cases

• C4PN _i , C5PN _i =1 if respondent chooses “probably no” in 4- or 5-alternative cases

• C4DN _i , C5DN _i =1 if respondent chooses “definitely no” in 4- or 5-alternative cases

The log-likelihood function for each model considered in this paper will be determined by the format of the relevant systematic utility-difference function, ΔV _i; it is simply the sum of the relevant components that apply for each type of response format. The preference parameters to be estimated are embodied in this utility difference. The ordered logit threshold parameters, α _mn and the differentials in the error dispersion, relative to the base case, are captured by the systematically varying parameter κ _i:

Log L

1.3 Optimization method

The unknown preference parameters (either as scalars, or as systematically varying parameters) are estimated by conventional maximum likelihood methods. Since the problem at hand is non-standard, however, the parameters cannot be estimated using packaged econometric software. We maximize the log-likelihood using MATLAB 7.1. We employ numeric derivatives and the BFGS algorithm, but recalculate the full Hessian matrix at the optimum using Greene’s formulas. Asymptotic t-test statistics are based on a symmetrized version of the Hessian, where the off-diagonal elements of the numeric Hessian, if different, are averaged.

Reprints and permissions