Welcome to Dr Lincoln's blog

Welcome for visiting my blog. Hope you enjoy the visit and always welcome back again. Have a nice day!

2009-05-08

A small contribution to the practical application of econometrics

My colleagues and I encountered an interesting econometric problem in an exercise in using the results from econometrical estimation during a project. For that project, we had to estimate a location effect for each of the eight regions and the results were used to determine the allocation of funds among eight organisations located in eight different regions. Due to external factors to those organisations beyond their individual control, their operations require different funding to take account the impact of their locations on their operation. So it was not just an academic exercise of econometrics, but also had a real impact on the allocation in the real world.

As with any econometric estimation, one may get the resultant estimates of which some are statistically significant and others insignificant. This was exactly the case with our project. We used dummy variables to capture the location effects for the eight different organisations. Organisation 8 was used as the reference or default, so its location effect was 0, this is because there can only normally be dummies for n-1 of the n organisations. It was deemed to be highly significant at the value of 0. Table 1 shows the results from the econometric estimation:

Table 1 Estimation results



We had further transformations done on these results to translate these estimates to the location factors for each organisation to use in the final allocation of funding. However, they are not essential to the focus of this article here and I will not discuss them and only focus on how we used these results as the inputs into those transformations.

Because there would be a real impact on the allocation of funding, we were faced with the question: should we use the estimated values for these organisation irrespective their statistical significance, or 0 value for those organisations whose estimates are statistically insignificant, or any other value and what they could be? What values to use would affect the funding, or the shares of the total of those eight organisations. We had to make a choice with consequential effects on the allocation outcomes.

There were two different arguments, one for using the actual estimated values and the other for using 0 value for those organisations concerned. The former argued that they were the actual estimates that produced best account of location effects, while the latter said they were statistically insignificant and that meant they were not statistically different from 0 value.

Both arguments had some rationale, depending how one looks at the issue. However, they are two extreme cases/values of possibly many values to choose, because one could argue that one could use a combination of the two, that is, any value between the two. The third approach could avoid the extreme values of using either the actual estimates or 0. But the central question remained: what value to use that would produce the most desirable or best outcome of the fund allocation?

We had pondered that question for a very long time. Finally, we came to the conclusion that the use of any combination of the two would be arbitrary, except perhaps one set of values. This set of values was informed by the data we used. They were determined by using the information on the level of statistical significance (p values) for the estimates. We used that information as the basis to discount the actual estimates and use the discounted estimates as the final values as the inputs in the final transformation processes for deriving the location factor to be used in the funding allocation.

More specifically, we combined the actual estimated values and 0 using the p values as the weights. For example, for organisation 6, we multiplied the actual estimate, 0.032, by 0.552 - the difference between 1 and its p-value of 0.448. This gave us a value of 0.017664. That was equivalent to the result of this following calculation: 0.448 * 0 + (1 – 0.448) * 0.032 = 0.017664, where the weights were 0.448 and 1 - 0.448 = 0.552.

We considered this as the most desirable choice for organisation 6. It was not arbitrarily determine by anyone’s arbitrary intervention, but based on information derived from the data and the estimation. It was a better result than either of the two extreme values. It was defensible statistically.

This was because even the conventional critical values in determining the statistical significance of the estimates were somehow arbitrary. For example, if one takes the critical value of significance level was 2 from the t-statistic, then someone could ask, what about any values that were smaller than but infinitely close to 2? Why should those infinitely small differences should make very large differences in the implied parameters to use, that is, between zero and the actual estimates?

Probably no one can answer that question satisfactorily, except to say that the choice of the critical values for judging statistical significance is largely a matter of convenience. In other words, people have to use a value and a infinite range of values could be used, but one has to make a choice, so has to pick a convenient number that can make the trick.

To conclude, although we were forced to make a choice for the values to use and had been agonised for a very long time, we finally did made an interesting discovery and came up with quite satisfactory results. Our approach may have implications for the practical use of econometric estimates in many situations. We think it is a statistically sound approach. It is simple but significant for practical application of econometrics.

No comments:

Post a Comment