As a big portion of intellectual entrepreneurship takes place in the non-profit sector, think tanks and other nongovernmental organizations face challenges similar to those of bureaucracies in measuring outcomes and defining success. Long-term profits obtained within a just rule of law is a wonderful measurement of success but not applicable to government agencies and non-profits.
State-owned companies and institutions in socialist countries used to focus on output measurements, e.g., tons of steel, pairs of shoes, or the number of students attending a university. Think tanks can fall into the same trap by focusing on the number of books distributed, reports published, conferences held, and programs managed, without making an effort to measure outcomes and results.
Governments and for-profit corporations also have units that work like think tanks. Some of the outcomes can surpass all imagination. Think of those in government and the academy who decided to connect computers to communicate, or those in internal corporate think tanks, such as the one assembled by Xerox during the ’70s, and how their thinking contributed to today’s world. My focus, however, as in other columns, will be on non-governmental, non-profit organizations and the challenge of measuring their outcomes.
A new category of think tank products, which in this age of communication and information is becoming more important, is: networking. Helping connect donors and experts with the ultimate beneficiaries is a major focus of some organizations, including the Atlas Network.
With such diversity, it is natural that organizations have different ways of measuring outcomes. They include measurements focusing on:
· Media appearances in major news outlets, TV, radio or newspapers. Think Tanks should be explicit about how they weigh each outlet. In the United States, for example, being published in The New York Times, The Washington Post, or the Wall Street Journal, carries more weight than being published in other papers.
· Estimated advertising value of media appearances. Think Tanks that have products or services that compete, or are offered through the private for-profit sector, have additional tools to measure their output. Estimated advertising value of media appearances is a good example. Publishing an article on the web or a blog controlled by the NGO, if it is not carried by other media outlets, tends to have much less value, and is much more difficult to measure than publishing an op-ed in a leading newspaper, or having several minutes in commercial TV channels to promote a special cause.
· Text or language that was present in a research document by the think tank being used in bills and laws in the legislature.
· Number of congressional testimonies.
· Improvements in major indices, such as the economic freedom indices produced by the Fraser Institute and the Heritage Foundation, the Doing Business Index prepared by the World Bank and IFC, the Global Competitiveness Index, and others. Enhancing economic liberty is the most relevant ultimate outcome for the think tanks that I cover in this column. How much of the improvement of a country or region can be attributed to work by specific think tanks is not an easy task. Even observers who do not favor free enterprise give credit to the Fraser Institute (Canada), the IEA(U.K.), and ILD (Peru), for helping change the climate of opinion which made increased respect for economic freedoms possible in their countries.
· Economic impact generated through a public policy proposal made by an institute. Examples of this are: increased transparency resulting from more simple tax structures (such as the flat tax), massive creation of capital by improving property titles (as it happened in Peru), or huge reductions in interest rates resulting from dollarization (as it happened in El Salvador).
· Increase in the number of donors and contributions and other improvements in internal processes. Think Tanks can be seen as a product in themselves. Like churches, associations, private schools, and other mediating societies, they become a relevant part of civil society, acting as a healthy buffer between the state and the individual. The continuity and stability of such independent organizations is viewed as a healthy outcome.
· Stopping a spending project in congress or a tax increase, or improving the cost-efficiency of a government. Pioneer Institute conducts a yearly“Better government competition.” Winning proposals have generated measurable outcomes, saving hundreds of millions to taxpayers.
· Generate wasteful spending by the other side. In similar fashion as the “Star wars” program during the Reagan administration made Soviet Russia overspend, some creative think tanks are arguing that their work is making the enemies of free enterprise waste precious funds.
· Number, quality, and acceptance rate of applicants to attend programs or to work for the institute. The more that the educational programs offered by think tanks mimic those offered by universities, the more they will be able to use similar outcome measurements.
· Impact in social media. Measurements of Facebook’s “talking about this,” and “likes.” “Talking about this” should be weighted more than “Likes” as the latter can be a one-time action reacting to a campaign. The number of Twitter followers is also an outcome more than an output. With increased use of paid promotion through Facebook and Twitter advertisements, some of these measures might lose value as outcomes. Web traffic statistics continue to be tricky as a reliable measure of outcome. Most think tanks do not report web traffic in a consistent manner. There is a temptation to manipulate figures to appear good to donors who, in general, are not web-savvy. The number of unique visitors is one of the most reliable measurements, but independent sites focusing on estimating web traffic, such as Alexa.com are not accurate. A combination of measurements of traffic, Facebook and Twitter presence would still be valuable and most think tanks are feeling obliged to report about them.
· YouTube views and other certifiable measurements. Think tanks are competing with other educational NGOs, such as the Institute for Humane Studies, (I.H.S.) or the Khancademy in this segment. I.H.S.www.:LearnLiberty.org series of short educational videos, already viewed more than 12 million times. Khan is slowly releasing classes on topics which think tank also address, from economics, to American civics and political history.
· Number of think tank publications adopted in university and college courses. This was one of the typical outcomes measured during the time when think tanks were more focused on research or dissemination of solid research than advocacy.
· Scholarly citations of papers and books written by think tank researchers.
It is time to try to reach other ambitious goals at think tanks. Cato Institute has an outstanding Center for Constitutional Studies. One of its outcomes has been to help redirect the debate on limits of governmental action. Cato’s president, John Allison, has been explicit about aiming even higher, taking the center to a point “when the professors at the Harvard Law School find it necessary to respond to the arguments of Cato scholars and when the Supreme Court Justices feel consistently obligated to consider the Cato perspective in reaching their judicial decisions.”
Achieving freer societies is not the task of think tanks alone. Media companies, universities, political parties and leadership, churches, and other actors also play relevant roles in the battle for ideas and their implementation. But as recent books such as “Masters of the Universe” and“Think Tanks in America” have shown, think tanks are a formidable force. An improved focus on outcomes rather than output will make them even more effective.
*Gonzalo Schwarz and Harry Kaldsted, of the Atlas Network, also conducted research for this piece.
* Alejandro Antonio (Alex) Chafuen, Ph.D., has been president of Atlas Economic Research Foundation since 1991. A member of the board of advisors to The Center for Vision & Values and a trustee of Grove City College, he is also the president and founder of the Hispanic American Center of Economic Research. Dr. Chafuen serves on several boards including the Chase Foundation of Virginia, the Acton Institute, the Fraser Institute (Canada), and is an Active Honorary Member of the John Templeton Foundation.