About the Database

This portal represents a systematic effort to collect English, peer-reviewed, social science journal articles about NGOs in development. At present we have collected articles published 1980-2014.

For the purposes of this research, we define ‘‘NGO” broadly as any nonprofit, non-governmental organization that works in the development, humanitarian, advocacy, or civil society sector in any country that was not an OECD member during the entire period of study (1980–2014).

To identify the relevant literature for analysis, we first created a list of keywords that would allow us to locate pertinent articles. These keywords relate to organizational terms (e.g., non-governmental organization, community-based organization) and to international development (e.g., names of developing countries, foreign aid terms). We used this list to search the Academic Search Premier Database of EBSCO, which produced a group of 11,373 articles whose title, abstract or keywords matched at least one organizational and one international development keyword. Next, to eliminate false positives, two research assistants read the title, abstract and keywords of each article to ensure that it was a research article in which NGOs were an object of analysis. In the case of disagreement or uncertainty among the research assistants, one of the authors decided. This process produced a corpus of 3402 articles; we were able to locate 3366 articles with PDFs suitable for digital analysis.

Access and download the data files

How Were the Keyword Counts Developed?

The keyword counts reflect the number of occurrences of each keyword in each article, excluding the reference section of the article. We used PowerGrep to analyze text files of each article, which we converted from PDFs.

The counts of keywords reflect merging of multiple spellings of some terms. So, “nongovernmental organization” includes “NGO” as well as “non-governmental organization.” Disaggregated counts of terms are available in the original data files.

How Was the Topic Model Developed?

We used the Structural Topic Modeling (STM) package in R to identify topics. The topic modeling algorithm works by using the observed frequency of the distribution of terms in the document to predict the distribution of terms in a topic. Structural topic modeling incorporates information about the document (metadata) into the distributions of terms; in other words, the algorithm ‘uses’ information about each document to assign words into topics.  We used publication year as a topic prevalence covariate.

The topic model we used draws on the academic corpus of 3,359 articles published between 1980 and 2014 for which we were able to locate an electronic version suitable for analysis. We selected a model with 45 topics, using the ‘selectModel’ command, which yielded 10 candidate models. We chose among the models by narrowing to the three models with the highest average levels of coherence and exclusivity. We then reviewed the topic term lists for the top three models to determine which topics had the greatest logical coherence.   Next, we validated the topics predictively, ensuring that the rise and fall of topics over time was consistent with what we would predict from historical events. For example, we found that the topic that included the terms “hiv,” “aids,” “hivaids,” “sex,” and “prevention” rose rapidly in the 1990s, which parallels the increase in HIV prevalence and funding for HIV research and programming. Finally, we validated the topics semantically by reviewing articles with high, medium, and low prevalence of various topics, to ensure that a reader could discern that the article was very much, somewhat, or hardly at all ”about” that topic.

We gratefully acknowledge the assistance of Brandon Stewart, co-author of the STM package, for help with its implementation. Further information is available upon request from the NKC PIs.

How Did We Identify These Articles?

We used a two-phase process to collect and screen journal articles. In the first phase, an automated search used a Boolean phrase query to retrieve citations from EBSCO Academic Search Premier (ASP), a database with access to a broad universe of academic literature in the social sciences. During a second phase, research assistants evaluated each citation’s title, abstract, and keywords to remove type I errors/false positives.

First Phase

The automated data collection process retrieved citations of publications potentially eligible for inclusion into the corpus.

Eligibility Criteria

The eligibility criteria for the automated search included five limiters:

  1. Records from scholarly peer-reviewed journals;
  2. Academic articles and not book reviews, research notes, editor comments, etc.;
  3. Published during the 35-year period January 1st, 1980 thru December 31st, 2014;
  4. Written in English;
  5. With title, abstract, or keyword content that matched search criteria related to keywords describing organizational and development terms

Automated Data Collection Process

We used two queries to search articles’ title, abstract, and keywords. We then exported the citation records to a central EndNote repository. We then manually verified the inclusion of records.

Primary Query

The primary query included approximately fifteen organizational terms—e.g., “non-governmental development organization,” “nonprofit organization,” “faith-based organization,” “third sector”—loosely synonymous with the NGO organizational form. Spelling variants and pluralized terms—”organization/organisation,” “association/associations,” “third sector/thirdsector”—and acronyms—”NGO,” “NPO,” “FBO”—expanded this list to approximately 100 terms. In order to pass the primary query, a record had to contain at least one organizational term in its title, abstract, or keywords.

Secondary Query

The secondary query combined terms to capture articles describing developing countries or development topics. In order to pass the secondary query, a record had to contain at least one location or development term in its title, abstract, or keywords.

The list of locations included over 200 variations on country names—”Timor-Leste,” “East Timor,” “Democratic Republic of the Congo,” “DRC”—and geographic areas—”global south,” “sub-Saharan Africa.” The only countries intentionally excluded were those that had OECD membership before 1980.

The list of development terms included more than 75 terms—”developing countries,” “international development,” “Millennium Development Goals”.

Database Selection

EBSCO’s Academic Search Premier is one of the most comprehensive storehouses of scientific research in the world. At the time of data collection for this systematic review, Academic Search Premier indexed and abstracted 12,201 peer-reviewed journals and curated full-text research from 4,772 academic journals. This fee-based subscription service is accessible through most academic institutions.

Search Functions

The EBSCO query tool is exceptionally robust. It allows users to query nearly twenty fields with permutations of various terms. This review targeted the title, abstract, and keyword fields. For advanced searches, fields can incorporate Boolean phrase queries that use field identifiers and logical operators to create search combinations that return precise query results. The primary query’s Boolean phrase instructed EBSCO to return results if and only if at least one of the 100 organizational terms appeared in the title, abstract, or keywords of an article. The Boolean phrase for the primary query is a lengthy, but logically simple, 1,136 words.

For even more advanced searches, the query tool allows Boolean phrases to nest within each other using the logical operators “and,” “or,” and “not.” This systematic review created two Boolean phrases—one each for the primary and secondary queries—and nested them in a manner that instructed EBSCO to return results if and only if an article passed the primary and secondary query. The final Boolean phrase contained 3,068 words. In order to verify replication of search result, the principal investigators used the filters described above to reconstruct the Boolean phrase query using the subscription credentials at American University, Emory University, Indiana University – Bloomington, and Princeton University. Search results were identical in all cases.

Missing Data: Abstracts and Keywords

EBSCO’s abstracting and keywording services are another reason why the PIs chose Academic Search Premier as the preferred primary data source. When EBSCO indexes a journal, it incorporates metadata—e.g., title, author, abstract, keywords—into its database for use as search datum. But article abstracts and keywords have not always been the industry standard. Even within publishers, the adoption of abstracts and keywords varied. In general, it appears abstracts became a standard among journals during the mid- to late-1980s; the integration of keywords followed in the late-1990s and early-2000s. When a record lacks certain information, such as an abstract or keywords, EBSCO hires subject matter experts to generate the content.

Second Phase

The first phase of the search produced 11,373 articles whose title, abstract or keywords matched at least one organizational and one international development keyword. Next, to eliminate false positives, two research assistants read the title, abstract and keywords of each article to ensure that it was a research article in which NGOs were an object of analysis. In the case of disagreement or uncertainty among the research assistants, one of the authors decided. This process produced a corpus of 3,402 articles. We then located a PDF version of 3,336 articles for use in computer-assisted content analysis. The remaining 66 articles could either not be located or converted into a format suitable for text analysis.

Appendix

Organizational Terms

  • third-sector
  • third sector
  • CBO
  • CBOs
  • community based organisation
  • community based organisations
  • community based organization
  • community based organizations
  • community-based organisation
  • community-based organisations
  • community-based organization
  • community-based organizations
  • civil society organisation
  • civil society organisations
  • civil society organization
  • civil society organizations
  • CSO
  • CSOs
  • community and voluntary organisation
  • community and voluntary organisations
  • community and voluntary organization
  • community and voluntary organizations
  • CVO
  • CVOs
  • faith based organisation
  • faith based organisations
  • faith based organization
  • faith based organizations
  • faith-based organisation
  • faith-based organisations
  • faith-based organization
  • faith-based organizations
  • FBO
  • FBOs
  • grassroots organisation
  • grassroots organisations
  • grassroots organization
  • grassroots organizations
  • GRO
  • GROs
  • HRO
  • HROs
  • human rights organisation
  • human rights organisations
  • human rights organization
  • human rights organizations
  • IANGO
  • IANGOs
  • international advocacy non-governmental organisation
  • international advocacy non-governmental organisations
  • international advocacy non-governmental organization
  • international advocacy non-governmental organizations
  • international advocacy nongovernmental organisation
  • international advocacy nongovernmental organisations
  • international advocacy nongovernmental organization
  • international advocacy nongovernmental organizations
  • INGO
  • INGOs
  • international non-governmental organisation
  • international non-governmental organisations
  • international non-governmental organization
  • international non-governmental organizations
  • international nongovernmental organisation
  • international nongovernmental organisations
  • international nongovernmental organization
  • international nongovernmental organizations
  • MFI
  • MFIs
  • microfinance institution
  • microfinance institutions
  • NGDO
  • NGDOs
  • non-governmental development organisation
  • non-governmental development organisations
  • non-governmental development organization
  • non-governmental development organizations
  • nongovernmental development organisation
  • nongovernmental development organisations
  • nongovernmental development organization
  • nongovernmental development organizations
  • NGO
  • NGOs
  • non-governmental organisation
  • non-governmental organisations
  • non-governmental organization
  • non-governmental organizations
  • nongovernmental organisation
  • nongovernmental organisations
  • nongovernmental organization
  • nongovernmental organizations
  • non-profit organisation
  • non-profit organisations
  • non-profit organization
  • non-profit organizations
  • nonprofit organisation
  • nonprofit organisations
  • nonprofit organization
  • nonprofit organizations
  • NPO
  • NPOs
  • nonprofit human services organizations
  • voluntary association
  • voluntary associations
  • PBO
  • PBOs
  • public benefit organisation
  • public benefit organisations
  • public benefit organization
  • public benefit organizations
  • private voluntary organisation
  • private voluntary organisations
  • private voluntary organization
  • private voluntary organizations
  • PVO
  • PVOs

Location Terms

  • Afghanistan
  • Albania
  • Algeria
  • American Samoa
  • Andorra
  • Angola
  • Antigua and Barbuda
  • Argentina
  • Armenia
  • Aruba
  • Azerbaijan
  • Bahamas
  • Bahrain
  • Bangladesh
  • Barbados
  • Belarus
  • Belize
  • Benin
  • Bermuda
  • Bhutan
  • Bolivia
  • Bosnia and Herzegovina
  • Botswana
  • Brazil
  • Brunei Darussalam
  • Bulgaria
  • Burkina Faso
  • Burundi
  • Cabo Verde
  • Cambodia
  • Cameroon
  • Cayman Islands
  • Central African Republic
  • Chad
  • Channel Islands
  • Chile
  • China
  • Colombia
  • Comoros
  • Costa Rica
  • Côte d’Ivoire
  • Ivory Coast
  • Croatia
  • Cuba
  • Curaçao
  • Cyprus
  • Czech Republic
  • Czechoslovakia
  • Democratic Republic of Korea
  • North Korea
  • Democratic Republic of the Congo
  • DRC
  • Djibouti
  • Dominica
  • Dominican Republic
  • Ecuador
  • Egypt
  • El Salvador
  • Equatorial Guinea
  • Eritrea
  • Estonia
  • Ethiopia
  • Faeroe Islands
  • Federated States of Micronesia
  • Fiji
  • French Polynesia
  • Gabon
  • Gambia
  • Gaza
  • Georgia
  • Ghana
  • Greenland
  • Grenada
  • Guam
  • Guatemala
  • Guinea
  • Guinea-Bisau
  • Guinea-Bissau
  • Guyana
  • Haiti
  • Honduras
  • Hong Kong
  • Hungary
  • India
  • Indonesia
  • Iran
  • Iraq
  • Isle of Man
  • Israel
  • Jamaica
  • Jordan
  • Kazakhstan
  • Kenya
  • Kiribati
  • Kosovo
  • Kuwait
  • Kyrgyz Republic
  • Lao
  • Latvia
  • Lebanon
  • Lesotho
  • Liberia
  • Libya
  • Liechtenstein
  • Lithuania
  • Macao
  • Macedonia
  • Madagascar
  • Malawi
  • Malaysia
  • Maldives
  • Mali
  • Malta
  • Marshall Islands
  • Mauritania
  • Mauritius
  • Mexico
  • Micronesia
  • Moldova
  • Monaco
  • Mongolia
  • Montenegro
  • Morocco
  • Mozambique
  • Myanmar
  • Namibia
  • Nepal
  • New Caledonia
  • Nicaragua
  • Niger
  • Nigeria
  • Northern Mariana Islands
  • OmanPakistan
  • Palau
  • Palestine
  • Panama
  • Papua New Guinea
  • Paraguay
  • Peru
  • Philippines
  • Poland
  • Puerto Rico
  • Qatar
  • Republic of Korea
  • South Korea
  • Republic of the Congo
  • Congo
  • Republic of Yemen
  • Romania
  • Russia
  • Russian Federation
  • Rwanda
  • Samoa
  • San Marino
  • São Tomé and Principe
  • Saudi Arabia
  • Senegal
  • Serbia
  • Seychelles
  • Sierra Leone
  • Singapore
  • Saint Maarten
  • Slovak Republic
  • Slovenia
  • Solomon Islands
  • Somalia
  • Somaliland
  • South Africa
  • South Sudan or
  • Soviet Union
  • USSR
  • Sri Lanka
  • St. Kitts and Nevis
  • St. Lucia
  • St. Martin
  • St. Vincent and the Grenadines
  • Sudan
  • Suriname
  • Swaziland
  • Syria
  • Syrian Arab Republic
  • Taiwan
  • Tajikistan
  • Tanzania
  • Thailand
  • Timor-Leste
  • East Timor
  • Togo
  • Tonga
  • Trinidad and Tobago
  • TunisiaTurkey
  • Turkmenistan
  • Turks and Caicos
  • Tuvalu
  • Uganda
  • Ukraine
  • United Arab Emirates
  • UAE
  • Uruguay
  • Uzbekistan
  • Vanuatu
  • Venezuela
  • Vietnam
  • Virgin Islands
  • West Bank
  • Western Sahara
  • Yemen
  • Yugoslavia
  • Zaire
  • Zambia
  • Zimbabwe
  • Africa
  • Asia
  • global south
  • Latin America
  • Non-Western
  • sub-Saharan Africa
  • third world
  • transnational
  • Central America
  • Eastern Europe
  • fragile state
  • Middle East
  • South America
  • Caribbean
  • Melanesia
  • Micronesia
  • Oceania
  • Oceanica
  • Polynesia
  • Soviet

Development Terms

  • aid chain
  • bilateral aid
  • CDD
  • community-driven development
  • developing countries
  • developing country
  • development aid
  • development assistance
  • economic development
  • foreign aid
  • humanitarian
  • international aid
  • international assistance
  • international development
  • LDC
  • LDCs
  • least developed countries
  • least developed country
  • least-developed countries
  • least-developed country
  • less developed countries
  • less developed country
  • less-developed countries
  • less-developed country
  • low-income countries
  • low-income country
  • low income countries
  • low income country
  • MDG
  • Millennium Development Goals
  • microfinance
  • micro-finance
  • multilateral aid
  • neediest countries
  • official aid
  • sustainable development
  • microcredit
  • micro-credit
  • non-state
  • nonstate
  • INGO
  • INGOs
  • international non-governmental organisation
  • international non-governmental organisations
  • international non-governmental organization
  • international non-governmental organizations
  • international nongovernmental organisation
  • international nongovernmental organisations
  • international nongovernmental organization
  • international nongovernmental organizations
  • NGDO
  • NGDOs
  • non-governmental development organisation
  • non-governmental development organisations
  • non-governmental development organization
  • non-governmental development organizations
  • nongovernmental development organisation
  • nongovernmental development organisations
  • nongovernmental development organization
  • nongovernmental development organizations
  • developing region
  • developing regions
  • developing world
  • world society
  • world polity
  • globalization
  • translocal