Traditionally,  identifying truth in politics comes from mainstream media and its fact checking. A recent Gallup poll, however, showed that only 29 percent of Americans trust fact checking.

Research in behavioral science suggests that we can address the spread of misinformation through a number of other effective strategies, which are brought together in the Pro-Truth Pledge (PTP) project. Several months ago, I wrote a post explaining the Pro-Truth Pledge and its mission. Since that time, two peer-reviewed studies have provided evidence of its effectiveness in changing the behavior of pledge-takers — private citizens and public figures  both— to be more truthful, for more than a month after they have taken the pledge. Both studies were published in prestigious psychology journals, Behavior and Social Issues and the Journal of Social and Political Psychology.

Quantitative evidence shows the Pledge is effective

The study in the peer-reviewed Journal of Social and Political Psychology suggests that taking the pledge results in a statistically significant increase in alignment with the behaviors of the pledge. The survey involved 24 participants filling out Likert scale (1–5) surveys self-reporting their Facebook engagement with news-relevant content on their own profiles and also with other people’s posts and in groups before and after they took the pledge, with 1 at lowest level of alignment to the pledge behaviors and 5 being full alignment. To avoid the Hawthorne effect of study participants being impacted by observation, the study did not evaluate current behavior, but past behavior.

We only recruited participants who took the pledge 4 or more weeks ago to fill out the survey, and asked them about their behavior after taking the pledge. Giving them this period also gave people an opportunity to have the immediate impact of taking the pledge fade from their mind, thus enabling an evaluation of the medium-term impact of the PTP on sharing news-relevant content.

This study method was informed by the approaches used by studies of whether honor codes address cheating, which is the most comparable form of intervention to the PTP. Such studies similarly rely on self-reporting by students on whether they have cheated or not cheated.

The study found that on one’s own Facebook profile, the median alignment with the PTP score before taking the PTP is 4 (SD=1.14), and the median alignment score after taking the PTP is 4.5 (SD=0.51). For engaging with newsworthy content on other people’s profiles and in groups, the median PTP alignment score before taking the Truth Pledge is 3.5 (SD=1.06).

The median PTP alignment score after taking the Truth Pledge is 4.5 (SD=0.65). For sharing content, 70.83% of participants (17 of 24 respondents) reported an increase of their PTP alignment after taking the PTP. The figure below provides a visual summary of the preliminary survey data.

Figure 1, Visual summary of preliminary survey data with PTP alignment in Facebook engagement

We conducted a second study, the one published in Behavior and Social Issues, to address the weakness of the first study’s reliance on self-reporting. The second study sampled 21 people, and involved researchers observing and evaluating the quality of Facebook engagement by study participants on their own Facebook profile.

Similarly to the first study, the second study avoided the Hawthorne effect of study participants being impacted by observation by evaluating past behavior. Researchers looked at the first ten Facebook posts with news-relevant content made four weeks after the pledge. Then, the researchers compared these ten posts to the first ten posts for the same period the year before the study participant took the pledge. Each post was coded according to quality, from 1 of lowest level of alignment with the PTP, to 5 of highest alignment.

The second study showed that the average PTP alignment before taking the pledge was 2.49, and after taking the pledge was 3.65, and conducted a paired t-test to examine whether Pro-Truth Pledge Alignment is significantly different after taking the PTP. The null hypothesis H0 for the paired t-test states that there is no significant alignment difference before and after taking the pledge and the alternative hypothesis H1 proposes a significant difference. There was a significant difference in the scores for Pledge Alignment before