Report: 2023 Family Diplomacy Training Impact

From July 9 to October 29 this year, Learning Life successfully held our second annual Family Diplomacy Initiative (FDI) training for family diplomats (FDs). This report details our process and results in measuring the impact of the training on participating FDs.

The FD training marks the continuation of FDI Phase 2: training a growing corps of volunteers worldwide to be family diplomats capable of effectively advocating for families that share similar challenges, from depression, disability or discrimination, to war, climate change and the displacements therefrom.  See the “Family Diplomacy Vision” poster below for more about the three planned phases of FDI’s development.

Launched in 2016, FDI is an ambitious, long-term, grassroots effort to connect, train and empower a growing international corps of family diplomats to participate in decision-making at local to global levels.  We envision a world more connected and caring because every family has one or more family diplomats, and those citizen diplomats advocate effectively via nonprofits, businesses, media and governments for the needs, concerns and aspirations of families worldwide.  From 2016 to 2021, Learning Life worked on FDI Phase 1, growing our international family diplomacy community on Facebook to now over 17,000 people worldwide, and conducting live, international dialogues on family issues.  As we continue increasing the number of people globally connected to FDI via Facebook, Phase 2, launched in 2022, over the coming years will develop the family diplomacy training, and gradually build a corps of committed FDs in collaboration with allied nonprofits across the world.  As we continue Phase 1 and 2, we will, in Phase 3, begin institutionalizing the placement of well-trained FDs with willing media, businesses, nonprofits and government agencies to give meaningful voice and decision-making power to families.  Thus, FDI is not a 10 year, nor even a 20 year project, but rather an ambitious, long-term movement to permanently empower families for a more caring world.

About the 2023 FD Training

As in 2022, in 2023 the FD training consisted of two parts:

Part 1: July 9-August 20 (seven meetings): FDs learned about (a) trends, patterns and issues facing families across the world, plus (b) citizen diplomacy (i.e., citizens collaborating internationally on shared interests and issues) and globalization in order to develop their basic knowledge as family-diplomats-in-training.  

Part 2: August 27-October 22 (nine meetings): The first two and last two meetings had the FDs perform their pre- and post-training stories, respectively.  In between, over five meetings, FDs learned about storytelling, then each practiced and performed a family story of their own.  The ability to tell family stories effectively is one powerful way to speak to the needs, concerns and aspirations of families.

The last meeting occurred on October 29 and focused on discussing what went well and what could be improved in the training.  Click here to learn more about the FD training, including eligibility, benefits, and other details.

The 2023 cohort of FD trainees consisted of 22 individuals from 12 countries: Brazil, Liberia, Nigeria, Ghana, Cameroon, Burundi, Uganda, Turkey, Afghanistan, Pakistan, Bangladesh and China.  Seven completed Part 1 of the training, two completed Part 2, and the rest (13) completed Parts 1 and 2.  (In 2023, seven trainees from China were able to complete Part 1 but not Part 2 due to the challenge of time zone differences.

Evaluation Method

For Part 1, Learning Life staff assessed the trainees’ improvement in knowledge about globalization, citizen diplomacy, and global family trends, patterns and issues by comparing their answers to the same survey question before and after six weekly training meetings from July 16 to August 20.  This simple, open-ended survey question was: “What do you know about citizen diplomacy, and the patterns, trends and issues facing families across the world?”  In both the pre- and post-training surveys, trainees had ten minutes to type their answers to the question.  Then, their pre- and post-training answers were mixed up and anonymized so the four volunteers who evaluated each trainees’ answers could not identify the persons, nor which answers were typed before or after the training.  Over about 2.5 hours in one weekend meeting in November, Learning Life’s founder and director, Dr. Paul Lachelier, first trained the four evaluators on how to numerically score the answers using the following simple rubric, then led the evaluators through the scoring of all Part 1 pre and post-training survey answers.

Rubric: “For each survey answer, score fact density (1, 2 or 3), then coherence (1, 2, or 3), then add up the two scores (lowest score=2, highest score=6).”  

Criterion / Score Poor (1) Fair (2) Good (3)
Fact Density Few if any facts are offered, and/or the facts offered are incorrect.  Some facts are offered, and some of those facts are correct.   The answer offers a lot of facts, and most or all of those facts are correct.   
Coherence Answer is unclear, and/or sentences are not logically connected in a coherent whole.   Answer is somewhat clear, and/or sentences are somewhat logically connected in a coherent whole.    Answer is clear, and/or sentences are logically connected in a coherent whole.  

For Part 2, Learning Life volunteers assessed the trainees’ improvement in their storytelling ability by comparing their story performances before and after five weekly story training and practice meetings from September 10 to October 8.  For their pre-training stories, the trainees were instructed not to prepare but rather to just tell a family story in five minutes or less on either August 27 or September 3.  Learning Life staff video recorded all these pre-stories. After the training, the trainees video recorded their post-training family stories and uploaded them to Learning Life’s Google Drive.  The post-stories were presented and discussed via Zoom on October 15 and 22.

In November, Dr. Lachelier organized eight Learning Life volunteers into two teams of four story evaluators. One team met on November 19, the other on the 26th.  Both teams met for about 2.5 hours, during which Dr. Lachelier first oriented them on the simple story-scoring rubric below, then each team scored half the stories, a mix of pre- and post-training stories.

Dr. Lachelier instructed the evaluators not to concern themselves with figuring out which were pre-training, and which were post-training stories, but rather to focus on the stories’ clarity and power.  He also instructed the scorers not to grade down the storytellers for any accent they have.  The evaluators generally did not know the trainees, and hence could not be biased due to familiarity.  However, the evaluators’ scores could have been biased by their perception of the voices or appearance of trainees since they could see and hear the trainees’ story performances.

After the evaluation teams scored the pre- and post-training stories, Dr. Lachelier organized and analyzed the scores with the assistance of Georgetown University student and Learning Life intern, Maggie Yang.

Rubric: “For each story, score clarity 1, 2 or 3, then power 1, 2, or 3, then add up the two scores (lowest score=2, highest score=6).”  

Criterion / Score Poor (1) Fair (2) Good (3)
Clarity The story wasn’t clear overall.   The story wasn’t clear in some parts.  The story was clear throughout.
Power The story was not delivered in an emotionally moving, thought provoking, or captivating way.  The story was delivered in a way that was sometimes emotionally moving, thought provoking, or captivating.   The story was overall delivered in a way that was emotionally moving, thought provoking, or captivating. 
Evaluation Results & Discussion

The twenty trainees who completed Part 1 improved their knowledge scores by 9% on average, from 4.13 out of 6 points prior to the training, to 4.65 out of 6 after the training.  Thirteen trainees improved their scores, seven performed worse.  The thirteen who improved had a 1.11 out of 6 or 19% average score increase, while the seven whose scores worsened had a 0.5 out of 6 or 8% average score decrease.

The fifteen trainees who completed Part 2 improved their storytelling scores by 28% on average, from 4.21 out of 6 points prior to the training, to 5.38 out of 6 after the training.  Fourteen of the fifteen trainees improved their scores, and one had no change in score. The fourteen who improved had a 1.16 out of 6 or 29% average score increase.

The trainees’ Part 1 essay answers in many cases contained more vague articulations of knowledge and expressions of opinions than demonstrations of specific, accurate knowledge.  Accordingly, Part 1 knowledge scores may improve more overall if we provide trainees, before their pre- and post-surveys, specific examples of answers that demonstrate less versus more knowledge, not just note that we are scoring improvement in knowledge.  Following proven pedagogical principles of repetition and recall, during the training, more review of key facts and concepts, like at the start of each training session as well as in Facebook FD group messages, and asking trainees to recall the knowledge in discussion, may also help enrich and cement their learning.

For evaluation of Part 2 storytelling, first, we should remain mindful of the order in which our evaluators see and score the stories as this may bias them to score higher or lower depending on, for example, whether they first view a well-told story or a poorly-told story.  Second, we could ask our evaluators to be more demanding in their evaluation scores to make it more challenging for our trainees to attain a certificate with honors.

Thus, overall, the trainees improved modestly in their knowledge of citizen diplomacy, globalization and family life, and considerably in their skill at storytelling.  There are some specific ways Learning Life can improve the training to boost knowledge learning, and the evaluation to increase the standards for honors.

 “​​The FD training made me a better storyteller.  I now know how to more effectively structure and present a story, and that enables me to advocate more effectively for myself, my family, and the issues we care about.” – Nusrat Jahan Nipa, Bangladesh


Learning Life is grateful to our trainees — Chirunim Agi-Otto, Juliana Arcanjo, Yibing Bai, Begum Burak, Terence Amah Chia, Yutong Chen, Ali Hussain, Eric Kajeneza, Joshua Kisubika, Ruiqi Li, Nusrat Jahan Nipa, Wenqing Pan, Leroy Quoi, Fatima Shah, Mohammad Siavash, Farzana Siraj, Abdul Basit Sulemana, Yujia Sun, Ruohan Yu, Xizhi Yu, Zineng Yuan, Bilal Zia — for their participation in the second year of our FD training.  We are also thankful to our 2022 FD trainees, Tenille Archie, Gustavo Carvajal and Maria Kavuma Kasaana for providing valuable feedback on our 2023 trainees’ family stories.   

We thank our 2023 interns — Summer Anwer, April Calderon, Keilyhan Echevarria Padilla, Nate Escobar, Maebelle Faragallah, Racquel Garcia, Aqua Harris, Ma’Shayla Hearns, Kayla Huong, Diya Jaisankar, Alaina Leasure, Chanel Leonard, Anya Neumeister, Eunjin Park, Harrison Reinisch, Mahum Shah, Dorothy Simon, Kailee Sullivan, Matt Turanchik, Jiayi Wang, Junlong Wang, Sami White, Maggie Yang — for supporting the FD training with their program research, outreach, marketing, applicant screening, storytelling advising and evaluation work.

We are also grateful to our trainers — Lisette Alvarez, Denise Bodman, David Meskill, Joe Toles, Bethany Van Vleet, and Ben Yavitz — for effectively teaching our FDs about family, globalization, and storytelling.  As well, thanks to our training evaluation volunteers — Peter Amponsah, Yasmine Ezzekmi, Darrell Irwin, Maebelle Faragallah, Jennifer Lopez, Cindy Mah, Gabriella Osei-Poku, and Alexandra Ravano — for their scoring of the trainees’ knowledge and storytelling ability. 

Last but far from least, we thank all our donors who help support our Family Diplomacy Initiative, with special appreciation to Michael Brown, Ana, Francois and Suzanne Lachelier, Sherry Mueller, Nusrat Sultana, Joe Toles for their generous financial support.