The indirect effects of the COVID-19 pandemic will outstrip those from the infection itself and will particularly affect the most vulnerable populations. To address these challenges in Bauchi State, Nigeria, we conducted participatory research, engaging local communities to share their insights with policymakers to co-create recovery policies and strategies that cater to the needs of everyone in the community. We utilized fuzzy cognitive maps to visually represent knowledge about the causes of an outcome. Groups of about eight participants created 96 maps illustrating the factors that worsened their experience during COVID-19. They showed the causal relationships between these factors using arrows and assigned weights to estimate the perceived strength of each relationship. We condensed the 158 reported factors into 25 thematic categories and calculated the net influence of each category. The five most influential categories for a bad experience during the COVID-19 pandemic across all the maps were hunger and lack of food, reduced businesses/jobs, increased household conflict, lockdown, and stress/mental health problems. Maps consistently revealed the importance of hunger and food insecurity across various demographics, including gender (male and female maps), urban, rural, and remote communities, service providers, local authorities, and people with disabilities. Hunger was often a result of reduced business/jobs and the lockdown. While most of the maps identified household conflict as the third most significant cause of distress, male-specific maps ranked it as the sixth most important factor together with stress and mental health problems. COVID-19 impacts are strongly gendered; the maps confirmed contrasting views between male and female participants. Developing strategies for COVID-19 recovery that are equitable, inclusive, and sustainable is both a challenge and an opportunity. We will work with local communities to incorporate the content of these maps into cellphilms that they will use to address the main challenges they identified.