Our Research Methodology & Use of AI
How AI Can Help Speed the Diffusion of Innovation Among Rescue Missions
Our vision for this newsletter and podcast is that CVU would serve as a major R&D hub for the rescue mission movement in the same way that Stanford has largely served as the R&D hub for Silicon Valley. A major goal in this is to speed the diffusion of innovations within the rescue mission movement. We believe that effective use of AI tools can be a game changer to rapidly spread innovations within the rescue mission movement.
Above is a diffusion of innovation diagram that shows how new innovations, technologies or social problems diffuse throughout society. Part of City Vision’s theory of change is that we believe new social problems are becoming widely diffused in society (i.e. the opioid epidemic, the affordable housing crisis, etc.) faster than the diffusion of innovation of solutions to address them.
We need new methods to speed this diffusion of innovative solutions. Most of what City Vision does relates to solving that problem: educating students through courses & degrees as well as this newsletter and podcast.
We believe that what rescue missions need is not more primary research and first-time innovations, but for research and innovations already discovered to be more rapidly diffused within the rescue mission movement. For rescue missions, the William Gibson quote “the future is already here, it's just not very evenly distributed,” means that many of the new solutions and innovations have already been discovered, but most rescue missions are not aware of them or have not yet implemented them.
The old slower methodologies used by traditional universities, centered around primary research, simply cannot keep up. Because of this, for our reports and podcast, we are not doing traditional primary research, which has good reasons to be very slow and follow careful mythologies. This approach to primary research is essential to enabling an innovation to happen for the first time, but a newsletter and podcast like this focused on compiling secondary research are very helpful to replicate innovations and best practices.
Based on that, our approach is to use AI extensively in our research gathering and summarizing sources and in assembling podcasts for rescue missions. The three values of City Vision University are Jesus, Justice and Technology. With technology as a core value, our approach to AI is rather than see it as a threat, we take an approach popularized by MIT professor Erik Brynjolfsson’s TED Talk on how we should Race with the Machines rather than against them.
City Vision University’s Innovation Ecosystem
City Vision University’s innovation ecosystem is shown in the diagram above. To compile our podcasts and reports, we are building on hundreds of years of combined experience within the rescue mission movement of our staff and faculty, and 240+ books used in our courses and thousands more that our team has read, over 100+ courses and hundreds of documents compiled from partners in our toolkits.
We use this wide range of sources to identify innovations and significant changes emerging that may be of interest to rescue missions, then focus a podcast and research report on it. Once we identify a potential topic for a podcast and research report, we start compiling sources to use, as described in the following section.
How We Compile and Fact-Check Podcast and Reports
We first start by compiling research sources to use in the report and podcast. Google’s Gemini Deep Research reports presents a very valuable tool in that it essentially compiles research based on almost the entire Internet (except for what is behind paywalls). Unlike some other AI tools, Gemini Deep Research is much easier to fact-check: it is less likely to produce AI hallucinations (made-up facts) and provides references for key claims. A typical Deep Research Report might have an average of 50 references that can easily be fact checked.
For a given Podcast/Report compiled by CVU, we might assemble one to 20 sources. Often half of those sources will be Gemini Deep Research reports that examine various facets of the issue. You can see an index of some of the sources we used here. We also often ask Gemini Deep Research to compile reports on all the publicly available information from our list of 300+ missions.
All of these reports are then compiled into a Google Drive folder that we provide as a link in the podcast and research report. Except for critical data, we typically do not provide references within the actual report itself. This is because a typical report might have 100-500 references already in the sub-reports in Google Drive folder, and if we tried to link all of these in the primary report it would simply take too long.
We then combine these sources into a NotebookLM, and typically provide a detailed prompt to the AI on how to develop the podcast. We listen to the podcast and then often go through two to four iterations of adding more sources and adjusting instructions until the output is close enough to what we want.
We then use the podcast transcript to provide the shell of an article. Depending on the purpose, we often will do extensive rewriting of the article based on our own expertise and goals with the audience.
So based on this, what portion of a typical podcast or article is written by AI? That is hard to say because what weight do you give, since all the sources are largely curated by a human expert. That person also writes 10-20 extensive prompts guiding the research. While we may rewrite 20-80% of an AI generated article shell, nearly all of the process was guided by the human expert. Based on that, and the philosophy of “racing with the machine” I would argue that for our reports and podcasts it is typically about half human expert and half machine, typically with more than 90% of the creative direction coming from the human side.
We take a three-tier approach to fact-checking AI research.
For any critical data that is used in the final podcast and report, we will manually check the sources.
We fact check taking a statistical sampling of the sources, so we have a good sense of the error/hallucination rate of the research (which is typically very low).
We typically have considerable expertise on our team on the topics we are writing about, so we can quickly spot any gross errors.
Having said that, it is highly likely that given this methodology prioritizing speed of diffusion of innovation, there may be some errors that slip through. If we were publishing primary research in peer-reviewed journals, this level of error rate would likely be a problem.
However, we believe we are playing more of a curating/editorial/journalistic function of diffusing secondary research to a wider audience. We believe that the benefit of speed of diffusion of innovation will outweigh the cost of minor errors slipping through.
There are those in academic research circles that take more of an inquisition approach with little forgiveness for any errors. This group wants to “cancel” anyone who makes a mistake. While there are good reasons for this in primary scientific research, we do not believe that applies here.
Because we are largely serving the rescue mission community that we are in relationship with, we ask for grace when there are any errors. We will be quick both to apologize and to correct those errors. What we will not do is have external secular standards of academic primary research imposed into a context where it both does not fit and would cause harm.
In our courses, we teach the Competing Values Framework (CVF). Within the CVF, traditional primary research methodology is largely within the Control quadrant, which by definition is optimized for a very low error rate. Our approach to this podcast/newsletter and diffusion of innovation is largely within the Create quadrant, which prioritizes speed, creativity and innovation with a more gracious treatment of errors to quickly learn from them and correct them.
That is the approach we will take in our newsletter. We believe this will enable us to quickly learn and iterate to improve this process. We believe that this approach will maximize the benefit to the larger movement by enabling our ability to innovate and respond to social crises to catch up with the speed they spread.
Our Strategy: Why We Use the Term “Rescue Mission”
We use the term “rescue mission” to avoid the problem I call the “the artist formerly known as Prince” problem. Prince replaced his name with a symbol, and then how do you talk about him? That's the challenge we have with rescue missions and even evangelicals. There's the movement formerly known as evangelical. What's happened is politics has really tainted that term “evangelical’.” Just about anyone these days that uses that term, especially around non-Christians, they have to qualify and say, "Yeah, we're not like the people who don't actually follow the Bible and are just mean."
What a lot of organizations have done, even the most conservative churches and ministries is to rebrand. In cities like Boston churches like the Southern Baptists and Assemblies of God, often are not branded as Southern Baptist. They remain a part of the denomination and they follow all the tenets, but they often use some sort of trendy, cool name. Even Campus Crusade for Christ recognized their brand had become a stumbling block on college campuses out of step with the times, so they renamed themselves “Cru.” It's important to retain the theology and values, regardless of brand.
There is the same issue with the movement formerly known as the rescue mission movement. There are similar challenges, and whether to rebrand is a complex question. It has to do with the value of the Gospel Rescue Mission brand in your region and the risk of rebranding. What's important is, are you retaining theology and values regardless of the rescue mission branding?
What’s important is to retain the core values of "gospel rescue missions". You could call this the movement formerly known as Gospel Rescue Missions, but that seems self-defeating. Calling it a movement for “life transformation ministry” is too generic and just opens the door to mission drift.
Those who are not familiar with the full history of the Gospel Rescue Mission movement and how it is both a descendant of but distinct from the City Mission Movement, may not understand the significance of the term “rescue” and how that relates to key factors that distinguish the two movements. While both the City Mission movement and the Rescue Mission movement focus on problems of urban poverty, there were three things that made the Rescue Mission Movement distinct from the City Mission movement:
Focus on the destitute and recovery: Because Jerry McAuley was himself previously a homeless alcoholic and his wife Maria was previously an alcoholic and prostitute, there was a much stronger emphasis on serving the most destitute and involving addiction recovery in ways that are distinct from the emphasis of City Mission.
Value on indigenous leadership from the destitute and “lived experience”. In models of urban ministry & missions often there is a distinction between ministry of the poor, with the poor and to the poor. Because of Jerry and Marie’s background they influenced the core “DNA” of many early rescue missions to lean more toward the “of/with the poor” ministry model with a stronger emphasis on the value of “lived experience.” This is contrasted to the better-off background of David Nasmith, the founder of the City Mission movement, which more typically had a “with/to” the poor ministry model. Throughout the Rescue Mission movement history there have been a mixture of both models, but historically the Rescue Mission
Distinct Roots of American Evangelicalism. While both the City Mission and Gospel Rescue Missions have their roots in evangelical revivals, there are distinct influences in the Second and especially the Third Great Awakening in the United States that had a profound impact on the rescue mission movement. The reason why it is called a “movement” is that for much of its history the leaders have had a missionary zeal to spread the movement and save lives. In addition, many other countries affiliated with the City Mission movement outside of the US have secularized more quickly which has placed increased pressure on City Missions in those countries to secularize. For both of these reasons, it is arguable that there is a stronger emphasis on the Gospel-Centeredness / Christ-Centeredness of the Gospel Rescue Mission movement throughout its history. This is a significant reason for the word “Gospel” in the “Gospel Rescue Mission” movement.
In considering these distinctives, what’s important isn’t that every organization uses all components of the brand of “Gospel Rescue Mission”. What is important is that we all understand and retain the values for all three components. There are currently strong pressures both in Christian and secular circles to “deconstruct” the values and emphases from all three words in the “Gospel Rescue Mission” movement. All of these represent mission drift.
Gospel: because of society’s values are changing there is pressure essentially to secularize
Rescue: because of increased pressure to professionalize as mission grow there is growing pressure to essentially “gentrify” the movement to move away from valuing lived experience and indigenous leadership “of/with” the poor to align with upper-middle class norms more typical in suburban megachurches to become “with/to” the poor.
Mission: the whole concept of “missions” has largely been deconstructed, even at the most evangelical institutions. While it is important to learn from the valid critiques of unhealthy “missions” mentalities, if we throw out the zeal and dynamism that comes with a missions mentality, then the Gospel Rescue Missions movement will no longer be a movement. The result could be to follow the common pattern of becoming another institution that follows the common pattern of institutional decline and secularization.
Individual missions, as well as the movement overall, need to resist these attempts to deconstruct the core values of the movement while also learning from their valid critiques. To fully delve into that discussion will involve another article.
Fundamentally why we use the terms “rescue mission” and “gospel rescue mission” and have focused this podcast/newsletter on this group is because rescue missions represent a strong community that has a strong common mission and values that God is using to transform the lives of millions of destitute people. Our goal in this podcast newsletter is to be one small part of that community to help make it more effective, while staying true to its values and mission.