John Klingelhofer

Senior Software Engineer

, our GenAI-powered recruitment tool that streamlines patient recruitment for clinical trials. We detailed how Muse leverages OpenAI’s 

, dynamic workflows, and strategic automation to generate high-quality recruitment content efficiently.

The Technical Architecture Powering Our AI Recruitment Tool Muse 

Introducing Muse

In this follow-up, we’ll take a deeper look at the execution layer that powers Muse’s assistants—focusing on how the Threads API ensures contextual continuity, how assistants orchestrate multi-step workflows, and how users interact with and refine AI-generated content. We’ll also discuss some key challenges we’ve encountered, including improving content quality with limited user input, ensuring creative consistency, and learning from structured reasoning models. By addressing these challenges, we’re continuously evolving Muse into a more adaptable and intelligent system for clinical trial recruitment.

This project relies heavily on OpenAI Assistants and, more importantly, the Threads API. Threads provide a structured way to maintain execution history and context—essential for managing multi-step workflows.

Additionally, the API’s ability to automatically query the vector store for relevant material simplifies assistant configuration. This means vector lookups don’t need to be explicitly defined in workflows, making the system more user-friendly.

The assistant gathers all necessary material before execution, tailored to its role:

 – Strategic context is dynamically assembled, as described earlier.

For validation assistants – A structured process runs:

Retrieves descriptions of both the content-producing assistant and the validation assistant.

Extracts all relevant material from its vector store to create a validation guideline document.

Caches the result for future reuse, avoiding redundant processing.

This gathered context is inserted into the thread. So far, context window limits haven’t been a concern, as threads remain well within OpenAI’s maximum context length.

The assistant executes all defined task steps sequentially.

If the workflow involves dynamic branching, the assistant:

Clones the parent thread for each item, running subsequent steps in parallel.

The process continues until all nodes in the workflow tree complete execution.

Most assistants output data that must conform to a specific database model or JSON schema, ensuring smooth integration with front-end components. Once finalization is complete and broader orchestration nears completion, the data is presented to the user.

While advanced AI models and well-designed assistants significantly enhance content generation, they aren’t yet capable of consistently delivering expert human-level performance. To ensure high-quality outputs, users can interact with and refine the content in multiple ways:

Users can open a dedicated chat session on any generated content. (See image below)

These chats extend the original OpenAI Thread API execution, meaning:

Users can explore the reasoning behind the AI’s output, discuss source materials, or guide content in a new direction.

The AI retains context from the original workflow, allowing for informed and coherent responses.

If the conversation results in a rewrite, users can apply the change instantly, with updates reflected in the application. The chat interface is powered by real-time streaming via Server-Sent Events (SSE), ensuring a fast, responsive experience similar to a typical ChatGPT session.

Users can directly edit content for more precise control.

For richly formatted content (e.g., advertisements), the system:

Converts them into Markdown for easy editing.

Restores them to their original rich format using a lightweight structured-output LLM upon saving.

While validator feedback is applied automatically to the initial output, any manual edits trigger a one-minute validation process to review the changes. Users can accept or reject validator recommendations. Over time, user feedback from this manual review will help refine automatic validations, improving relevance, accuracy, and inclusivity.

This collaborative feedback loop ensures that AI-generated content continues to evolve and align with expert human judgment.

While Muse produces robust outputs, there are still key areas for improvement as development progresses.

1. Enhancing Content Quality with Limited User Input

One of the biggest challenges is ensuring novel and insightful content generation, even when user-provided data is sparse. A promising solution is to introduce more agentic behavior, enabling the system to:

Autonomously determine when additional real-world data is needed.

Select appropriate external sources to supplement missing information.

This evolution would make Muse more proactive in filling contextual gaps and ensuring high-quality outputs regardless of input variability. There is also an opportunity here to leverage Deep Research as well.

2. Improving Consistency in Creative Outputs

Muse’s most valuable contributions—such as identifying insightful patient groupings—require a level of creative ingenuity that can be challenging for a non-deterministic system.

Current Limitation – Some projects don’t always produce the most optimal results due to the inherent variability in LLM-generated outputs.

Potential Solution – Incorporating self-evaluation logic, allowing the system to:

Assess whether its initial response sufficiently meets task requirements.

Iterate or refine its approach before finalizing an output.

Models like o1, which follow a chain-of-thought reasoning approach, tend to deliver more consistent responses in structured tasks. This is because they allow for continued processing when initial outputs fall short.

While directly integrating o1 isn’t feasible due to cost, speed, and creative performance trade-offs, adopting similar recursive reasoning techniques within our workflow could enhance consistency and output quality.

By addressing these challenges, Muse will become even more adaptive, intelligent, and reliable in generating high-value content.

Muse represents a major step forward in AI-driven patient recruitment, but its true power lies in its ability to evolve. By leveraging OpenAI’s Assistants and Threads API, we’ve built a system that not only streamlines content creation but also fosters meaningful user collaboration. With real-time interaction, validation loops, and dynamic workflow execution, Muse ensures that AI-generated content remains accurate, compliant, and adaptable to the unique needs of each clinical trial.

However, our work doesn’t stop here. Challenges like enhancing content quality with limited input, improving creative consistency, and adopting more structured reasoning techniques will shape our next phase of development. By integrating smarter decision-making, refining validation processes, and continuously learning from user feedback, we aim to make Muse an even more intelligent and proactive tool.

As we push the boundaries of AI in clinical trial recruitment - and across clinical development in general - we’re excited about what’s ahead, and we look forward to sharing more innovations that bridge the gap between automation and human expertise.

Interested in joining our team to work on AI for clinical development? Check out our open roles 

Careers

, our GenAI-powered recruitment tool that streamlines patient recruitment for clinical trials. In this post, we’ll take a deeper look at the execution layer that powers Muse’s assistants.

Deep Dive into the AI Features and Technical Implementation of Muse

Patient recruitment has long been one of the toughest hurdles in running a clinical trial

. Defining a patient recruitment strategy and crafting recruitment materials that engage participants within regulatory guidelines—and ensuring they appeal to the diverse range of individuals needed for a trial to deliver accurate and meaningful results—is no small feat, and typically takes many months across a number of distinct teams. This challenge is frequently a bottleneck and a major-cost driver in the process of drug development.

Formation Bio’s AI system for patient recruitment

, represents a novel approach to overcome this challenge, harnessing the latest in GenAI technology to streamline the recruitment process for clinical trials (Figure 1). Its architecture is built around OpenAI's Assistants API tools (Figure 2). From a small handful of trial-relevant documents, Muse is able to produce everything ranging from prescreening questionnaires to fully-realized ads in a matter of minutes. Most importantly, it will synthesize a wide range of incoming information into strategic insights and various target audiences, and adapt the created content to best align with each of these audiences. The application of GenAI continues after initial creation, with conversational agents and sophisticated automatic validation enabling users to edit and adapt this content precisely to their target populations in a quick and intuitive manner.

In this post, we’ll explore the high-level architecture and key components that power Muse, offering a behind-the-scenes look at how we leverage OpenAI’s LLMs to create an efficient and effective tool for crafting recruitment strategies and content. We'll start by going over our data inputs, do an overview on Assistants, how Assistants are orchestrated and executed, how users interact with the Assistant outputs, and any challenges or additional opportunities to explore.

Muse’s outputs are entirely based on the data provided by the user, supplemented only by industry rules and guidelines that inform validation processes. Typically, user-provided documents include a study protocol, which outlines the eligibility criteria and objectives of a clinical trial. However, users can upload any relevant information to enhance Muse’s understanding.

Common sources of additional data include:

 – Background on the disease, including symptoms and impacts that may not be detailed in the protocol.

 – Statistical insights into the target patient population.

 – Information on existing treatments and competitors.

Once these files are processed and stored in a vector database, Muse’s AI assistants are ready to generate insights.

Each discrete task is handled by a dedicated OpenAI Assistant, which leverages machine learning and natural language processing (NLP) to analyze text, generate responses, and provide relevant information based on extensive training data. These assistants process user input, predict the most appropriate response, and can utilize tools such as web search or calculations to enhance their outputs.

Our implementation extends beyond OpenAI’s standard framework. In addition to configuring system prompts and key parameters (e.g., temperature), we design multi-step assistant workflows, chaining prompts together to generate more sophisticated and context-aware outputs.

Our AI assistants are grouped into three key categories, each designed to handle distinct aspects of the workflow:

 – Focused on research and analysis that inform content creation.

 – Establishes a standardized set of terms to ensure medical terminology remains accessible across all materials.

 – Extracts novel aspects of the trial from protocols and research, identifying unique characteristics that differentiate the study. For example, it may combine protocol details with treatment landscape data to highlight a drug’s distinctive effectiveness.

 – Uses attributes to identify meaningful patient groupings, allowing for more targeted and relevant outreach

 – Leverage strategic outputs to generate recruitment materials tailored to the identified segments.

 – Evaluate content through various lenses (e.g., IRB compliance, see image from Muse below) and provide structured feedback.

These assistants use dedicated vector stores containing regulatory guidelines and industry best practices to deliver precise, context-aware recommendations—similar to a human reviewer.

The IRB must approve all direct advertising materials for clinical trials before they can be used. In traditional workflows, this leads to significant back-and-forth revisions to ensure compliance. Many regulatory requirements are subtle—for example:

Ensuring the tone doesn’t overpromise or imply a guaranteed positive outcome.

Avoiding excessive emphasis on financial incentives for participants.

By equipping an assistant with IRB regulatory documents and defining a multi-step review process, we can surface IRB-like feedback early in the content creation process. This reduces compliance issues and minimizes last-minute revisions when the material is formally reviewed.

Manually programming and managing dependencies across multiple assistants would be both time-consuming and brittle. To keep our application flexible and support the seamless addition of new agents at any level, we designed it with a loosely coupled architecture in two key ways:

Each assistant’s workflow is configurable through a no-code interface, allowing users to chain prompts together into multi-step processes. For example, when defining an assistant to generate patient segments, the workflow might include:

 – Analyzing all generated attributes to identify meaningful patterns.

 – Identifying underserved populations or traditionally hard-to-recruit groups that might be a good fit for the trial.

 – Enriching identified segments with expected race, age, and demographic insights to guide content adaptation.

To enhance efficiency, we enable dynamic branching, where each workflow step clones the parent thread and executes in parallel with other branches (see 

Not all strategic elements are equally relevant to every assistant. Instead of manually defining these relationships, we allow assistants to self-determine relevant context at runtime by:

Analyzing their own description and comparing it with the descriptions of strategic assistants.

Automatically inserting relevant strategic outputs at the beginning of new threads to ensure contextual awareness.

This approach ensures assistants operate with the right strategic context without requiring explicit dependencies, making the system highly adaptable and scalable.

To illustrate how dynamic context gathering works, consider a simplified version of the Prescreening Form creation process. The system automatically pulls in relevant strategic outputs while filtering out unnecessary ones. 

It accepts outputs from strategic assistants that extract Inclusion/Exclusion (I/E) criteria and the glossary, ensuring consistency in terminology and regulatory compliance.

It excludes study-differentiating attributes, recognizing that these are more relevant for advertising rather than the strict guidelines required for prescreening forms.

Efficient Processing via Dynamic Branching

After the question outlining step, we employ dynamic branching to streamline execution:

Each question is assigned its own parallel subprocess, rather than processing the entire questionnaire at once.

This allows multiple evaluation and rewriting prompts to run concurrently, ensuring faster, more focused refinement of each question.

By leveraging context-aware selection and parallelized execution, our system maintains precision while optimizing efficiency.

This flexibility has significantly improved the quality of Muse’s outputs, allowing developers and domain experts to easily refine and adjust workflows without requiring Python expertise. By lowering the technical barrier, our system enables seamless collaboration and continuous optimization.

When generating content for a project, assistants are executed in a structured sequence:

These assistants analyze key inputs, such as glossary terms, eligibility criteria, and study attributes, providing foundational context.

Once strategic assistants complete, content assistants leverage their outputs dynamically, as described earlier.

3. Validators Review and Apply Feedback: 

As each content assistant completes its task, validation assistants evaluate the output and apply necessary modifications for compliance and clarity.

After generating the initial content, we refine it for each patient segment in an adaptation step.

Rather than recreating content from scratch, this phase adjusts language, tone, and emphasis to better align with the segment’s needs.

Early in development, we considered re-running the full content creation flow per segment but found that LLMs excel at fine-tuning for specific audiences—allowing us to optimize efficiency without sacrificing quality.

Despite the complexity of this workflow—potentially involving hundreds of individual tasks across multiple segments—our system is designed to:

Trigger sub-tasks immediately once dependencies are met.

As a result, the entire process - which typically requires many pharma teams across multiple months – can be completed in under 15 minutes.

Muse represents a significant leap forward in leveraging GenAI for patient recruitment in clinical trials. By integrating OpenAI’s Assistants API, dynamic workflows, and strategic automation, we have created a system that not only accelerates content generation but also enhances compliance, adaptability, and precision. The modular and loosely coupled architecture allows for continuous evolution, ensuring that Muse remains a powerful and scalable tool for recruitment teams.

Looking ahead, we see opportunities to refine our AI models further, incorporate real-time learning from recruitment outcomes, and expand Muse’s capabilities to cover even more aspects of the trial lifecycle. As the industry continues to embrace AI-driven efficiencies, tools like Muse will play a pivotal role in reshaping how clinical trials recruit and engage diverse patient populations.

 of this blog series we’ll take a deeper look at the execution layer that powers Muse’s assistants—focusing on how the Threads API ensures contextual continuity, how assistants orchestrate multi-step workflows, and how users interact with and refine AI-generated content.

In this post, we’ll explore the high-level architecture and key components that power Muse, offering a behind-the-scenes look at how we leverage OpenAI’s LLMs to create an efficient and effective tool for crafting recruitment strategies and content.

Articles by John Klingelhofer

Deep Dive into the AI Features and Technical Implementation of Muse

The Technical Architecture Powering Our AI Recruitment Tool Muse