November 14, 2023
Participants
Opening Remarks
Oliver Ball: Hey, welcome to all those who are just dialing in. We'll give it just a few more moments for others to join the call before we begin.
Okay, let's get going today then. So hey everybody, thank you so much for joining us today for the fourth of the unbridled Webinar series by Dark Horse consulting. I am your series host Oliver Ball, part of the Business Development team here at Dark Horse.
I am very excited to see what we have in store for today's panel discussion. Just to set the stage a little, I don't think any other group and industry has the depth of experience that this group does in the regulatory affairs, the cell and gene therapy space. This group has not only decades of experience actually working at the FDA but continues to interact frequently with the agency through our extensive client base.
You know we've had now, I think, over 350 clients over the 9 years that we've been running across more than 650 projects. So the privilege of constant exposure to the agencies thinking on a number of topics, including some of the most pressing issues that people are facing today.
And this puts us in a good position not only to help our clients directly but also to provide discussions such as today's where we hope to offer you a few of the insights into some of the things that the agency are thinking about, and some of the strategies that we're suggesting to our clients to address some of these challenges.
Before we get into it, I just want to remind the group that we will be having another webinar next month on the thirteenth of December where our 3 managing partners will be discussing some of their observations, their comments, and their thoughts on what the 2023 calendar year has brought us, as well as gazing into that highly polished crystal balls and thinking about what 2024 might bring. So you know, these 3 have a lot of experience and a lot of insight into what's happening in space. So do register for that. Now, you can register for that online today and should be a really good discussion.
I just want to remind everybody as well that today's discussion is intended as an interactive discussion. So please do submit any questions that come up during the course of the presentations using the Q&A function in the zoom interface, and we will try to get through as many of those as we can, and we never have time to get through all of them. So you know, if there's something that you would really like to discuss with us, we're always happy to have a one on one conversation. So just let us know if that's the case.
We will also try and get through some of the questions that many of you submitted when you registered today. As well. So thank you for those who submitted interesting and thoughtful questions when you registered.
So without further ado, I'd like to hand over the floor to our panel chair for today, Katie Spink. She is one of our 3 managing partners and our chief operating officer. She will be introducing the panel today and chairing the discussion. So, Katie, I will hand over to you.
Panel Introductions
Katy Spink: Hi, thank you, Oli. Hi, everyone! As Oli said, I'm Katie Spink. I'm the chief operating officer and one of the managing partners here, and I'm really privileged to be joining you today as the chair of this distinguished panel. I have with me today 6 of my esteemed colleagues for what should be a really interesting and wide-ranging discussion.
So joining me today are Kim Benton, who is a master principal and also the head of our regulatory department. Keith Coates is a senior principal at the practice and is the head of our facilities and engineering compliance department. Don Fink is a master practice expert. Tal Sauls is a practice expert, and Brent Morse and Kevin Willsey are both principals in the practice. I'm going to begin by asking each of them to introduce themselves and just share a couple of minutes commenting on a topic that's been top of their minds recently relating to the regulatory space. So I'll start with Kim. Can you tell us a little bit more about your background and share some insights on a topic of interest to you?
Kim Benton: Katie, happy to. I'm Kim Benton, as Katie mentioned. I'm master principal and head of regulatory at Dark Horse. I joined in January of 2023, and before that I spent over 22 years at FDA CBER in regulation of cell and gene therapies, and about 20 years of that was as a supervisor at various levels of supervision, and my last position was the associate director for regulatory management, where I oversaw all the regulatory work in the office coordinating with the other leaders and advised the office director.
So you know one thing that obviously I've been watching very closely is the stand up of the new super office and how that has come about, how positions have been filled there. So it was mentioned by Nicole Verdun when she took over in July. She's done various sort of meet and greet type fireside chat sessions at I know ARM did one as a video session at the ASGCT Policy Summit, and I believe she was at meeting on the mesa. But I wasn't there. But she's made efforts to have the community selling gene therapy community, you know, get to know her and how she is planning to bring the super office together and stand it up and how it's going to function, because that's a unique structure. It's unique to CBER.
And so she's mentioned that a lot of the management positions have been filled. 5 of the 6 office director positions have been filled, but one that hasn't, and I know we're all interested in hearing is the office director for the clinical office, office of clinical evaluation. So once that position is filled, and maybe then the other positions under that, the division directors will be in place below that.
She's mentioned that one of her priorities is continuing to increase communication and sharing of information with stakeholders. So that's very important. And she's mentioned that a lot of the priorities of the office will need to actually fill these positions, both managerial and reviewers. So it's unclear to me yet how many of the review positions have been filled, both vacancies that came from attrition, and the new FTEs that were given to the office in PDUFA.
One concern that I have with filling of these positions is again, there's another potential government shutdown coming. Hopefully, that will be avoided. But again, it may be another continuing resolution. So there's always that sort of looming over FDA until there's the budget. So for the most of the review activities for INDs are not impacted by a government shutdown. But the hiring process would be shut down. So filling those vacancies would be paused, and have to restart that process or pick it back up when the shutdown is over.
So I can take another few seconds about a potential government shutdown. I mentioned IND activities generally proceed fine in a shutdown, because majority of FDA's budget comes from user fees and not from appropriations so previously collected fees that are in the bank are used to keep the lights on, to keep activities, ongoing review activities for any product that is a PDUFA covered product. So in office of therapeutic products, that's everything but allergenic core blood. So most of the stakeholders are not impacted. What could be impacted would be any products that would be submitted during a shutdown that have to pay a new user fee or are subject to a user fee. So new BLAs, 510Ks, PMAs, that type of thing that review may not commence during a shutdown and other thing that may not happen is travel to conferences or inspection travel. So those are the types of things that I will be looking for if there's a shutdown.
Katy Spink: Great. Thank you, Kim, very interesting and good to hear at least that clients filings reviews won't be delayed. Heath, let's go to you next.
Heath Coats: Very good. Thanks, Katie. Hi, I'm Heath Coates, senior principal with the practice. I've been with practice now for about 16 months, and, as previously announced, I'm head of the facilities and engineering compliance group here at Dark Horse. I've been in the consulting space for the last 4 years, basically helping clients with aspects of facility review, preparing for pre-license or pre-approval inspections, and then also helping those clients remediate any inspectional observations, warning letters, or consent decrees that they may be given.
I am able to perform that function since I served as a member of FDA CBER division of manufacturing and product quality for about 7 years. I was a reviewer and an inspector with DMPQ, and was primarily responsible for facility CMC elements of contamination control serving as what was called a technical lead for type C facilities, meetings and, more importantly, really, leading pre-license and pre-approval inspections where the inspection teams would assess all systems of GMP.
I was fortunate enough to be on the inspection team for 2 of the 3 facilities for the first approved CAR-T product back in 2017. My, how time flies! Prior to my time at FDA I ran a validation program for close to 10 years at a cell therapy contract manufacturing organization, where again we performed all aspects of required validation activities.
Back in my early days in industry, I gained a lot of hands on experience from all types of activities such as DNA plasmid prep, sequencing tissue culture activities, benchtop fermentation and also protein purification. I was fortunate enough to have a master's thesis project where I generated and characterized a hematopoietic stem cell line from fetal mouse liver cells. So I've been fortunate to have a lot of experiences over the years.
Kind of the thought that's on my mind right now, I'm pretty much all compliance all the time, but I also really try to strive to incorporate that into day to day operations. I'm sure it's a challenge for companies to transition from a product development clinical study mindset into the mindset of developing appropriate control strategies to develop a commercial manufacturing process and, more importantly, keep that process sustainable and compliant.
You know, the CFRs are those guide rails, and they are written very generically to give applicants and sponsors leeway and how they implement their control strategies. But we need to remember that you know what's written, the CFRs for GMPs are the bare minimum and you know, the expectation is to go above and beyond to really incorporate an intrinsic quality culture and oversight in day-to-day operations.
You know, there are other entities out there who have provided review letters and SOPs, quantifying how warning letters have gone down in the past couple of years. But complete response letters have gone up. You know that in coupling with the day and age we're currently in with accelerated approval timelines and things of that nature, those accelerated timelines do not lower the bar for compliance with GMPs.
There's also plenty of articles out there summarizing really the most numerous types of inspectional findings that have really been around for the past 20 years, and some of these recurring inspectional deficiencies are really product agnostic, and are outside of the world of cell and gene therapy, and they focus on such topics as deficiencies and deviations and investigation handling, deficiencies in the way written procedures are written sampling plans aspects of process and product control, and how those records are reviewed after execution, stability program deficiencies.
Cleaning and maintenance of equipment is a recurring theme as well as personnel training and the personnel qualifications. And then, again, as I said, just back to production records, really mimicking or documenting essential and critical operations in the manufacturing process. And it is really a head scratcher, and that how these same type of topics have really been around for decades without significant changing of the types of topics. And I'm somewhat puzzled in how these haven't really been able to be corrected over the years, because, you know, with the way these 483s get documented and publicized, you know, FDA really can give a roadmap as to what incorporated and what corrective actions can be incorporated.
You know, it does seem like in some cases my guess or reasoning, for this is that in some cases the quality oversight and compliance isn't really developed to be intrinsic through the entire process, and really only starts getting focused later in the development cycle of the product and sooner to the BLA submission.
In most cases applicants don't take full advantage of FDA inspections. When FDA is on site for an inspection, they have a captive FDA audience, for however long FDA is on site, you know. Use that audience to your advantage, and I think there's an opportunity to change the mindset from surviving an FDA inspection to really embracing that opportunity to engage dialogue with the agency.
I think in some cases, also, you know, companies may try to go it alone and not take advantage of all the reference materials or qualified subject matter experts that are in the industry to help with that incorporation of compliance best practices, whether those be again actual full time employees or consultants, there's certainly equal opportunity to do both.
And then I realize that there's a cost associated with that. But I would willingly suggest that you know the costs associated with developing knowledge to do things right the first time is a lower cost then the costs associated with remediation efforts from getting a 483, a complete response letter or God forbid a warning letter. I think also there, the last point for this my thoughts are, I think, unfortunately, there are some, also, there are some consulting companies out there that are really focused on providing problems instead of solutions for clients problems. I think again, you know, in that particular case, you, in the spirit of being a full service resource, you know, if you identify an opportunity for improvement, it's your obligation to suggest a remediation activity as well.
I also think the how risk is assessed, right? Risk, you know, risk always needs to be based on a threat to patient safety rather than whether or not it would be a risk to a regulatory inspection. And then I think again, just the one last point I'll make, is that you know, again these ongoing deficiencies with trends moving towards, you know, decentralized manufacturing activities and point of care opportunities. Those are conditions that are going to be more challenging to have routine oversight and compliance control. So the sooner we can get, you know, robust corrective measures into some of these standard and recurring problems the better off we have the better off we're going to be for some of these new platforms.
Thanks.
Katy Spink: Great, thank you, Heath, a great reminder to us all to build quality throughout development, to deliver our best to patients. Next, I'm going to ask Don to introduce himself and tell us a little bit about what's on his mind.
Donald Fink: Thanks, Katie, and a shout out to Oli and the BD team for organizing and corralling these folks together for this session. Quickly, I've only had 2 jobs. Dark Horse is one of them. My entire career was spent at the FDA up until 2020 over the last 20 plus years in cell and gene therapies as a CMC reviewer with that expertise. And I continue to leverage that with the clients that I deal with today.
What's interesting to me at this time in this place has been the activity of the advisory committee meeting and its use and the role of the CTGTAC as part of decision making going forward. We've had 3 meetings in 6 months which says to you all, at the level of a product approval, the field is maturing, and it's maturing pretty rapidly. I would have been happy in my early days to have 3 meetings in 3 years much less 3 in one year. But now we're getting them at a pretty rapid pace. And if you look across the spectrum from Sarepta, Brainstorm and Vertex, the products are all different. There's transgene delivery through vector mediated, you've got cell based product with Brainstorm, and you have Vertex through the genomic editing. This is a challenge for a single committee to be able to handle and manufacture decision making in that complexity.
A couple of features that I'd like to just share across the board regarding these meetings. So the first 2, Sarepta and Brainstorm, I use euphemistically, they were conducted over protests and over protests of the review team because the review team really had reached conclusions around what they felt where the product should be headed, and in each case a sponsor was able to get to an advisory committee meeting, one driven by management at the Center directors level at CBER for purposes of getting full exposure in an area of accelerated approval for a first in class product, and the other and a novel product cell based product which over time had not been shown to be clinically effective, and yet it was felt to adjudicate that in a public forum was everyone's benefit.
If we look at this Sarepta case, whoa, way short turnaround. They weren't going to have a advisory committee meeting until after the mid cycle meeting, so hats off that had to be organized. Heavy lift by reviewers to get that to happen.
If you look at the Brainstorm issue where they wanted to have a public airing of the product in a format that could be heard by multiple parties, the committee was doubled in size, allowing experts to be brought in, and to have a full conversation about that.
And finally, with Vertex is, I think, is a new paradigm going forward, and we'll have others talk about Vertex outcomes later. Related to using advisory committee for advice and not for voting. That's a Commissioner Califf interest. And I think you're going to see more of that. And here we had Vertex talking about a safety issue largely.
The common thread through all of these was CMC in my view, because I'm CMC reviewer. But if you look into what impact did some of the decision making? For example, in Sarepta, there was a disconnect between the commercial manufacturing process and the process that had been used to get most of the clinical data, and that commercial process wasn't really going to be fully vetted out until the confirmatory trial. And I think we all know what happened with the confirmatory trial.
The second would be in case of Brainstorm, an autologous product on demand had to be manufactured here. The big issue, of course, is inconsistency, and another is in the potency assay and a lack of really a potency assay that allowed for adjudication of clinical results and some expectations around improving that. And then, finally, we had the precision of the genome editing tool and what the consequences of an imprecise and poorly controlled product could be there. So CMC, while not featured, it's always a clinical discussion. Underneath all of that CMC had an important role to play.
And so finally, I just would want to say, with respect, in all cases, what these committee meetings allow for is transparency. It allows for the agency to open the door on the regulatory review process, the regulatory review thinking, and in some cases even broadly distribute that current thinking in a public platform. And with that I'll turn it back to you, Katie.
Katy Spink: Great, thank you, Don, and we'll certainly be spending more time talking about those AdCom meetings later in the session as well. Key topic of interest for sure. So next I'm going to turn it over to Tal.
Tal Salz: Hello! I'm Tal Salz. I'm a practice expert in the regulatory group at Dark Horse. Actually, next week would be my one year anniversary with Dark Horse. Prior to that I was a cell and gene therapy reviewer at the FDA. I was brought in as a commissioner's fellow in 2016. Scott Gottlieb was the Commissioners at that time and in addition to the routine review work, which I've reviewed, variety of products, I worked on a regulatory project which focused on CGT comparability at a time when there was little framework for that and there was no approved gene therapy product at the time. It was kind of like a year before the first gene therapy was approved and then I took part in developing the CGT comparability guidance that was recently published. And naturally I've been spending a lot of my time at Dark Horse consulting on comparability. So I think it's fair to say that comparability has been on my mind.
I also want to mention while at the FDA I was also involved in harmonization and other guidances and external engagements on different capacity. And I'll end at that because I think we're going to talk a little bit more about comparability after introductions.
Katy Spink: Yes, great, thank you, Tal. And certainly we're going to spend some time on comparability later in the session. That was one of the most frequently asked questions among our registrants. But I also want to take the opportunity to advertise another webinar that Tal and Kim and another member of our practice, Jacob are going to be participating in through the RAPS DC Baltimore chapter. That's going to be on December fourteenth. So check out our LinkedIn feed for more information. Or I think Oli's going to post in the chat as well the link to the registration. So if you don't get enough in our session today, you can go even deeper in that in about a month.
Tal Salz: And, Katie, I'll just add to that that on Thursday I'm actually also giving a talk on a potential standardization in comparability. It's a workshop that was organized by NIST, FDA and USP. So if you want to hear about standardization you can tune into that one as well.
Katy Spink: Perfect. Thank you, Tal. All right. Next I will turn it over to Brent.
Brent Morse: Thank you, Katie. My name is Brent Morse. I'm a principal at Dark Horse. I've been with the practice about 18 months now. And prior to Dark Horse, I spent time at CRISPR Therapeutics, where I built out Analytical Development Group there as responsible for developing the analytical strategy for their gene edited stem cell products and also their off the shelf CAR-T products. After that I also served as vice president of process and analytical development at VOR Biopharma, again overseeing the development of gene editing and edited stem cell products and their CAR-T products in both roles I oversaw external manufacture of gene editing reagents, such as Cas9, guide RNA and viral vector.
So what's been on my mind? To me probably the most exciting recent news was the Advisory Committee meeting for EXSL. So that's Vertex's CRISPR gene edited hematopoietic stem cell product for patients with sickle cell disease.
Any of you who tuned into that saw that the efficacy data was very compelling and for the topic of the day, which was the question of whether the off target analysis was adequate to assess risk. It seems that the advisory committee mostly agreed with Vertex's arguments, and that's that the number of patient batches assessed for off target editing, as well as the relevant sample sizes that they were evaluating in the 1,000 genomes database was sufficient, and that there I think the consensus was that there wouldn't be much to gain by additional testing.
So for a developer, the question that might come to mind is whether any general conclusions can be drawn here from the AdCom and whether this could inform their approach to development. Does it signal how the agency is going to consider off target for future CRISPR'd products?
And based on the discussion, it seems like they're proceeding in a very pragmatic way, trying to strike a balance between safety and practicality. So at least in this setting, a well reasoned argument, supported by appropriate data, was considered as a suitable alternative to strictly exhaustive testing.
So that being said, as a developer, I wouldn't necessarily assume that Vertex's approach will meet the bar in the future. There are a lot of product specific aspects here that were likely considered that we didn't necessarily have visibility to. One of them we did was that the guide RNA off target profile at a base level it was pretty clean, and not all developers are going to have that advantage.
We also don't know the purity of the guide RNA, the impurity profile, the relative reproducibility of manufacturer. So just on the guide RNA alone, there are a lot of factors that were likely contributed to the review process so far.
And then the other thing I'd like to just point out is as the field progresses, as we kind of alluded to earlier, there's a lot of activity in the space, and the products are becoming more complex. Many more edits, we're venturing into the field, incorporating multiple new modalities with each with their own off target aspects to consider. We're also venturing into indications with larger populations, so all of these could certainly contribute to maybe a recalibration of the scrutiny that may be applied to these products. So I think all these will need to be considered on their own, so overall very positive development for the field. Probably a little early to extrapolate too much at this point. So thank you.
Katy Spink: Yeah, great thanks. Thank you, Brent. And that was really an interesting development in the last couple of months. But we'll spend some more time on as well later. So before I turn it over to our last panelist, Kevin, to introduce himself, I just want to remind those listening in to please submit questions in the QA. If there's anything you want to hear about and again, thanks to those who submitted in your registration, we'll certainly be touching on some of those as well. But if you have one that comes to you as you're listening to our panelists speak, go ahead and submit it through the QA button.
All right Kevin, can you please introduce yourself?
Kevin Whittlesey: Great thanks, Katie. Hi, everybody Kevin Whittlesey, I'm a principal at Dark Horse. I've been with the practice almost 2 years now. Been my entire career 25 plus years in various aspects of gene and cell therapy. I started in non viral gene therapy and then moved into cell therapy. I then went to the FDA like Tal. I was a Commissioner's fellow where I worked as a reviewer for cell and gene modified cell based products with the agency a couple of years. I then left the agency and wanted to take that reviewer perspective and make it practically useful more on the product development side of the house. I've done so in a few different capacities.
Most recently, as project lead for multiple AAV based gene therapy products, taking that regulatory process understanding and utilizing that to design highly efficient and effective non clinical programs that will efficiently and effectively get to and through the FDA and get your IND open as quickly as possible. So that's my regulatory lens.
And what's been on my mind has been, it is been one, a quite a broad, reaching change in the non clinical, regulatory expectations of the FDA. And that's the implementation of SEND data requirements for nonclinical data. That's new. SEND is an acronym that stands for the standard for exchange of nonclinical data. And this is something that's been coming for a long time.
The center for drugs, CDER has had that requirement in place since 2016, and we know it's been coming to CBER, but we haven't known exactly when. There have been several delays to the implementation of it, and it is now in place and it for it is required for certain non clinical studies that were initiated after March fifteenth of this year.
And that means, if you have a study that is initiated started after March fifteenth that is either a single or repeat dose, toxicology study or a tumorigenicity study, it must have SEND data accompanying that submission. And the implications are fairly significant. Anyone who's had a study quoted by a CRO, they will commonly ask for that potential for tens of thousands of additional dollars for the development of that SEND data package. So it's non-trivial and it can have huge impacts on your regulatory submission.
If you have, if you submit any study in certain key locations of module, for you have to include a particular kind of file that is what the electronic gateway looks at to determine when your study was initiated. And if your study was initiated after that trigger date of March fifteenth of this year, it will then expect to see certain components SEND data file submission. If that SEND submission is not there, the electronic gateway will not allow your IND to be submitted. It will throw an error code, and it will not allow your IND to be submitted. So it's important to get it right, and the particular challenge of that to cell and gene therapies is that the implementation of SEND requirements for cell and gene therapies are not fully in place.
The next round of the implementation guide will have certain aspects of certain kinds of experiments, certain data sets certain units in terms of the actual collection of the data that will better capture and more fully capture cell and gene therapies. But they're not there yet. So the requirement is there, and you have to submit the aspects of your data that are able to be captured in the current SEND format requirements and then include an accompanying document to explain what the differences are of what's in your study report versus what's in the SEND data.
So the fact that this implementation is in place, but the requirements and expectations and capability to do so for cell and gene therapy is a challenging place right now that and it's going to affect all sponsors conducting toxicology studies that have started after March fifteenth of this year.
Thank you.
Katy Spink: Thank you, Kevin.
Discussion on Comparability
Katy Spink: So I think the most frequently asked topic in questions that were submitted by our registrants related to the comparability draft guidance and requests for further information on comparability. And so I'm going to start there and follow up questions to the panel in particular, we had a lot of interest about expectations at different stages of development early versus late stage products, and also some questions about statistical approaches to comparability. So I'll turn it over first to Tal and then I'll invite the rest of the panel to chime in as well on either or both of those topics related to comparability.
Tal Salz: Okay, so I might be biased. But I think that the guidance provide a really good framework for CGT comparability. And I can speak from experience that it was very thought through. There's a lot that went into it.
The guidance is meant to provide tools and brings on considerations. It's not a recipe. So if you're looking for a recipe, come to Dark Horse, we can help you craft one. We can review your proposal and your protocol. But the guidance really meant to guide you, and it covers a really large spectrum of product types. And each sponsor has a different toolbox and you know, I think we can all agree that comparability study design for cellular product might not look like a study designed for tissue engineer product or for viral vector and what I would recommend sponsors to do is to read the guidance once, maybe twice, and go through the exercise of checking boxes. Make sure that you have all the elements that are mentioned in the guidance. Take obviously what's relevant for you. But I think that it nicely outline the elements and the considerations that one should have when developing a protocol.
I would say that comparability is complex. Like I mentioned, I'm giving a talk on Thursday on the potential for standardization and comparability. The complexity, I think, is the drive for standardization. But at the same time it's also the drive for flexibility. And although these 2 things may not necessarily conflict, they could. So we have to be careful, and I think that the guidance is very careful about it, and allows flexibility.
I'll get back to your question about the phase appropriate comparability. The guidance does mention that comparability, the rigorousness of comparability, is expected to increase with clinical development. Which is true, but it's not to say that statistical methods shouldn't be used broadly.
I think that so Jacob is going to give some talk about statistical methods in the RAPS talk. I'm not a Biostatistician, but I think I can say a few things about that. So just like you would do statistics when you submit a scientific paper to a journal, right to show that a particular gene is up regulated under a particular condition. I think that comparability for clinical development rise to this occasion, and you're expected to use statistics tools at all stages of development when you are developing a comparability protocol. Now, the rigorous increase and that could mean many things right that could, and that is driven by your product knowledge. But your clinical knowledge. So if in phase one you may not need as much number of lots. You may not need a power, you know you there are some, there is some flexibility there. It could be that you're not looking at maybe all CQAs cause maybe you don't know CQAs, but as you move on, you would expect to use a more rigorous statistical approach and to utilize all your knowledge, including clinical experience, to drive those comparability acceptance criteria and justification for these acceptance criteria.
And so you know, the guidance doesn't is not prescriptive about statistical method, and that's a good thing, but it does bring up things that industry doesn't really like in terms of the word power, the word of number of lots. I think these are challenges for industry, for sure. But the good things, it's not telling you what power, right? It's not telling you the number of lots that you need. So it does leave some flexibility there.
It does mention, though, that the 2 sample T test is not appropriate because the because of the assumption. Basically, the null hypothesis is that the 2 groups are significantly different and failing to failing to reject the null hypothesis doesn't mean that you accept it, and the guidance, I think, explain this very well. So it's not prescriptive, but it does give you some guidelines on what is okay and what is not okay. And the things that you need to consider. It does mention also 2 approaches to comparability acceptance criteria which you can whichever one but you have to integrate statistical tool. So it's talking about equivalence approach and mentioned specifically TOST in which you are testing the or you establishing the largest acceptable difference between the pre and post change means. And that is driven by your clinical knowledge, scientific knowledge, right. It's not a statistical parameter. And then the other approach is quality ranges, which is, it's a range also based on your clinical and scientific knowledge of what would be of I guess, where the range in which safety and effectiveness are not adversely impacted. So these are 2 different approaches, but both requires a statistical method, and the statistical method is determined by the sponsor based on the toolbox and based on justification that you have to provide to FDA.
So that is kind of what I can say regarding the specific questions that you ask, Katie.
Katy Spink: Great! Thank you, Tal.
Katy Spink: Don, I'm wondering if you would like to comment next. I know you were part of an ARM working group that commented on the guidance. Anything you would add to what Tal just said?
Donald Fink: So I mean, I think Tal accurately assessed the key elements. I think when they go through the ARM, the ARM working group was over 60 or 70 members working on it so represented a broad swath across the field of interest, and in addition to some of these comments. I think where the rub is, I think what one sentence that catches everyone attention is says, we understand that you are encouraged to engineer manufacturing changes that improve the overall yield and efficiency of manufacturing and the quality of the product. And then that same statement says, but if you make too, if it improves it too much and makes it too good, you may have a different product, and that kind of rubs around? People say, well, then, why would we? What would be our? Why would we innovate? Why would we go into to make these manufacturing changes if we improve it to a point where now we have a new product, and we have to kind of go back out and start in a different way. So there's a little bit of concern about that. I think there's also elements about where are we going to get enough material from what parts of our development history to be able to bring to bear on an analytical exercise. And for me, having listened to the conversations.
I think there's a little bit of a disconnect. I mean, if you start with comparability back when the first kind of you know, guidance is for coming out. There was the term highly similar. Well, highly similar, has disappeared pretty much from any language, and that's no longer highly similar. It's now equivalents testing. If you want, sufficiency of a comparability exercise on a statistical method, and I think the field recognizes, particularly in rare disease space and limited number of subjects and evolution's pretty rapidly along a manufacturing schema. There's just not going to be the tools or the capability to generate a statistically significant statistically, a valuable package. So I think that's another concern that was there. And I think Tal mentioned on that. It's not prescriptive, it's guidance it provides opportunity.
But I think also the guidance to be fair needs to highlight in addition, may you may need more than just a numbers game, and they do mention that. And they do reference it. But you may need clinical data, pre-clinical data to support, and maybe even additional clinical data. So I think those are some of the themes that kind of crept up there was over 78 comments. So it was pretty voluminous in its assessment. But I'll leave it with those additional observations.
Katy Spink: Great. So we have a question that came in from the audience on which is with the with the 2. One sided test procedure, you need to have a reasonable idea of what limits to pick in order to apply that statistical approach in your opinion, is it acceptable to start with a broad scientific estimate for a range in phase one, and then modify that estimate as more data comes in, I'm going to ask Tal to comment on that place.
Tal Salz: Yeah, I think in general, yes, I mean, I think the devil is in the detail, right? I mean, what is what is broad right? That would be a review issue. But yes, of course we don't expect you to have that product knowledge and clinical knowledge in phase one you just haven't collected enough clinical data to say that. So I think, using the tools that you have and spend some time on justification and explaining the FDA why you establish the equivalence range the way that you establish it, would work in your benefit. I think it's those submissions where there's just a number, and there's no explanation of how you come up with a number that is, that are very deficient and cause a lot of back and forth with the FDA. So yeah.
Katy Spink: Thanks. Any more comments from the panel on comparability before we move on to another topic of interest?
Heath Coats: Katie, yeah, just a quick logistics aspect of comparability in general. And again, just to remind the audience, from a from the logistics side. You know, as your product manufacturing process matures and or assays that help you analyze that product, we you want to try to make sure that if possible, you can keep retain samples from some of your early phase production batches in the event that if you're manufacturing process changes, or even your assays change, then you'll have that retain sample pool to hopefully show more of a direct comparability.
Katy Spink: Great, thank you, Heath.
Discussion on Vertex's EXSL AdCom
Katy Spink: So another topic that was a great interest to our registrants was the key takeaways from the EXSL outcome, and I know Brent shared a few brief thoughts on that topic. But, we had a quite a robust discussion going on during the AdCom, as we were watching it together. So I know our panel has a lot more to say about that. So maybe, Brent, I'll ask you to expand on your comments a bit first and then invite the rest of the panel to chime in.
Brent Morse: Sure. Yeah. So I think one of the questions that was put before the panel was, you know, what is the what is the appropriate balance, you know. So how far should we go to test, and there was actually a bit of a debate on the panel as to you know, should we? Should we? Should we be asking the sponsor to exhaustively test or test more significant number of patients or drug product intended for patients and follow them patients more closely. And the I think the question becomes, you know, what are you going to get out of that information?
And also how practical of an approach is it so yes, you could. I suppose you could some subject all the panel or all the samples to whole genome sequencing? Then you have to be concerned. Well, what depth are we going for? How much is this going to cost? Because at the depths you might need to have a meaningful analysis that this might be impractical and take a long time and not be cost effective. I think the other thing that comes up is, you know, for future developers is thinking about how their particular editing strategy what their approach are taking, how that, how that fits into thinking about the justification for how much, how many samples to test. So you know, as I mentioned earlier, your guide may not be as clean as the Vertex guide, you may have different types of off targets to assess. So I think the again the devil will be in the details there, and it's you're really going to have to take a step by step approach, thinking how clean are my their agents I'm using, what are their what other impurity profiles. And how might that affect my off target assessment? So a lot a lot of details here to consider as you as you move along. So.
Donald Fink: Hey, and, Brent, this is Don. So what I thought was so is I looked at it, cause I'm not a gene jockey per se, and I defer to your expertise here. But I what I loved about it. I thought there was a really nice framing a framework about, you know, sort of 3 levels of approach to be taken. And this is while the data can be different in each case you start in silico, you then identify, and then you verify. And I thought that was an excellent approach that could anybody could use in the genome editing space as sort of a guide is what FDA is thinking. What? How would you respond to that? Did you think that was right?
Brent Morse: Yeah, I agree. I mean I think they they did an excellent job of laying it out in crafting their argument. I don't I? And I think they were fairly persuasive that you know the numbers may, in their case the numbers may appear like they're not that significant. But if you, if you actually, if you actually look at what they presented and what they could hope to get by additional analysis. I think their argument was spot on there. There's not really going to be you know, much additional benefit for doing additional testing. I think that's true.
Donald Fink: I think the argument right. Was it. The incidence of this is so low. Even if it were to occur, you'd have to do thousands of patients to maybe find one event, and you know. So that was something that made sense to me. Yeah.
FDA Staffing and Bandwidth Challenges
Katy Spink: So I think another very timely topic is one that was touched on by Kim and her introductory comments, and I think, perhaps a bit by Heath as well. We've certainly been seeing Dark Horse and representing our clients in front of FDA. Some challenges that have come up as a result of bandwidth constraints at the agency. And Kim, I know you mentioned the staffing. But I'm wondering if our panel can comment a little bit on whether we're seeing any signs of improvement there. And you know how we feel that the FDA staffing is impacting the day to day that our clients see both on review as well as on inspection related topics.
Kim Benton: I think there's 2 points we've heard, and the agencies heard loud and clear, and we hear it at every meeting that stakeholders did not do not like written response only meetings. And so has that decreased. Have we seen that and also with interact meetings since it became a formal PDUFA meeting last year have is the rate of granting interacts. Has that changed? Do we know that yet?
And also how stakeholders have used the new request for clarification procedure that can be used after any meeting to request FDA to if they don't. They do not understand FDA's written response, or even response, live in a meeting? They can ask FDA, really, what does it mean? Do you really? Did you really mean X. If that's the response they got, so I'd like to hear from the panel what they've seen with stakeholders.
Kevin Whittlesey: Maybe Kevin, in the pre-clinical space, you may be the, you know, a great person to take the lead on this. Sure thanks, Kim. Yeah, I so I so exactly as you said I. I'm not sure if we have enough data yet regarding frequency of interacts to be able to comment on that. I mean, we know that roughly, a third of requests succeed in having meetings granted whether or not that's going to shift over time, I think, is to be determined. But certainly very encouragingly, we are seeing pre-IND zoom telephonic meetings returning that, whereas, especially during COVID, they were all written responses only so we have even very just very recently seen clients successfully have actual pre-IND meetings, and that's incredibly valuable, because very often the tone of those preliminary written responses that you get seem very finite and closed. And here's what we expect. But then, when you're able to actually have the conversation with the agency there. Can you usually understand a lot more nuance, and have the opportunity to discuss it with them and say, Well, hey? What if we? What if we do this, might this approach be feasible, and can very often come yield a much better outcome for sponsors? So I think that's really going to benefit sponsors a lot in terms of their IND enabling programs to see, to see, return to, to telephone calls for pre-INDs.
Katy Spink: Great. Thank you, Kevin Keith. Maybe I can ask you to comment on the DMPQ side of the equation.
Heath Coats: Sure, right in my perspective again in the 4 years since I've been out, you know I know that you know my colleagues in DMPQ are still severely overworked with still recovering and digging out from all the EUA submissions that came in during during the pandemic.
I know that there also have been a number of experienced very well, experienced inspectors and reviewers that have actually left DMPQ as well, and I think that, coupled with reorganization inside of DMPQ to, you know, to meet the needs of their internal customers like OTP and the other offices that DMPQ supports. You know they now have 3 branches in DMPQ which, again, the individuals that are acting as team leads and branch chiefs used to be the experienced inspectors, and so it is a challenge for them to, you know, to train new staff. I know they have a number of positions that are currently open as well to fill. So I think again, it's just the it's just the bandwidth they're just. They're spread very thin and can't necessarily take the time on a submission in a full compliance deep dive that they used to be able to with the experience and bandwidth they had.
Katy Spink: Okay, terrific.
Sarepta's Accelerated Approval and Failed Confirmatory Trial
Katy Spink: And we've only got about 5 minutes left. But there's definitely one more topic I want to make sure that we get to. Because it's been in the news a lot. So for the Sarepta product we had the AdCom in May we had approval in June, and then, as was noted earlier, we had a recent announcement that the confirmatory trial failed the primary endpoint.
I'd love to have our panel comment on. You know what they think this might mean for the field is ELEVIDYS likely to be withdrawn from the market. Do we think the FDA's going to be more conservative about accelerated approvals going forward, or really any other insights you'd like to share.
Donald Fink: Where should we start? Maybe maybe on 2 points, and I'll start by simply saying, I don't think this is evidence for withdrawing accelerated approval. I think that you know, at worst case it's do another confirmatory trial. Now that is, here's where the can of worms comes open, and this was referenced at the advisory community. They were even concerned about completing this ongoing, well enrolled confirmatory trial, provide data. So just to go back and say, I'll do another confirmatory. There's no timeline on that that I can see. You've got competing products also trying to get their product in the same space.
So I had. I had this kind of bizarre thought. I mean, what could be another reason to withdraw, forget that they failed the confirmatory. What could be another cogent reason? And that could be well if we continue indefinitely to allow this accelerated approval to stay in place without moving to a full license. How is that going to impede development from other innovators who have to compete in the space for patients and who are not able to do so because they are like, Well, I there's an approved product out there. That's what I want. I don't want to be enrolled. So that's sort of a maybe a backdoor outside way to put additional pressure on the accelerated approval reconsideration. But I don't. I don't see that changing anytime soon.
Kim Benton: Yeah, I haven't seen any signals yet, either. I haven't heard any commentary. I do think I saw a quote from Doctor Marks talk after that press release about Sarepta study, but you know his strong support for using accelerated approval for rare disease. I haven't heard a change there. I also not quite sure how to interpret some of the things in Sarepta's press release. Have they submitted? They indicated that they were going to submit for an expansion of their label or for either for older age groups or for to get, you know, full approval based on the data they thought they saw positive signal on secondary endpoints.
Limited in the press, release what I saw looked like a fraction of a second improvement on a couple of endpoints. But that's not the full data, you know. And so they have to submit an efficacy supplement for either of the things they're trying to do if they're trying to expand the age group or full approval. So I don't know if that has happened. That would be a priority review clocks. I think that's a 6 month clock. So I expect we'll see a press release if they've submitted for that be definitive, and then we'll, you know. Unfortunately, again, it's that we're going to have to wait and see what happens with other projects, with other products about accelerated approval. But I don't. I don't think this is necessarily going to hinder accelerated approval and cell and gene therapy based on, you know many comments by Peter Marks.
Donald Fink: Just before we go with. Leave this, Kim. I just want to say I was surprised to be fair that with all of this going on in the background, knowing this confirmatory trial could fail. The agency did not have a statement that it could release within days of saying, Here's what might happen next. I just was surprised by that you would think this is still a vacuum, and you they could have at least said something. I do. You find it, I mean, on your experience internally, policy that surprise you in any way.
Kim Benton: Now, I didn't expect a statement to come out. I don't think there have been other statements, because it. It has to. I mean, there is a process, if agency. If the agency intends to move toward removing accelerated approval, you know there's the process goes through a notification of the sponsor hearing all that otherwise any other discussions with the sponsor. You know FDA does not comment on those, so I wasn't surprised by that.
Closing Remarks
Katy Spink: All right great. Well, let me just hand it back to Oli to close this out because we are out of time. But thank you to our panel for a really thoughtful and insightful discussion. I certainly learned a lot.
Oliver Ball: Thank you, Katie, and thank you panelists. This is really interesting. If there are any topics that anybody in the audience would be interested in discussing further, we're always open to talking one on one just for a quick reminder that this webinar will be available for on demand viewing. So if you have colleagues who would have liked to watch it today, but haven't been able to join live, then you'll be able to find information about how to access that by email following this webinar as well as on our websites.
And also as a final reminder for next month's webinar on December thirteenth, with our managing partners. That will be our next session and Young Bronze Excellence series. So thank you all very much for attending today, and we look forward to seeing you at the next one.
Donald Fink: Thanks, Oli.
Heath Coats: Thanks.
Panelists: Thank you.