Patient Engagement Program
ICER’s Patient Engagement Program seeks input from patients and patient advocacy organizations to gather their perspectives on the patient, family, and caregiver experience of a disease and its treatment. ICER’s approach to patient engagement is intended to provide a flexible system under which individual patients can describe their lived experiences and advocacy organizations can contribute in multiple ways depending on their organizational focus and resources.
For Patient Advocacy Organizations
Several initiatives of the Patient Engagement Program are geared specifically toward patient advocacy organizations. These initiatives, which are described below, are intended to augment a patient organization’s ability to inform ICER’s research.
- Outreach to patient groups in advance of topic announcement to provide an overview of ICER procedures and to facilitate connections with other patient leaders who have participated in an ICER review. For major therapeutic classes (i.e., immunomodulators for which ICER has performed a class review, treatments for multiple sclerosis), ICER will schedule an annual conference call or meeting to discuss the emerging pipeline of new treatments, get patient input on key priorities, and explore opportunities to gather new data on outcomes of care that are important to patients and families;
- Work with patient groups at the outset of each review to determine whether/how patient groups can contribute empirically to the economic model;
- Identify gaps in key outcomes measures and work with patient groups to identify sources of RWE and potentially develop patient survey to inform the economic model and provide additional information regarding potential other benefits or disadvantages and contextual considerations surrounding the disease and its treatment;
- At the conclusion of each review,
- Invite participating patient groups for a formal debrief on the experience
- Offer to co-write and promote a letter to FDA and other stakeholders with proposals for improving the generation of patient-relevant data as part of the drug development process.
This guide was created to help you tell us what we should know about the patient experience. Often, patient experiences are not captured in research or other evidence. That is why we need to know what’s important to you and your community. In this guide, you will see the various stages in our process, understand where we think we need your input most, and receive guidance on the types of input that can be the most influential at each stage.
What is ICER?
ICER, the Institute for Clinical and Economic Review, is a non-profit research organization that evaluates evidence on the value of medical tests, treatments and delivery system innovations. ICER was founded over 10 years ago with an ethical goal in mind: to try to provide a fair and objective analysis of evidence as the starting point for bringing all stakeholders—patients, doctors, drug makers, insurers, and others— together to seek better ways to help patients gain sustainable access to high-value care. We want to help create a health care system in which innovation is rewarded fairly, but without breaking the bank, so that all patients will be able to benefit from the very best treatments at a price they can afford.
What does ICER do?
ICER produces several evidence reports every year, each focused on a different topic. Each report includes:
- Information on the patient experience and patient views that can broaden our understanding of the value of different interventions,
- A review of evidence on how well each drug or other type of intervention works and how the risks and benefits compare with other care options, and
- An analysis of how the long- and short-term costs of the intervention line up with the added benefits for patients.
Each report is developed with input from multiple sources, including patient advocacy groups, clinical experts, and manufacturers. When a final draft is completed, each report is the subject of a public meeting, during which an independent expert committee convened by ICER discusses the report and votes on the evidence and testimony presented. These three deliberative committees are:
- The California Technology Assessment Forum (CTAF),
- The Midwest Comparative Effectiveness Public Advisory Council (Midwest CEPAC), and
- The New England Comparative Effectiveness Public Advisory Council (New England CEPAC).
Meetings are an important opportunity for all stakeholders to come together and collaborate on finding solutions to some of health care’s most challenging questions. They are also one opportunity for patients and others to provide input. Based on the report and discussion at the meeting, ICER produces recommendations for insurers, manufacturers, clinical specialty societies, and patient groups. These recommendations suggest ways that the evidence on effectiveness and value can best be applied to coverage policies, pricing and payment negotiations, patient materials, and the future research agenda to achieve the ultimate goal: all patients will be able to benefit from the very best treatments at a price they can afford.
ICER listens to patients
ICER values input from all stakeholders, especially patients and patient groups. Patients are at the core of ICER’s mission to help provide an independent source of analysis of evidence on effectiveness and value to improve the quality of care that patients receive. Without input from patients, our reports are missing critical information on how a condition affects day-to-day life, the financial and insurance challenges patients face in accessing their treatment, information about what outcomes or side effects are of most importance to patients, and much more.
We know our reports are better and more useful for everyone when they include the real-world patient perspective. But we also know that patients can be concerned about contributing to a process whose end result they cannot necessarily predict. We make sure that all stakeholders know that contributing information or advice to ICER is in no way an endorsement of our work.
Some examples of how we’ve incorporated patient perspectives include:
- Patients with psoriasis highlighted how significantly it affected their quality of life at work and school, and pointed to step therapy policies and insurance coverage as a key barrier to appropriate treatment. Our final policy implications reflected these concerns.
- Patient groups we spoke to about our review of abuse-deterrent opioid formulations highlighted the need to balance patients’ needs for pain control with the risks of opioid misuse. This concern informed the context of our report.
- For a review of treatments for atopic dermatitis, patients highlighted added burdens of the disease- such as sleep disruption—that are not well-documented in typical clinical trials.
- We worked with a multiple sclerosis patient group to field a new survey to gather information on aspects of treatments that patients found most important to them. Data on responses from over 18,000 patients was included in our report and was an important complement to clinical data that was narrowly focused on relapse rates.
|In a recent press release, the National Psoriasis Foundation’s CEO commented on their work with ICER throughout the report process:
“The National Psoriasis Foundation is very pleased with the final recommendations and in the significant role we have played to provide input which informed the final report. The final report accurately reflects the challenges of living with psoriatic disease and it recommends insurers expand the tools physicians have available to care for patients managing this complex disease over a lifetime,” said NPF CEO Randy Beranek.
How you can participate in ICER’s process
The overall report development process takes approximately eight months between topic announcement and the final revision to the report following the public meeting. Below is a figure that summarizes key points where all stakeholders, including patient groups, can contribute to report development. The pages thereafter provide additional information on each step.
- Give Open Input on a New Topic
- Comment on the Draft Scoping Document
- Comment on the Draft Evidence Report
- Register to Make Oral Comments at the Meeting
- Attend a Public Meeting
ICER sends an email to all mailing lists at the beginning of a project to announce each new topic. At the same time, ICER publicly posts the timeline for the project on the Meeting page of the ICER website. This timeline provides dates for key steps in the process. When the topic is announced, the period for Open Input begins. Topics are chosen based on a number of factors, including input from the public. Learn more about how we choose topics, or submit a suggestion for a future review.
Give Open Input on a New Topic (Weeks 1-3)
As part of ICER’s Patient Engagement Program, outreach to patient groups begins in advance of topic announcement. We provide an overview of ICER procedures and facilitate connections with other patient leaders who have participated in an ICER review. For major therapeutic classes (i.e., immunomodulators for which ICER has performed a class review, treatments for multiple sclerosis, etc.), ICER will schedule an annual conference call or meeting to discuss the emerging pipeline of new treatments, get patient input on key priorities, and explore opportunities to gather new data on outcomes of care that are important to patients and families. ICER provides early notifications to patient groups from other disease areas when it has high certainty that it will pursue an assessment pertaining to their focus.
ICER sends an email to major patient groups and all mailing lists at the beginning of a project to announce each new topic. At the same time, ICER publicly posts the timeline for the project on the meeting page of the ICER website. This timeline provides dates for key steps in the process. When the topic is announced, the period for open input begins. Topics are chosen based on a number of factors, including input from the public. Learn more about how we choose topics, or submit a suggestion for a future review.
What happens in this step?
- After topic announcement, the Open Input period runs for three weeks.
- While open input submissions are being accepted, ICER begins to develop a Draft Scoping Document
What information is most helpful?
At this stage, any information is helpful. We want to learn as much as possible from all stakeholders about the topic.
Some examples include:
- Clinical benefits that matter most to patients
- Other non-clinical benefits or disadvantages of new and existing treatment options
- Contextual considerations surrounding the disease and its treatment (i.e., high lifetime burden of illness, uncertainty about long-term risks of treatment, etc.)
- Research publications or patient-generated data to supplement clinical trial data
How do I give input?
- Provide a public comment via email to firstname.lastname@example.org or submit a response to our Patient Input Questionnaire. We provide three weeks to submit public comments (no page limits), but there is no deadline to submit a response through the Patient Input Questionnaire.
How does ICER use my input?
- ICER uses the information to help guide the scoping process and inform our initial research plan. More specifically, much of the input we receive is ultimately translated into what we call “PICOTS.” PICOTS stands for Populations (people), Interventions (treatments of interest), Comparators (other available treatments), Outcomes (benefits and harms), Timing (length of study), and Settings (where care is given). For example, if we hear from patients with rheumatoid arthritis that the treatment under review provides relief of morning joint stiffness, we will then look for specific data that reflects this “outcome,” or clinical benefit.
- In addition, patient input may also provide information about aspects of treatment or the disease that are not be described or captured in available clinical or economic data.
- All submissions are considered throughout the scoping process and are incorporated into the Revised Scoping Document.
- Information received towards the end of the Open Input Period may not be considered for the Draft Scoping Document, but will be taken into account in the revised version. Information submitted during the Open Input Period provides useful background throughout the report development process.
Draft Scoping Document
Based on independent research and conversations with clinical experts, manufacturers, policy makers, insurers, and patient groups, ICER develops a Draft Scoping Document that is posted on the website that outlines the plan of research, drugs or treatments to review, patient populations to include, and other related information. We encourage input on the specifics included in the draft scoping document.
Comment on the Draft Scoping Document (Weeks 4-6)
Based on independent research and conversations with clinical experts, manufacturers, policy makers, insurers, and patient groups, ICER develops a Draft Scoping Document that is posted on the website that outlines the plan of research, drugs or treatments to review, patient populations to include, and other related information. Specifically, ICER uses patient feedback from public comments and Patient Input Questionnaire responses to guide specific components of the Draft Scoping Document, including our PICOTS framework. As we note in the section above, we encourage input on the specifics included in the Draft Scoping Document.
What happens in this step?
- The Draft Scoping Document is posted to the website and announced by email.
- The document is open to a three-week public comment period.
What information is most helpful?
- In contrast to the Open Input Period, during this stage, we’re largely looking for feedback directly related to the content of our Draft Scoping Document, as patient input will help us revise and clarify before we release our Revised Scoping Document. This could mean input on what is included or left out of the Draft Scoping Document that is most important to patients (e.g., patient subgroups, treatments, or additional benefits to examine).
- ICER will also accept responses to the Patient Input Questionnaire. These responses are helpful throughout the review process.
- Any additional patient-related resources
- Which clinical trial data are most related to patient-centered outcomes If formatting requirements present a burden to any patient, accommodations can be made on a case-by-case basis. Please email email@example.com or call (617) 528-4013 for more information.
How do I give input?
- ICER accepts comments for three weeks.
- The deadline is listed in the posting announcement and on the website.
- Email comments to firstname.lastname@example.org or submit a response to our Patient Input Questionnaire.
- Emailed comments must be in the following format:
- Submit as an attached Word document
- Three page maximum (not including references or appendices)
- 12-point Times New Roman font
How does ICER use my input?
- ICER reviews all comments received and considers what changes need to be made to the scope before work on the report begins. For example, ICER might include an additional patient subgroup, comparator, or outcome measure. We refer to components like these as PICOTS in our scoping documents (please see the previous section for further information on PICOTS).
- Input received during the scoping phase informs ICER’s selection of outcomes measures to include and prioritize in our clinical and economic assessments. Further, public comments and conversations with patient groups inform the second section of our reports, “Patient Perspectives,” which precedes the sections describing the clinical and economic evidence.
- Written comments, including those submitted by individual patients, will be posted publicly to the ICER website with the Evidence Report.
Revised Scoping Document
After public comments close, ICER updates the scoping document based on comments and releases a Revised Scoping Document. This document also includes a brief summary of information learned through open input and public comment as well as ICER’s rationale behind which major suggestions we did and did not incorporate. The document is posted to the ICER website, along with comments received on the draft version.
Draft Evidence Report
The Draft Evidence Report includes background and context about the topic, reviews evidence, provides preliminary cost-effectiveness analyses, and summarizes how patient input was used to inform the draft. It reflects only ICER’s independent analysis of currently available information and does not make policy recommendations. The Draft Evidence Report and Draft Voting Questions are posted on the ICER website, and an announcement is sent to ICER’s mailing list.
Comment on the Draft Evidence Report and Voting Questions (Weeks 22-25)
What happens in this step?
- Based on the topic, ICER and patient groups may partner to conduct a formal survey to gather patient preferences for treatment. The results of these surveys may inform our economic model, especially when important information is missing from the clinical evidence.
- Before we publish the Draft Evidence Report, we send it out for external review to make sure that the document is accurate and thorough. We typically invite several clinicians to review the clinical and economic evidence, and ask a representative from a patient group to review the sections describing the patient experience.
- The Draft Evidence Report is posted on the website and announced by email.
- Stakeholders review and provide comment over a four-week period (this period is longer for class reviews).
- This is also the time to request to make an oral comment at the public meeting.
What information is most helpful?
- Suggestions for additional data
- Added details for context
- More explanation of additional benefits or disadvantages of a treatment
- Anything else that, from a patient perspective, the report misses. We encourage patients to comment on Section 2, “Patient Perspectives,” as well as Section 6, “Potential Other Benefits or Disadvantages and Contextual Considerations.”
How do I give input?
- The specific deadline is in the announcement and is posted on ICER’s website.
- Email comments to email@example.com. We will confirm receipt.
- Comments must be in the following format*:
*NOTE: If formatting requirements present a burden to any patient, accommodations can be made on a case-by-case basis. Please email firstname.lastname@example.org or call (617) 528-4013 for more information.
How does ICER use my input?
- ICER carefully considers all comments relevant to every section of the report.
- Based on specific feedback we receive on the patient-centered sections, Sections 2 and 6, we may revise to include additional factors or considerations important to patients (i.e., caregiver impact, comorbidities, etc.). In addition, comments related to the financial burden of a disease or treatment may also be helpful, such as specific information about out-of-pocket costs or other medical expenses.
- We also welcome patient input on other sections of the report, including the clinical end economic analyses. We encourage patients to provide as much or as little feedback as they wish. ICER considers all patient input important, no matter how much is provided.
- All changes are reflected in an updated Evidence Report that is published before the public meeting takes place.
- Written comments, including those submitted by individual patients, will be posted publicly to the ICER website with the Evidence Report and will be distributed to members of the relevant voting body prior to the public meeting.
- A document with ICER’s response to suggestions will also be posted.
Register for Oral Comment at the Public Meeting
This is also the period to request to speak at the public meeting. Patient comments provide additional important context and help inform discussion during the panel vote and policy roundtable. Each meeting includes up to 45 minutes for oral comment before the voting panel takes its votes. Comments are limited to five minutes. To request to speak, please follow the below steps:
- Send request to ICER via email email@example.com including the name, title, and organization (if applicable) of the proposed speaker.
- ICER staff will confirm receipt of the request, and reply with a link to an online conflict of interest form that must be filled out accurately to complete registration.
- Any reported conflicts of interest will be disclosed at the meeting but will not prevent participation.
Since there may be more requests than can be accommodated during the meeting, and to help provide the opportunity for a broad range of perspectives to be heard, public comment slots will only be confirmed after the deadline for requests has passed. Priority for public comment slots will be given to patients with the condition under study and subject-matter experts from the patient advocacy, clinical, and research communities.
Public commenters may not use a slide presentation or distribute materials to the panel or audience members prior to, or during, the public meeting.
Evidence Report and Revised Voting Questions
The Evidence Report includes changes made based on public comments and includes health benefit price benchmarks. This is the price for a treatment that matches with the benefits to patients that have been shown in the evidence. It is posted to ICER’s website, and an announcement is emailed to subscribers. Along with the report, a revised set of voting questions, all public comments received, and ICER’s response to comments are posted to the website. This version of the report is shared with the voting panel in advance of the public meeting.
Attend a Public Meeting (Week 30)
What happens at the meeting?
- Report authors give an overview of report findings.
- Pre-registered speakers give oral public comments.
- The evidence review group votes on the strength of evidence for the topic under review. Report authors, clinical experts, and patient representatives are available to answer questions.
- A Policy Roundtable, composed of patients, doctors, insurers, and other stakeholders convenes to discuss how the evidence applies to real-world policy and practice.
How can I participate?
- Attend: Meetings are free and open to the public. We recommend registering in advance to ensure your spot. Registration typically opens when the Draft Evidence Report is posted.
- Comment: Register to give an oral public comment using the instructions above.
- Watch: A live webcast is offered for most meetings. ICER also posts a recording of the webcast on their website within a week after the meeting. More information is posted to the meeting page of the topic you’re interested in.
- Be a member of the policy roundtable: One to two patient representatives are invited to participate in the policy roundtable, typically alongside one or two each of clinical experts, payers, and drug manufacturers.rs.
How does ICER use my input?
- ICER uses Public Meeting discussions to inform the policy implications that are included in the Final Evidence Report and Meeting Summary.
- Public commenters are invited to submit a 750-word summary of their oral comment to be included in the Final Evidence Report and Meeting Summary.
Final Evidence Report and Meeting Summary
The Final Evidence Report and Meeting Summary includes a summary of discussions, public comments, voting results, and policy statements from the roundtable discussion during the meeting. It takes about one to two weeks to publish the Final Evidence Report and Meeting Summary. The report is posted to the ICER website and announced by email.
A short, summarized version of the report, called “Report-At-A-Glance,” is posted alongside the full Final Evidence Report and Meeting Summary.
We hope this guide gave you a good overview of our process and how you can get involved. Of course, there is more to our process than just what’s here. Visit our website for more information.
Here are some helpful links:
- To find out more about how we look at the effectiveness and value of drugs, tests, or treatments, visit the How We Do Our Work page.
- To learn how organizations around the world conduct similar assessments, read this Guide to Understanding Health Technology Assessment.
Still have questions, or suggestions for improvements to this guide? Contact us at firstname.lastname@example.org. We look forward to hearing from you!