Lou Diamond – Medstat
Linda Golodner - NCL
Kathryn McDonald – Stanford
William Sullivan - CMS
Larry Bostian - NCL
Helen Wu - NQF
Mary Jo Goolsby – AANP
Francesca Cunningham – VA
Paula Griswold –
Rebecca Burkholder – NCL
Alison Rein – NCL
Nancy Ostrove – FDA
Jean Slutsky – AHRQ
Jeremy Grimshaw – OHRI
Laurie Feinberg – CMS
Susan Raetzman – AARP
Allen Vaida - ISMP
Objective of themeeting
What is the nature of the group?
What is the nature of the challenge?
Subjects for Consideration:
Inpatient / outpatient
OTC / prescription
System / patients
Communication and education
What would a clearinghouse look like?
How would we operationalize such an initiative?
What is the scale of the initiative from a resource point of view, and how would funding be achieved?
What is going to be coming out of the meeting?
A document describing the various dimensions of the project, with some description of funding requirements
Will then need to identify sources for securing the funding
Guidance from group about funding sources and how to proceed will be solicited
What is specific about the gap that we are trying to fill?
No formal gap analysis to inform the discussion, but hope to get some of that from the group
Should the focus be on high-risk scenarios?
Subset of the population?
Are high risk situations fully defined?
Do we know where the focus will be?
Will have to touch on those issues today, as we try to operationalize the clearinghouse
Taking the 4 initiatives, action is more for consumers, but change is more for systems
Yes, but the clearinghouse could be for both
Good to know / keep in mind what the problem is, so that the solution will match the problem – might not wind up where we want to be. Also, whenever there is a process, want to outline the process – so, where along the line is the “big payoff?” Is it at the prescription level?
Where are the payoffs that have the largest potential benefit?
Coalition did go through the process of defining the problem of medication use in the elderly. Did not go through medication process steps to identify where the majority of the problems were occurring – done on an interactive basis, but not formalized. Purpose of today’s meeting is to define scope of the clearinghouse. It might be that, at the end of today, we need to do some of that work to move forward.
Background – AHRQ Experience with CPG Clearinghouse
Clinical Practice Guideline Clearinghouse Discussion – focus of overuse versus underuse.
Mid 90s – feeling that there should be a clearinghouse
What would such a thing look like?
Wanted to identify why there was a clamor to centralize standards – clinical practice guidelines
Form of the clearinghouse should be virtual (web based):
· Easily updated
· Not paper based
Was developed on a platform that was web enabled
Formed coalition to get buy-in from numerous guideline-issuing organizations (AMA and AAHP)
in place for 5 years, and guided decisions about scope and i
Wanted to insert directive guidance so that updating of guidelines would be considered - Maximum limit of 5 years
Have seen a shift in the content of the clearinghouse from new to updated guidelines
Scope of clearinghouse:
· Set priority list (similar to IOM)
· Large impact on public programs
Abstracted same fields to enable comparison across guidelines
· In similar disease areas
National Quality Measures Clearinghouse
· Houses national healthcare disparities report
· Can search on contents of quality tools
Mature versus new?
· 250,000 visits per month for a mature clearinghouse
· NQM clearinghouse (new) – 4 to 6 thousand visits per day
Have created PDA downloads and real-time feeds to facilitate use
How do individual consumers know about/access these clearinghouses?
Free and open to the public, but no consumer-specific documents
Quality Tools: only recently live on the web, but plan campaign
Have had 4 “customer satisfaction” surveys
High level of satisfaction – 80-90%
Has there been any thought to survey the target population?
Penetration is still very low, but hard to capture
Weekly email sign up that issues notices – goes to about 26,000 people, but also gets distributed beyond that.
Do the other 2 clearinghouses have partners?
No, just AHRQ
one of the clearinghouses has a panel of experts who meet to advise
developed a nome
How many staff were required to launch clearinghouse? How many to maintain?
She was the staff for development/design
Was conceived by one director of AHRQ and developed by another
Jean developed scope of work and had contractors build system platform
Worked with national library of medicine
Required medical librarians and physicians to work on technical staff
Have now developed links to reference other sources
Now have “wish list” of enhancements to implement as budget permits
Defining the scope of the clearinghouse - Discussion
One lesson is that we need to recognize that clinical practice guideline (CPG) is in fact a definable product – has some level of boundaries – synthesized literature that points to what “should be.”
Can be defined along a number of dimensions.
· What we may be getting into could be much less definable
· Iterative process that changes over time
What is a quality tool?
Have established criteria that likely will change over time
What is CPG scale?
1.25 million (roughly)
Most people don’t understand that AHRQ is an incredible organization – does all of this work with fewer than 300 people. Do a lot with little resources
Need to define parameters
Want to raise 2 issues: Crucial decision points
Evidence base or
o Preference is to look at more limited sphere to ensure that we “know it works”
o Need to draw boundaries to optimize integrity of the clearinghouse
o How much/little do we want to constrain?
o Can think of arguments for both, but want to make sure that they are considered
back to AHRQ report, how many had to meet evidentiary i
How much should be explicitly evidence driven?
Do search out guidelines in AHRQ clearinghouse
Only evaluated if there was evidence that it worked.
There is a lot we can do, but there is a real difference between clinical and practice guidelines
Running into several similar issues:
· What is a practice?
· What audience is targeted?
· How do we capture what’s good?
Difficult issue of whether you want to think about “best” practices – Once you put something is in your clearinghouse, people think that it has been endorsed by you
It is, therefore, very important to define a priori
what is i
Might be important to have 2 categories:
· One that is for “good” practices
· Another that is for practices in the “test phase”
Would help organizations review progress and synthesize information from others before you get to a “stamp of approval”
What does Quality Healthcare.org do?
Don’t want to give people the impression that there is evidence, even when there is not?
Must find some way to convey level of evidence
Can you declare it?
May not get the people in the middle – get comments from those who either love it or hate it.
Need to have some parameters, but don’t want to stifle innovation in the field.
Want to set a floor, but not set the bar too high.
Documents say “safe” not “best”
Really have stayed away from using the “best” term. Want to ensure use of safe practices, but that does not mean it’s the “best” practice
Evidence about hand washing, but also evidence about getting people to wash their hands – how do you get it to happen
Speaks to the nebulous nature of considering a “practice” as opposed to a clinical guideline
Struggled with what the practice trying to achieve? What is the target?
This automatically creates some structure
If you ask for that kind of pairing, you define the structure in a more meaningful way
What percent of guidelines is submitted versus called for?
95% submitted and 5% called for
Expects the reverse for different systems.
CPG clearinghouse is, in large part, a passive set of activities – pulls it together and then lets it sit.
Depends on how you describe passive.
Have created computerized clinical decision support list serve – work to get information from clearinghouse into their systems in “real time” (XML, PDA, RSS)
What are the copy write provisions?
Full text by developers
Summaries by AHRQ – request use of citation
Route suggested by LD has implications.
Don’t know yet whether many of the practices in use are best or safe.
For the surveys of practices, can have people passively look for those. But if you have a more critical need, you can be more active in providing that information
Struggling with idea of having two-tiered systems: even if you have different levels of evidence, it makes sense. Need to think hard about what we’re trying to accomplish.
AHRQ was really trying to push the field, but that might not be the case for development of safe practice guidelines? Are we trying to figure out what is less versus more promising?
Individual experience might open the door to very product oriented interventions, which may lead down a more commercial path.
Larger group of coalition has discussed the issue of pushing the field versus defining what is known. The group (coalition) really wants to facilitate change, so managing that expectation is a challenge – very complex
What’s hard with safe practices is not what to do, but how to get people to do it – what are the implementation challenges?
o Helpful tips
Critical point, because now there is a direct link to implementation strategies from clearinghouse
Can you take the techniques and get people to actualize them – moving toward higher level of knowledge – why do you think you know what you know?
Agrees with PG. Biggest challenge for them is implementation. The VA is still a diverse system, so implementation in one area may not be as advantageous in another area – speaks to need to target evaluation to relevant area or population. Need to determine how best to implement AND how best to measure.
Wants to answer generic question about what the scope of the clearinghouse should be. Please write it down and we will capture on the screen.
Are we talking about scope or purpose?
Start with purpose and then get to the scope
Are we in agreement about who this is supposed to be targeted to?
This is a piece of the broader question of purpose.
Without knowing size of universe, ideally would like this to identify approaches that exist for improving safe use of medicine in outpatients
To convey what we know about:
Adoption of practice
And that the level of evidence is communicated
Impossible to imagine a situation in which consumer would NOT be audience, but obviously would involve providers and pharmacists – full team
Not a lot to add now, but the methods will be almost as important as the target.
Might be several different ways to communicate proper way to do something.
May be methods that can be generalized.
Information resource for stakeholders, consumers, providers, industry
Develop educational initiatives targeted at stakeholder groups
Develop communication/dissemination mechanisms
Accessible, centralized place to locate information on established and evolving information in safe and effective use of pharmaceuticals in delivery of care
Easily accessible source of health care practices that could maximize safe and effective medication use by elderly at-risk populations
Primary audience is health care providers, with patients as secondary target to inform them about practices that HCP should be implementing.
Audience has to be practitioners
Consumers are important, but want to encourage system change
Focus should be on change:
Should look at risk management pieces, and other types of things that are change oriented.
Looking at base of the system that will inform next steps (e.g., education)
Agrees with a lot of what was said
Wants an audience of consumers as well.
Wants to accomplish the purpose of shared learning and virtual exchange of ideas
Creating that opportunity may accelerate improvement.
Clearinghouse will provide guidance by serving as source of safe and appropriate use of medications commonly prescribed to elderly population
Wants to focus on multiple audiences: Providers, Elderly consumers
Want to identify safe practices for high-risk populations
Look at drug-drug interactions
Duplicate therapies – especially where there are multiple care providers
Change in therapy/dose adjustment – problems of how to guide practitioners/patients on how to do this safely
Centralized resource with evidence based information about how to provide safe care
Multiple audiences with varying communication tools
Why: consumer misuse of medication
Who: providers, at least initially
Consumers would be targeted at some point in the future
How: centralized resource – not guidance per se, but just another resource (passive as opposed to motivated to achieve change)
Audience: those who want to “fix” safety gaps – those who have seen a failure and are empowered to make change
Distinction of providers versus consumers is more relevant if you think in terms of groups able to get information out there
Purpose: Provide information to help them make changes – must have information nuggets that are: sorted, real-time, synthesized, and structured
Would be market driven in thinking about design and communication strategy
Agrees with concept of marketing approach
KM input has brought in a different perspective
Can be a long-term process of development of processes that will help a range of stakeholders
Role for Federal level of policy making
May get some key questions from consumer perspective
Let’s say what we want to achieve, and not specify target audience at this point. Scope can always be changed, but purpose is more static
Comments on scope of clearinghouse
Additional comments about project scope/description?
Audience: Thinks that there is a role for consumers or consumer organizations.
Not just because they are target of behavior change, but also because they need to know what to expect from health care providers.
Agree that there could be a role for information in the clearinghouse that contributes to safe and appropriate use – not necessarily a practice, but useful information
Sense that “broad” is ok
Has to deal with multiple points in the system
(eg., Provider, manufacturer) Need to list solvable problems, and then methods for doing so (Matrix)
Like multi audience and matrix notion
Want evidence based system (with synthesis), but allow for information-sharing forum
synthesis a component – no, trying to i
Every time a new study is introduced, the synthesis and guideline may need to be updated
Not opposed to availability of individual studies to lay public, despite potential for over reaction
Do we want best practices on individual pharmaceutical use, or should it cut across diseases? How do we organize the system:
· Where is it implemented (where in intervention chain)
· By disease
· By classification of pharmaceutical
Also agrees that principles of systematic review should be used to aggregate what we know
How do you slice the cake?
Too narrow: no studies or single studies, which diminishes evidence base
Too broad: not useful
Agrees with purpose statement…
Given that, 3 major issues:
· Focus on HCP as primary, important that it be translated in such a way that consumers can understand what to expect
Must be some way
· Need to address needs of target audience
o Don’t really ask target population what they want
Wants to refocus on “safe” not “best” practices
Wants to focus on previously identified stakeholders – higher evidence standard for consumer level access of information
Should be evidence basis, but so many things are not in the literature. May be in experiential phase, but not written up in peer review.
Would want to see those practices i
Want implementation tips – what makes something successful, more cost effective, etc.
Has concern that intervention evaluations don’t disaggregate idea from management – doesn’t necessarily mean that the idea is bad.
Often times, implementations fail because of the system barriers. Clearinghouse role can be to identify those barriers
Example of penicillin as “failed intervention” because no one wanted to take the new drug – be a guinea pig
Want access to available evidence in a systematic and appropriate way.
Many things are in progress. Could identify projects as such so that information could be shared
Want broad scope, but think that there needs to be a starting point – small chunk
Want to keep it fairly broad
May miss key pieces of
information if you start off with too narrow a focus – elderly, high-risk,
etc…Available studies may i
Inpatient versus outpatient?
Can focus on one, but can use information more broadly where it applies. May be both.
Interesting thing about quality...
How do you know if it’s a quality hospital?
Will the deliverable be a web-based tool – yes.
4th dimension of population in a more general way
Thinks of food side:
where are the alerts that consumers would need? There are critical points when you need to
take a certain action – those should be identified and i
Operational model of the clearinghouse
Good discussion of purpose and scope
Doesn’t see a great deal of discrepancy
Will share comments and extract synthesis to come up with one coherent statement of purpose and scope. Will circulate for feedback and discuss as necessary. Will be an iterative process.
What is the operational model?
1) Public - private sector partnership
c. Buy in
2) Criteria for operational confidence
Buy-in – also need to ensure that it is used. Want to test out various models to make sure
that it is known about and used. This is
enhanced by participation in design process.
Can’t rely on funding from HCP groups, but may be able to collaborate with provider group that would be most interested in contribution and use. Want to be perceived as a valuable resource. Also need pharmacy professionals.
Is there a reason why pharmacy groups or practices are not present? Were not able to make it to this meeting; not an intended exclusion as both groups are interested.
Should this be sitting within the public sector?
National Patient Safety Agency in the
David Cousins – follow up to learn more about the initiative
What are the criteria for housing?
-conflicts of interest
National Quality Forum as possible place to house it
Housing should be determined in part by funding source
Money is the message
Who would be interested?
Can process of synthesis be outsourced?
Only if there is a fundamental understanding about why synthesis is important and critical to partnership
· Content Expertise
· Methodological Expertise
· Technical Expertise
Want organizational expertise – real world implementation skills
Would clearinghouse rely on technical expert panel?
On one level, it is only as good as the panel you convene and the ability of the staff to implement panel recommendations.
There are a series of academic institutions that may have operational competencies to house the clearinghouse (CERTs)
Is the resource open and free?
Capabilities statement of organization to do the work versus governance of who would oversee the work process.
AHRQ came up with requirements, and then awarded contract based on ability to perform tasks
Governance is more critical to specify a this point – different models
Could SOS Rx serve as governing body?
Leapfrog example – wanted someone to run a site, but didn’t have funding so it is still hanging in the balance.
SOS Rx is volunteer organization – not an entity.
NCL is spearheading activity of coalition
Need to define what the initiative could be, at some level more specific than what we have currently.
SOS Rx could not house clearinghouse, despite funding. Want to find “home” for the clearinghouse and determine funding needs.
Want to compile enough detail to pull together a funding request document
Governance piece –
Private, government, foundation, or some combination
What is the organizational/business model?
Need funding in perpetuity, not just initial seed money.
Task in first RFP – turn fully government-funded process into privately funded entity
Scrapped the idea, as it was impossible to see a business plan that didn’t have access fees, which ran counter to mission
Want non-profit group to provide credibility that is not biased toward one sector.
Entity with all these competencies managing the CH, and governance by expert panel
If we were to put together a steering committee that would define and change scope of the clearinghouse, what organizations would be on the committee? For NGC –it was health plans and AMA.
A subgroup of SOS Rx plus representation from pharmacy, academia, pharmacists
Important to disseminate information to stakeholders and keep them involved.
If we want this to be used it is important to keep all the groups engaged.
Funding, steering committee, expert panel, possibly a sub-panel with other stakeholders - these could be all part of the clearinghouse
Coordination and integration with other similar efforts
This project is focused on medication use - not only safety, elderly - not all patients, and outpatient - not inpatient.
Need to discuss other best practices related to safety
NQF has 30 safe practices covering a wide range of safety issues, not just medication.
Summary of NQF project - Looking at prescription medications. Wanting to design safe practices targeting outpatient arena. Looking at all the safe practices in existence and sort through them to figure out scope and criteria. Commissioning two papers and will hold workshop in the fall with all stakeholders. Will look to see if there is enough information out there to endorse a set of outpatient safe practices.
The project is similar to this clearinghouse idea. Will determine after information is gathered if there is enough to proceed forward.
Have issued a call for practices.
Timing for this project?
Within this year
If the science is there will proceed forward. Workshop is in the fall. Looking at practices part of general population, but will look at those practices effective with low-lit population.
What is different from SOS Rx CH?
Would it be safe to say NQF is doing this?
As projects evolve, need to ask if scopes are different? End goal – CH of information with multiple end users. SOS Rx project is still evolving.
Not sure how to integrate unless you have a clearly defined project. Unable to say now that they are two different projects, or the same project
This CH – a long-term project. The evidence base is changing, and will continue to change. Not a single topic area, but a long-term update required.
Mail order pharmacy – raises patient safety issues. VA
established systems approach to track errors.
75% are mail order prescriptions – need a good system for tracking. Need to look at safe practices from every
angle. VA has used
Need understanding of what organizations are
doing. A “to do” list
coming out of this meeting which will i
NQF is distinct from the SOS Rx
Budget and process
What we have done today is framed some of the issues that need to be looked at more carefully. Will seek input from panel. SOS Rx will be seeking a partner to operationalize this. A scope document is needed, an RFP, and a review panel to select a partner.
Scale of this project is around 1 to 1.5 million annually.
We have not decided if a virtual CH or paper
Could it be both? Mostly virtual, but also some opportunity for paper for certain consumers.
What form of paper? Need to find a way to distribute.
Need to determine the model of the
Developed communication program “Think it Through” and each group determined how to disseminate and use it.
Were any organizations approached about housing such a CH? Such as FDA, AHRQ, CMS.
Discussing a generic process regardless of who will fund.
On budget, depends on roll out of
Attempting to give a sense of the scale of the project.
There must be budgets for different stages of the
What is the point below that this project is not worth doing?
Need to give the SOS Rx Coalition a sense of the scale of this project.
Depends on functionality of the product.
With AHRQ, we developed a CD on what the NGC would look like, showed it around the country, and that is how they eventually got partners on board.
Possible funding for a prototype
Maybe funding for a feasibility study, before funding for an actual CH
Multiple deliverables in the CH
NQF reached consensus on what there project should be. We are not doing that?
This is not attempt to take practices to adopt as best practices and standards. But we will describe them in terms of their evidence base.
What parts of AHRQ report would be useful?
Each chaplet was a practice that described the
effectiveness of the practice. Could envision that each chaplet would be part of the
Did NQF initial look at literature show that there is not much evidence-based information on the outpatient practices?
There is a lot of evidence out there, but need to review the quality of the evidence.
Costs related to the work needed to maintain the system.
The end point of NQF is to land on a consensus standard. Evidence needs to be high. We have an obligation to use the best evidence. Magnitude of what we are trying to do is much greater. But there is a need to focus.
The group that manages the clearinghouse must be competent – they need to possess knowledge of the content.
Important to make decisions on process of reviewing studies, how to search for them, synthesizing the studies. The scale of this project will depend on how you go about this.
Don’t want to modify your approach later. It is important to set out the process you will be using first.
How much of the universe does the CH need to cover? It is most important to capture all the practices, or to focus on a few and do it well? Important to decide what the process will be.
Some of these decisions need to made
as we define the scope of the
If you need to prioritize, more important to contain all practices, then to cover fewer and be more robust on coverage
Choice of design involves a trade off on coverage of
practices to what type of evidence you want to i
The setting where safe practices are implemented is important and relates to effectiveness.
How interactive does CH want to be? Can be very time consuming to answer questions, especially from consumers.
What is staff time required for NGC?
Some of these questions are harder to answers because we don’t have the information that matters most.
The issue of what is the universe? Could we slice it by type of intervention, etc?
It would be helpful if we could get some of the information before we go down the entire path.
Patient safety area is new enough, might be helpful to
get a broad smattering of areas. CH can
keep adding information to it. The
clearinghouse would not have to cover all practices, but each practice that is
One approach is go where the evidence is. What we will need to grapple with is that some of these decisions need to be made before the CH is operational. Or, need for CH once operational, to continue to address issues.
Maintain a registry of practices on safe use of medications, others could access as well. Build a synthesis based on needs of stakeholders. The universe is not so huge that we can’t identify the studies on safe medication use.
Wagner framework could be used.
The more you could specify up front the less expensive a project will be. When changes are made to the basic design, it becomes very expensive.
Ongoing operational costs are estimated to be in the range of 1.5 million.
Should consider a pilot feasibility study – 1)a high level prototype mock up, OR 2) some exploratory work to look at this project. This would cost over $200,000.
The CH would not be done on a fee for service basis. We are left with listing potential funding sources. Private sector – foundations, but it is unlikely that RWJ and Markle would fund an ongoing project. But might fund feasibility study.
Consider who will ultimately house the
Is it conceivable that FDA would house this?
Area that FDA is interested in, but FDA has not housed something it has not created. May not be high probability if the project is also funded from the outside. And FDA would not fund as an outside project.
AHRQ has very few discretionary funds right now.
IOM study has a funded mandate for safe use of medications.
Have been able to commingle private and federal funds at AHRQ.
If IOM mandates study, could be more funds available.
18 month IOM study – a longer time frame.
Coalition can set forth what they think about funding, if they think government should fund this, then should publicize that. Projects fully funded by the government must meet certain criteria.
Will IOM study, and 18 month time frame, delay other government funding?
Delay we need to deal with is the appropriations cycle.
May be using 1% set aside funds.
Do the SOS Rx groups have legislative muscle?
If there is seed money already for a project, it might help to obtain additional government funds.
Some other options to consider for funding - state insurance commissioners who regulate PBMs, or drug companies, may be willing to fund some of this.
Drug companies may be willing to fund. Stay away from Pharma, but NPC may be a source
Are there issues we need to discuss – ones that we have not discussed, or issues that need further discussion?
We will send out a summary of your comments. We would ask you to edit as you wish. In addition, will construct a synthesis of the comments to answer: Purpose, scope, operational issues, coordination and integration, governance, scale of the project, and a listing of funding options with some analysis of options.
We will identify some of the key questions still open and in need of resolution before we go to next steps. Not just answering but also posing the alternatives.
There is a need for an overall framework.
April 7th – full coalition meeting. Lou will give the report to the group.
Fascinating and learned a lot
Key issues addressed, realistic in the end
Stimulating, lots of questions to answer
Long way to go.
Looks forward to the end product
Looking forward to next steps
No a territorial discussion
Learned a lot and this is a huge opportunity.
Have a ways to go
Really good project, hope there is a way to go forward.
Thanks for the robust discussion