Good afternoon, everyone. We are going to go ahead and get started. Welcome to the 2019 Measures Under Consideration Pre-rulemaking Open Forum Discussion #2. So we had a session already on Thursday, and this is the exact kind of identical presentation to that. So we’re going to go ahead and get started. My name is Kim Rawlings and I am with CMS. I am the lead for the Measures Management System (MMS) and along with Helen Dollar-Maples am the lead for the pre-rulemaking process, and we’ll also hear from Vince Brown from Battelle who is the lead contractor for this task. So today we will be having all of the lines muted during the presentation, just to avoid any background noise. And then we will open up the lines at the end of the call to have any live questions. The session is being recorded, and we will be uploading it to the CMS website for future viewing within the next couple of weeks, and the slides will be made available in the next couple of days. Like I said, if you have any questions, we’ll have questions at the end of the presentation; however, just so you don’t lose your question in the middle of the presentation, please do feel free to use WebEx and submit your question through the Q&A function, and we will review all of those at the end. So we have three main goals today. One, I’m going to do a very quick overview of the pre-rulemaking cycle and kind of where we are. And then both Vince and I will talk about where we are in the process, and Vince will start to discuss what’s new for 2019 and go into a demonstration of JIRA so you can see how it works and see what the system looks like. And then we’ll conclude with a question and answer (Q&A) session. So just as a quick recap of what measures go into JIRA and what measures you should be submitting, just a good reminder that measures that are currently in use in the program do not have to go through—do not have to go on the measures under consideration (MUC) List again. They only have to go through that one time. And then once they’re in the program that’s it, the one exception being that if you have updated the measure and there are substantive changes to that measure like a change in setting, a change in population, et cetera, then you would want to put that measure back on the MUC List for consideration because there were substantive changes. Also to note that measures that were submitted to be on a prior years’ MUC List but were not acceptable, are eligible for resubmission. To save yourself some time I would definitely make sure that when you are resubmitting that measure, though, that you really look at the reasons why we weren’t able to put it on the MUC List and try and rectify that. Was it not a priority at the time and now it is? Was the measure not fully developed and now you’ve finished testing and everything and it’s good to go, you know? What were those reasons that it wasn’t accepted and how have you worked to again, make that measure better or make it more aligned with our programs, but again you are fully eligible to resubmit that measure. Again, just to note that measure specifications as we all know, they change over time. We do those evaluations yearly to make sure that our measure is still being current and so when large changes happen, the measure needs to be submitted on the MUC List again. And then just to kind of put it into a little bit of context of where we are in the process. So hopefully, you all know that JIRA is actively open now. It opened February 26th. JIRA closes for all measures June 3rd. That’s a Monday, so I highly, highly, highly recommend that you get—that you start submitting that measure into JIRA the Friday before that May 31st, the absolute latest, just because we want to make sure that you have adequate technical assistance (TA) and time, et cetera, to submit your measure in JIRA. So yes, JIRA closes on June 3rd for all programs. And then by mid-July CMS will have an internal draft of the MUC List kind of prepared, and we will use that to discuss the MUC List and give a preview of the MUC List to our federal-only stakeholders, and so all of our sister agencies like CDC, ASPE, AHRQ, et cetera. And then once they have that preview and give some initial feedback, then in late July we will actually start the MUC clearance process. It will go through CMS as well as HHS clearance, and then be published no later than December 1st. So with that, I’m going to pass it to Vince who is going to lead us in the discussion on what’s new for 2019 in JIRA. Thank you. Thank you, Kim, for that introduction. My name is Vince Brown. I work for Battelle. We are a contractor to CMS and we help them manage the measures as they come in. The purpose of today’s call is to catch any participants who were not able to be on last Thursday’s Open Forum. The outline as Kim noted is similar. We do allow ample time for questions. So if you were on last Thursday’s call and had a question that didn’t get answered or had come up with a new question, we will be glad to try to respond to that today. I do try to pitch today’s presentation to anyone who may be completely new to JIRA and never has used it before, or also to highlight for those who have been using it for the last few years, to show what has changed from the prior year. The short answer on that is if you used JIRA last year, you will not notice very much drastic change for 2019. We did add a couple of questions and tweaked a few of the screen guidance, but we’ll go over that here in the presentation. So one change for this year was the program that was previously called “EHR Incentive” has been editorially changed to now say “Promoting Interoperability.” This is for eligible hospitals (EHs) and critical access hospitals (CAHs). So if you were used to seeing “EHR Incentive” that program has a slightly different name in 2019. The other major change for this year is in the area of the healthcare priority, there are six possible choices there. And then under each priority are a series of Meaningful Measure Areas (MMAs). That’s still a required field and this year, for 2019, we’re calling it the primary priority and primary Meaningful Measure Area (MMA), but we added two new questions which are optional right after them. I’ll show them to you when we do the demonstration. You now have the option to select a secondary priority, and if you would like, a secondary Meaningful Measure Area (MMA). Some users gave us feedback that it was hard to fit their measure into a given one priority or one Meaningful Measure Area, so we’ve added this secondary for you to provide more information about your candidate measure at your option. A few of the drop-down menus have different choices, especially in the area of data sources and how the measure data will be reported back to CMS. So you’ll see slightly different choices there, again based on user feedback from 2018. We did also enhance the screen guidance in several places to remind or encourage users to use the “comments” field. If the drop-down list of choices for a given question does not include the exact terminology that your program uses or that your medical specialty, you know, wants to use, then you’re invited to use the “comments” field at the end to elaborate or clarify exactly what you mean. This will help those who are reviewing your measure to understand it better. So again, the interface will look much as it did in 2018. Some of the drop-down lists are quite long. So if you don’t see what you’re looking at as soon as you click on that drop-down, be sure to use the elevator at the right and scroll down to see all the choices. Likewise, all of the drop-downs have a choice for “other.” So again, if what you’re looking for is not on our prepopulated drop-down choose “other.” And then again go to the “comments” at the bottom and tell us what that “other” is, so that the reviewers have the best information. Every year we have caught some measure titles can be quite lengthy, and so just a reminder that that field is limited to 255 characters max. If you paste a block of text in there that’s longer than that, it will get truncated and so just be aware of that. And then again, the JIRA interface has about 70 fields or data questions on it. About half of those are required. If you’re on a required field and you don’t know the answer or don’t have the answer, please put “N/A” in there. JIRA will not let you go forward, if you leave a required field blank. So we encourage you to put “N/A” in there just as a consistent signal that that information was not available or not applicable. Many of our users have been glad to have the feature in JIRA to create and save a custom filter. You can set up a filter or a sort screen to look just the way you want it so that you’re focusing in on those measures that are of interest to your particular kind of work. So then you can save that filter and it just saves you rebuilding the search structure each time. You can save it and open it again the next time you’re in JIRA. You can also export custom reports as CSV or comma-separated value database files and that’s handy. You can take the CSV and then go into Excel or whatever other database you might like to use to do different types of analyses on the values in the database, and tied to the search and filter function you can set up a personalized dashboard that lets you again, set up the screen with one, two or more different views of the data on one dashboard. And then every time you go into JIRA, that will automatically update with any new records that meet the criteria that you set up in your search or filter session. So that concludes the slide part. Let’s change over to the screen sharing, if you don’t mind, Martin. Okay, thank you. So I just put this splash screen up here. This is the Office of the National Coordinator for Health IT (ONC), and I just put the sign-in screen up there to remind you that it is limited access, so you first have to have an ONC JIRA account. And then in our user guide which is on the pre-rulemaking website, it has the guidance there for how to apply for access to the MUC List project within JIRA. There are many, many projects in ONC JIRA. 2019 measures under consideration (MUC) is one of those. So once you have access, then you will login. Notice mine defaults to 2017. If you’re signing in new for the first time this year, it will I’m thinking default to 2019, but I just have to remember to go up here to “projects” and select the current year’s project. Now it shows in the upper left corner that we are on the right one, “2019 measures under consideration.” You’ll see here that there have been no candidate measures submitted. One of our reasons for having these webinars is to encourage our stakeholders to start putting their candidate measures into JIRA, so you’re welcome to do that at any time. We look forward to seeing them. I want to—if you have a measure that you’re ready to submit, then you would go up here to the upper left. There’s a yellow “create” button, and that causes a new screen to pop up. So what we’ll do for this demonstration is walk through that interface that you see the scrollbar at the right, so that’s how you move through the questions and make sure that you’re on the right project, “2019″ and that your issue type is “measure submission.” We do have options for questions or feedback or modifying a measure that’s been submitted, but to put a new measure in, that wants to be, I call it the “green plus sign” or the “measure submission issue type.” The first question we ask is about the components. That’s the JIRA word for “programs.” So when I hit this drop-down, you’ll see the programs that you have to choose from. You can select as many as are applicable for your particular measure. Here’s the one that changed its name. It’s called “Medicare and Medicaid Promoting Interoperability.” Again, that used to be called “EHR Incentive.” It’s the same program, it just has a different name as far as I know. I will point out, too, just like last year, if you’re submitting a candidate measure under MIPS you have a decision to make between cost or quality. I think the people that are preparing and submitting cost measures know who they are, and that’s a pretty well defined set. So for most of the others, that will be the MIPS-Quality program. I’ll draw your attention to these red asterisks right after the field label. It’s kind of small, you may or may not be able to see it. You will when you get into JIRA, but that red asterisk indicates a required field, so you have to select something in that field or JIRA will not accept your record. I’ll just scroll through here and give you an idea. Again, “summary” is what JIRA calls the measure title. Description — numerator, denominator. These are specifications that I think most people are familiar with. One change we made this year. We had a field called “exclusions” and we’ve added the word “exceptions” to that title. So sometimes we get people asking, “Can we put exceptions in here as well?” The answer is yes, that field is intended to capture any subtle differences between those. I want to scroll now to the “priority.” Again, the six healthcare priorities will look just as they did last year, but one change we’ve made this year is we’re calling this required field the “primary” healthcare priority. This again is required, so to submit a measure you have to select one of these. I’ll just select “chronic disease” to begin with for an example. Once you select your priority, then your Meaningful Measure Area (MMA) which is the very next question will populate. You’ll have the Meaningful Measure Areas (MMAs) that pertain to that priority. If you change your mind, you can go back up here and say, “Oh, no, it’s really an engagement measure.” Now, I have different choices here under the Meaningful Measures. So those two, the primary ones are required. And then you’ll notice these next two, the secondary priority and the secondary Meaningful Measure Area (MMA) do not have the asterisk, so that means those are at your option. The drop-downs are set up identically to the primary, so you can have—you can select whichever secondary priority and whichever secondary Meaningful Measure Area (MMA) is appropriate. One more small change for 2019 is the NQF ID number. It had always been a four-digit identifier. The NQF or the National Quality Forum is now using a letter after the numerical ID for some measures. We most often see an “e,” for example. So if you have that “e” after your NQF ID for your measure, there is now room in JIRA for you to add whatever letter you want after the four digits. Again, we’ll scroll through and a number of these are not required fields. I always encourage users if you have the information, to take the time to put it in. What that does is it helps the people who are reviewing the measure in clearance throughout CMS or HHS, and even when it gets to the Measure Applications Partnership (MAP) reviews that happen in December and January, those reviewers, the multi-stakeholder review committees will have better information the more data that you’re able to put into JIRA. So that’s why we encourage you to fill in as many fields as possible. Here are questions on data sources, registries. How is the measure expected to be reported? If it is an eCQM, it has specific requirements for what you must attach so just look to the screen guidance there. That catches people it seems every year, so they want to make sure that you have a feasibility scorecard and Bonnie test cases for any electronic clinical quality measure or eCQM. I think most of you who are developing eCQMs are familiar with that. We’re getting down to the bottom of the screen. Here are links to prior years’ MAP recommendations reports, just to save you some time. We do require a measure steward be listed, and you’re also welcome to add a long-term measure steward. If you are the steward now but you know that that will be changing in the foreseeable future, you’re welcome to add who the long-term steward will be for that. That’s again an optional field, but it may help CMS to track down the steward in future years. Here’s the “comments” field. Again, for any one where you selected “other” or where the drop-down list was not—did not have the terms that you were looking for, the “comments” field here at the bottom is where you would specify that. You would say which field you’re talking about and then add whatever information will help elaborate or clarify on what you mean about your measure. The attachment field is where you can put a measure information form (MIF). If it’s an eCQM, you can put your Bonnie test cases. If it’s a MIPS measure, it’s required to have a peer-reviewed journal article form filled out. This is where you would attach that for any MIPS measure. You can drop files there, drag and drop, or you can use this “browse” function to go find the file wherever you have it stored. You can attach up to 10MB to any record. If you have more than 10MB and if you have extensive data, contact us. There’s information at the bottom of this that I’ll show, or you can go on the pre-rulemaking website and the user guide has where to contact the MMS Support email. Let us know and we’ll work out another way for you to get your large files over to CMS and the others who want to review that record. When you’ve populated everything, then you would hit the “create” button here at the bottom. That will cause that record to go into JIRA database. I do want to point out, too, one feature that we often get questions on. JIRA does not let you save and come back later. We wish it did, but it does not support that function, so that’s why we encourage everyone to look on the pre-rulemaking website for a blank template that has all these fields in an editable Word file format. You can do one of those for each measure. And then that’s the way for you offline to gather your data in one handy place. And then once you get into JIRA, you can copy and paste from your Word file into the JIRA field and you’re sure you’ve got all your required and optional fields covered. It saves time. It saves you to go hunt for things while you’re in JIRA, and it solves the problem of not being able to save and come back later, so a lot of people have found that to be helpful. Oh, I did want to demonstrate and what I will do is cancel this and go back to the “2018 project,” because it actually has some measures in it. What you’ll see here is a display of last year’s measures database. I wanted to show you. You can set up—let’s say we want to look up any measure dealing with cancer. So you search for that and these are—what this is showing us is now we have 48 records that have the word “cancer” appearing anywhere in there. You can also select which columns to display by using this “columns” function over at the far right, so you have many, many choices here. Typically, what I will do is type in the search the field I’m looking for and then use these checkboxes to put the columns on the screen that you want. Once you get the screen looking just the way you want, let’s say you’re going to be looking for only cancer measures this year. Then you would click “save as,” and that lets you—up here at the upper left, and that lets you save that filter. And then you’ll be able to go back and find that filter later. So, in other words, it saves you from rebuilding that search or filter structure. Likewise, the “dashboards” are up here and I’ll just show you a quick one here. I think this goes back to the year 2017. What you can do is set up a screen here that shows you the measures you want. And then you can also add a new gadget over here. They call them “gadgets.” Each little screen or subset of the dashboard is a gadget, so you can set up a different filter to appear on the right side of your screen. Some people like to be able to compare two different lists. Maybe they’re looking at long-term care hospital and inpatient rehab or something like that, and so you can have both of those displays on the same dashboard. Again, you can edit the layout. You can manage dashboards and create new ones. This is how you find one, and so then you can save your dashboard and again, come back to it again and again. Each time you go to that dashboard, it will be updated with any new measures that have been added since the last time you visited, so that’s pretty handy. I wanted to also show the “export” function. Again, going back to 2018, because it actually has measures in it. On the upper right is a button called “export.” You have various options here. The two that we use most often are CSV “current fields,” or CSV “all fields.” What the—CSV stands for comma-separated value and so that just exports the file to your downloads folder by default. The “current fields” will show you only the columns that you have, or the fields that you have on your screen at that time. “All fields” will give you every field in JIRA, so there are times when you want all fields. There are times—most of the time when I’m doing it, I set up the screen as I want it and then use “current fields,” but that’s a handy way to get data out of here to then use in Excel or some other database program. So again, that’s up here under the “export” function. I think that I will now switch away from the screen sharing and we’ll go into question and answer (Q&A) mode, if that’s all right, Martin. So Kim, do you want to describe how we’ll do the questions, or do you want me or Martin to do that? Ah, sure, I can describe it. Just basically, you know, feel free to use the Q&A box to—feature to insert any of your questions. And then once we’ve kind of worked through all of those, then we will open up the lines to see if there are any quote, unquote “live” questions. Martin very graciously put the instructions on how to submit a Q&A question, so I think we’ll give it a minute or so and see if any questions come through. There was one. This is Vince. There was one question that just came in online. It said, “Do fields other than the summary have a word limit?” These are open text fields, and most of them do not have a word limit. If they do have a word—if they do have a character limit like “summary” or “title,” it will say that in the screen guidance which is the small print right under that field. So if it does not have a character limit specified in the on-screen guidance, you can assume that there is really no limit, you know, other than the attention of your reader or something like that. One thing that we do try to put in our user guide is to be aware of—they come into JIRA in a database format, so anything having to do with tables or graphic symbols, especially things like greater than, less than or plus and minus, sometimes technical will have special symbol sets or Greek characters and so forth. Sometimes those will get lost, and so we encourage people to make the text fields as plain as they can. That tends to work better and just is less confusing for those downstream who need to review the measure. Someone asked, “Will these slides be available after the webinar?” Yes, on the CMS pre-rulemaking website which is open to the public, these slides will be available in downloadable format. And then as Kim indicated at the top we also, CMS also makes a YouTube video which is for on demand viewing at your convenience. If you have colleagues who were not able to be on either of our Open Forums, then within a few weeks, those should be also on the pre- rulemaking website in a link toward the bottom of that page, and that will have closed caption- accessible video that you can watch part or all of it at any time. It will have the audio and video of all the slides and demonstrations, as well as the closed captioning. This one may be for Kim or others who may be on from CMS. “Is there a minimum amount of data for providers required for reliability/validity testing for registry measures?” I don’t honestly know the answer to that. It would be great if the person who asked the question, if you could expand on it a little bit. And then just kind of a separate note, registry measures, like for QCDRs, just wanting to be clear. They do not go through the pre-rulemaking process. That’s a completely separate process, but really, you know, everything depends on the measure, you know. This kind of standards and factors, you know, for a process measure can be very different from an outcome measure are going to be very different from an eCQM outcome measure. So, you know, it really depends on—without more information, all I can say is that it really depends on your measure and, you know, how your testing data turned out. I will look and this is Vince. It occurs to me in the Kickoff we had a question kind of on this line, but it had to do with how many settings or something like that were required. I honestly don’t have that transcript right in front of me and can’t remember from a week ago, but when that—I’ll look to that and see, but when that video comes out or the slides with the transcript on it, you might be able to see in the question and answer (Q&A) portion from last Tuesday’s Kickoff call. There may be a question having to do with how much data is required, and I think if I remember from prior calls and prior years, the general answer is “the more the better,” but I don’t want to say more than that without being sure. The other comment that I would make on the question of how much data is required, is there is a CMS Measures Blueprint that’s published again, within the Measures Management System (MMS). That Blueprint has quite a bit of information on the testing schemes and requirements, alpha and beta testing and so forth, and how measures are documented in terms of their testing. So the Blueprint may have further guidance on that, beyond what’s in our user guide. Yeah, thank you, Vince. That’s a great point. Yeah, you know, I mean, again, you know, I can’t give—you know, there’s no magic kind of number. It all depends on your measure but, you know, when we think about kind of testing and the level that you should meet, you know, I mean, you’re using the right words. You know, it should be, you know, quote, unquote, we like to say “endorsement-ready,” you know, if not endorsed already. But, you know, we realize for various reasons that endorsement may not be possible before it goes on the MUC List and so, you know, we really do need it to be, you know, reliable and have those high levels of reliability and validity. In the Blueprint, I believe it’s Chapter 21 or maybe 22, you know, is on measure testing and, you know, has a lot of great information to detail how that works and, you know, go through alpha testing, beta testing, feasibility testing, et cetera. So that might be a great resource. Of course, if you still have questions, you can feel free to reach out to us via MMS Support and we’ll try our best to provide some more technical assistance (TA) and guidance. There is a—Kim, there is a follow-up to that question. It said, the one that the caller was asking about, “is a CQM that’s being tested through a registry,” so it may or may not be a QCDR. I don’t know. Yes, thank you for that clarification and, you know, again, you know, feel free to email us with kind of some more specifics, but I don’t know how much more I can say other than it really depends on your measure, and I would use the Blueprint as a resource. Yes, so our next session is Thursday, two days from today and it will be the Needs & Priorities session. At that call each of the programs that are subject to pre-rulemaking or the participating pre-rulemaking will have a few minutes to present their current status, the measures that they now have in the field, and also then each program lead or his or her representative will
detail the needs and priorities that they see. Again, these are referenced to the healthcare priorities and the Meaningful Measure Areas that are important to CMS and other stakeholders. So that will be Thursday from 1:00-3:00 pm EST scheduled, and there will be time for questions and answers (Q&A) at that call. Again, a reminder that JIRA will close for new measures on Monday, June 3rd. There is a link on this slide to the pre-rulemaking website. If you Google “CMS pre-rulemaking” you’ll find it and it has quite a bit of information and many, many resources on there that you can use to refer to to learn the history of a given measure, or to see what the Measure Applications Partnerships (MAP) have recommended with respect to a measure in past years, and also guidance for this year in how to get your measure information together and how to send that candidate measure specification in to CMS. So this lists the contacts — Kim Rawlings and Helen Dollar-Maples there at CMS with their emails. And then again, my name is Vince Brown. I work for Battelle, and we help measure owners, stakeholders, and measure stewards to work their way through JIRA. If questions come up, or if you hit any obstacles, we’re glad to do our best to help. We’ve referred to this submission template blank. It’s on the pre-rulemaking website. There are also files out there such as frequently asked questions (FAQs) and a JIRA user guide. The user guide is fairly detailed. I’ll draw your attention also to a quick start guide that is a two-pager. If all you want to do is get a measure in in a hurry or as fast as you can and get going, then you could refer to that two-page quick start guide and that may be handy, too, for some users. Again, there’s the link to the pre-rulemaking website in case you would want that. Kim, do you have any concluding remarks? Just thank you everyone for joining, and we look forward to seeing your measures in JIRA. Okay, I guess that concludes today’s webinar. Thank you, everyone for your attention.