Difference between revisions of "Safety-enhanced design (MU3))"

From OpenEMR Project Wiki
 
(184 intermediate revisions by 3 users not shown)
Line 1: Line 1:
= Overview =
= Overview =
:Working with Columbia HIT
:The first usability testing was completed by the 2022 Cohort of the Professional Certificate in Health Information Technology at Columbia University for the a1, a2, a5, a9, a14 certification criteria.
:An 2024 addendum to the usability study is being conducted by OpenEMR for the b11 certification criteria.


= Criteria that are testing =
= Usability Test Report =
:*a1: Computerized provider order entry (CPOE) – medications
:[[Media:OpenEMR Usability Report.pdf|OpenEMR Usability Test Report - 2022]]
:*a2: CPOE – laboratory
 
:*a5: Demographics
= UCD Process =
:*a9: Clinical decision support
:[https://nvlpubs.nist.gov/nistpubs/Legacy/IR/nistir7741.pdf NIST 7741]
:*a14: Implantable device list
 
= Criteria that tested =
:*a1: Computerized provider order entry (CPOE) – medications (2022 task 2) (completed in 2022 and hidden below)
:*a2: CPOE – laboratory (2022 task 4) (completed in 2022 and hidden below)
:*a5: Demographics (2022 task 1) (completed in 2022 and hidden below)
:*a9: Clinical decision support (2022 task 1) (completed in 2022 and hidden below)
:*a14: Implantable device list (2022 task 3) (completed in 2022 and hidden below)
:*'''b11: Decision Support Intervention 2024 addendum (2024 task 1,2,3)'''


= Research =
= Research =
Line 17: Line 25:
:*https://slicompliance.com/wp-content/uploads/2022/01/FreeChiro_18_SED_Report.pdf
:*https://slicompliance.com/wp-content/uploads/2022/01/FreeChiro_18_SED_Report.pdf
:*https://slicompliance.com/wp-content/uploads/2021/12/Strateq_EHR_UsabilityReport_08Dec2021.pdf
:*https://slicompliance.com/wp-content/uploads/2021/12/Strateq_EHR_UsabilityReport_08Dec2021.pdf
<br>


= Usability demos =
= Usability official testing demos =


*There are 3 duplicate demos that contain 1 patient (Ed Smith with one encounter) with one lab option (PSA) and with ICD10 code set installed and one provider (the admin user with name of Frank Johnson). Demos can be reset after the test is complete. Demos also reset daily at around 1:00 am pacific time. When demos reset, they are build on top of most recent codebase. Demos use private credentials (ie. public will not be able to access them).
* These are duplicate official testing demos that contain 1 patient (Ed Smith). Official testing demos completely reset daily at around 1:00 am pacific time; when official testing demos completely reset, they are rebuilt on top of most recent codebase. Official testing demos use private credentials (ie. public will not be able to access them) that are only known to the official testers.
:* [https://edu.openemr.io/a/openemr/index.php Usability Demo 1]
<!-- bug here in the actual first demo without a letter when opening patient summary screen, so will not use it -->
:* [https://edu.openemr.io/b/openemr/index.php Usability Demo 2]
:* [https://eight.openemr.io/a/openemr/index.php Testing Demo A]
:* [https://edu.openemr.io/c/openemr/index.php Usability Demo 3]
:* [https://eight.openemr.io/b/openemr/index.php Testing Demo B]
:* [https://eight.openemr.io/c/openemr/index.php Testing Demo C]
:* [https://eight.openemr.io/d/openemr/index.php Testing Demo D]
:* [https://eight.openemr.io/e/openemr/index.php Testing Demo E]
:* [https://eight.openemr.io/log/logPhp.txt PHP Log]
<br>


= Usability testing instruction sets =
= Usability training demos =
* The point of these training demos is to allow training by multiple users, which pick Ed Smith patient to run through the below training guides on. Training demos completely reset daily at around 1:00 am pacific time; when training demos completely reset, they are rebuilt on top of most recent codebase. Training demos use public credentials, which are username:admin password:Administrator1!
<!-- bug here in the actual first demo without a letter when opening patient summary screen, so will not use it -->
:* [https://seven.openemr.io/a/openemr/index.php Training Demo A]
:* [https://seven.openemr.io/b/openemr/index.php Training Demo B]
:* [https://seven.openemr.io/c/openemr/index.php Training Demo C]
:* [https://seven.openemr.io/d/openemr/index.php Training Demo D]
:* [https://seven.openemr.io/e/openemr/index.php Training Demo E]
:* [https://seven.openemr.io/log/logPhp.txt PHP Log]
<br>
 
= Usability testing scenario =
:* Ed Smith is a patient in the user's clinic. In this scenario, the user will 'update Prostate Cancer Screening rule to explain the rule's usage of patient's biological birth sex', 'provide rule feedback on incorrect application of Pap Smear rule for Ed Smith due to incorrect biological birth sex', and 'update 3rd party decision support intervention source attribute field information'.
 
:* The following is the testing guide that will be used during testing by the users: [[Usability Testing Guide]]
 
:* The task material below is for user training.
<br>
 
<!--


==Task 1: Enter and modify demographics.==
==Task 1: Enter and modify demographics.==
* Items tested:
* Test:
:*a5: Demographics
:* Edit demographics for 'Ed Smith'
:*a9: Clinical decision support
* Testing overview:
:* Open patient 'Ed Smith'
:* Edit demographics
::* Gender Identity choose 'Identifies as Male'
::* Gender Identity choose 'Identifies as Male'
::* Sexual Orientation choose 'Straight or Heterosexual'
::* Sexual Orientation choose 'Straight or Heterosexual'
::* DOB change to '1968-12-30'
::* DOB change to '1968-12-30'
:* Clinical decision support will display window with new due reminders based off age (including Colon Cancer Screening).
:* After saving above demographic changes, clinical decision support will display window with new due reminders based off age (including Colon Cancer Screening).
* Specific testing steps:
* Specific testing steps:
:* '''Under construction (steps with screenshots)'''
::[[File:task1-1.png|600px|none|thumb|Enter 'Smith' in patient selector at top right and then click the search icon/button.]]
::[[File:task1-2.png|600px|none|thumb|Click the 'Smith, Ed' patient.]]
::[[File:task1-3.png|600px|none|thumb|Click the edit icon/button for Demographics.]]
::[[File:task1-4.png|600px|none|thumb|Gender Identity set 'Identifies as Male'. Sexual Orientation set 'Straight or Heterosexual'. DOB change to '1968-12-30'. Then click 'Save'.]]
::[[File:task1-5.png|600px|none|thumb|Click 'OK'.]]
<br>


==Task 2: Enter in medication order.==
==Task 2: Enter in medication order.==
* Item tested:
* Test:
:* a1: Computerized provider order entry (CPOE) – medications
:* Add prescription for 'Ed Smith'
* Testing overview:
::* Lipitor 20mg tabs Per Oris q.d. with quantity of 30
:* Add prescription
::* Lipitor 20mg PO QDAY
* Specific testing steps:
* Specific testing steps:
:* '''Under construction (steps with screenshots)'''
::[[File:task2-1.png|600px|none|thumb|Scroll to bottom of patient summary screen and click the edit icon/button for Prescriptions.]]
::[[File:task2-2.png|600px|none|thumb|Click 'Add']]
::[[File:task2-3.png|600px|none|thumb|Enter 'Lipitor' in the textbox and then select the Lipitor entry.]]
::[[File:task2-4.png|600px|none|thumb|Enter Quantity 30. Enter Medicine Units 20 mg. Enter Directions 1 in tablet Per Oris q.d. Then click 'Save'.]]
::[[File:task2-5.png|600px|none|thumb|Click 'Quit'.]]
<br>


==Task 3: Add a implantable device.==
==Task 3: Add a implantable device.==
* Item tested:
* Test:
:* a14: Implantable device list
:* Add Implantable Device for 'Ed Smith'
* Testing overview:
:* Add Implantable Device
::* UDI is (01)00889095205923(11)141231(17)150707(10)A213B1(21)1234
::* UDI is (01)00889095205923(11)141231(17)150707(10)A213B1(21)1234
* Specific testing steps:
* Specific testing steps:
:* '''Under construction (steps with screenshots)'''
::[[File:task3-1.png|600px|none|thumb|Click 'Issues' at top of patient summary screen.]]
::[[File:task3-2.png|600px|none|thumb|Click 'Add' at Medical Devices.]]
::[[File:task3-3.png|600px|none|thumb|Paste following into UDI textbox: (01)00889095205923(11)141231(17)150707(10)A213B1(21)1234<br>Then click 'Process UDI'.]]
::[[File:task3-4.png|600px|none|thumb|This is what screen looks like after clicking 'Process UDI'.]]
::[[File:task3-5.png|600px|none|thumb|Then scroll down and click 'Save'.]]
<br>


==Task 4: Order a lab.==
==Task 4: Order a lab.==
* Item tested:
* Test:
:* a2: CPOE – laboratory
:* Order lab for 'Ed Smith'
* Testing overview:
:* Order lab
::* PSA
::* PSA
* Specific testing steps:
* Specific testing steps:
:* '''Under construction (steps with screenshots)'''
::[[File:task4-1.png|600px|none|thumb|Click 'Visit Past Encounters' at top of screen.]]
::[[File:task4-2.png|600px|none|thumb|Click '2022-01-18 Office Visit' from dropdown.]]
::[[File:task4-4.png|600px|none|thumb|In encounter menu, click 'Administrative', then click 'Procedure Order'.]]
::[[File:task4-5.png|600px|none|thumb|Click Procedure Test textbox.]]
::[[File:task4-6.png|600px|none|thumb|Enter 'PSA' and click 'Search'.]]
::[[File:task4-7.png|600px|none|thumb|Click 'psa_level_order'.]]
::[[File:task4-8.png|600px|none|thumb|Click 'Save'.]]
<br>
<br>
-->
==Task 1: View and update Decision Support Intervention source attribute information.==
* Test:
:* Update Prostate Cancer Screening rule to explain the rule's usage of patient's biological birth sex.
* Scenario:
:* The OpenEMR internal CDR is missing source attribute information on how it's Prostate Cancer Screening rule is using the patient's biological birth sex information in its rule calculations. I need to update the source attribute information for this rule to explain that the sex data point is used to determine if the patient is male and should be screened for prostate cancer. This will help clinicians make informed decisions as to the efficacy and safety of the rule.
* Specific Testing steps:
::[[File:task1-1.png|600px|none|thumb|Enter 'Smith' in patient selector at top right and then click the search icon/button.]]
::[[File:task1-2.png|600px|none|thumb|Click the 'Smith, Ed' patient.]]
::[[File:Task5_Step3_InfoIcon.png|600px|none|thumb|Under Clinical Reminders Click info icon for 'Assessment: Prostate Cancer Screening']]
::[[File:Task5_Step4_FindPatientAttribute.png|600px|none|thumb|Find Patient's Sex source attribute information]]
::[[File:Task5_Step5_ClickEditButton.png|600px|none|thumb|Click the "Edit Rule Source Attributes" button]]
::[[File:Task5_Step6_EditAttribute.png|600px|none|thumb|Find the Patient's Sex source attribute to edit]]
::[[File:Task5_Step7_EnterAttribute.png|600px|none|thumb|Enter in the 'Rule Usage of Patient's Sex' the message 'This rule uses birth sex to determine if the patient should be screened for prostate cancer']]
::[[File:Task5_Step8_SaveAttribute.png|600px|none|thumb|Click the save button to save the rule]]
::[[File:Task5_Step9_SaveInterval.png|600px|none|thumb|Click the save button to save the rule schedule]]
==Task 2: Provide feedback on incorrect Decision Support Intervention rule usage.==
* Test:
:* Provide rule feedback on incorrect application of Pap Smear rule for 'Ed Smith' due to incorrect biological sex.
* Scenario:
:* 'Ed Smith' in the Clinical Decisions section of his patient dashboard is showing that he needs a pap smear exam.  However, Ed's biological birth sex is male and he has no cervix.  As a clinician I need to provide feedback to the CDR designer of this rule that it is improperly being applied to Ed and the rule needs to be fixed.
* Specific Testing steps:
::[[File:task1-1.png|600px|none|thumb|Enter 'Smith' in patient selector at top right and then click the search icon/button.]]
::[[File:task1-2.png|600px|none|thumb|Click the 'Smith, Ed' patient.]]
::[[File:Task6_Step3.png|600px|none|thumb|Click info icon for 'Examination: Pap Smear']]
::[[File:Task6_Step4.png|600px|none|thumb|Scroll down to Feedback]]
::[[File:Task6_Step5.png|600px|none|thumb|Enter 'Rule should not have flagged Ed Smith for pap smear as Ed's birth sex is male']]
::[[File:Task6_Step6.png|600px|none|thumb|Click the 'Submit Feedback' button]]
::[[File:Task6_Step7.png|600px|none|thumb|Confirm 'Feedback submitted successfully' message]]
==Task 3: View and update 3rd party Decision Support Intervention source attribute information.==
* Test:
:* Update 3rd party decision support intervention source attribute field information.
* Scenario:
:* The Pain Management Summary DSI engine is missing source attribute information on what kinds of patients the 3rd Party DSI will target in its AI engine. In order to provide clinicians safety information on the target populations of this DSI, this attribute needs to be updated.
* Specific Testing steps:
::[[File:task1-1.png|600px|none|thumb|Enter 'Smith' in patient selector at top right and then click the search icon/button.]]
::[[File:task1-2.png|600px|none|thumb|Click the 'Smith, Ed' patient.]]
::[[File:Task7_Step1.png|600px|none|thumb|Find and expand the SMART Enabled Apps on the patient's Medical Record Dashboard]]
::[[File:Task7_Step2.png|600px|none|thumb|Click on the DSI Source Information to view the source attributes of the application]]
::[[File:Task7_Step3.png|600px|none|thumb|Find the source attribute for "Intended patient population(s) for the intervention`s use"]]
::[[File:Task7_Step4.png|600px|none|thumb|Find and click the 'Edit' button]]
::[[File:Task7_Step5.png|600px|none|thumb|Find the source attribute "Intended patient population(s) for the intervention`s use"]]
::[[File:Task7_Step6.png|600px|none|thumb|Update the source attribute to be "All patients who have had some form of chronic pain condition or diagnosis"]]
::[[File:Task7_Step7.png|600px|none|thumb|Find and click the 'Save' button]]
::[[File:Task7_Step8.png|600px|none|thumb|Verify you see the 'Save successful' message]]

Latest revision as of 19:10, 30 October 2024

Overview

The first usability testing was completed by the 2022 Cohort of the Professional Certificate in Health Information Technology at Columbia University for the a1, a2, a5, a9, a14 certification criteria.
An 2024 addendum to the usability study is being conducted by OpenEMR for the b11 certification criteria.

Usability Test Report

OpenEMR Usability Test Report - 2022

UCD Process

NIST 7741

Criteria that tested

  • a1: Computerized provider order entry (CPOE) – medications (2022 task 2) (completed in 2022 and hidden below)
  • a2: CPOE – laboratory (2022 task 4) (completed in 2022 and hidden below)
  • a5: Demographics (2022 task 1) (completed in 2022 and hidden below)
  • a9: Clinical decision support (2022 task 1) (completed in 2022 and hidden below)
  • a14: Implantable device list (2022 task 3) (completed in 2022 and hidden below)
  • b11: Decision Support Intervention 2024 addendum (2024 task 1,2,3)

Research

Actual rule:
Example reports:


Usability official testing demos

  • These are duplicate official testing demos that contain 1 patient (Ed Smith). Official testing demos completely reset daily at around 1:00 am pacific time; when official testing demos completely reset, they are rebuilt on top of most recent codebase. Official testing demos use private credentials (ie. public will not be able to access them) that are only known to the official testers.


Usability training demos

  • The point of these training demos is to allow training by multiple users, which pick Ed Smith patient to run through the below training guides on. Training demos completely reset daily at around 1:00 am pacific time; when training demos completely reset, they are rebuilt on top of most recent codebase. Training demos use public credentials, which are username:admin password:Administrator1!


Usability testing scenario

  • Ed Smith is a patient in the user's clinic. In this scenario, the user will 'update Prostate Cancer Screening rule to explain the rule's usage of patient's biological birth sex', 'provide rule feedback on incorrect application of Pap Smear rule for Ed Smith due to incorrect biological birth sex', and 'update 3rd party decision support intervention source attribute field information'.
  • The task material below is for user training.



Task 1: View and update Decision Support Intervention source attribute information.

  • Test:
  • Update Prostate Cancer Screening rule to explain the rule's usage of patient's biological birth sex.
  • Scenario:
  • The OpenEMR internal CDR is missing source attribute information on how it's Prostate Cancer Screening rule is using the patient's biological birth sex information in its rule calculations. I need to update the source attribute information for this rule to explain that the sex data point is used to determine if the patient is male and should be screened for prostate cancer. This will help clinicians make informed decisions as to the efficacy and safety of the rule.
  • Specific Testing steps:
Enter 'Smith' in patient selector at top right and then click the search icon/button.
Click the 'Smith, Ed' patient.
Under Clinical Reminders Click info icon for 'Assessment: Prostate Cancer Screening'
Find Patient's Sex source attribute information
Click the "Edit Rule Source Attributes" button
Find the Patient's Sex source attribute to edit
Enter in the 'Rule Usage of Patient's Sex' the message 'This rule uses birth sex to determine if the patient should be screened for prostate cancer'
Click the save button to save the rule
Click the save button to save the rule schedule

Task 2: Provide feedback on incorrect Decision Support Intervention rule usage.

  • Test:
  • Provide rule feedback on incorrect application of Pap Smear rule for 'Ed Smith' due to incorrect biological sex.
  • Scenario:
  • 'Ed Smith' in the Clinical Decisions section of his patient dashboard is showing that he needs a pap smear exam. However, Ed's biological birth sex is male and he has no cervix. As a clinician I need to provide feedback to the CDR designer of this rule that it is improperly being applied to Ed and the rule needs to be fixed.
  • Specific Testing steps:
Enter 'Smith' in patient selector at top right and then click the search icon/button.
Click the 'Smith, Ed' patient.
Click info icon for 'Examination: Pap Smear'
Scroll down to Feedback
Enter 'Rule should not have flagged Ed Smith for pap smear as Ed's birth sex is male'
Click the 'Submit Feedback' button
Confirm 'Feedback submitted successfully' message

Task 3: View and update 3rd party Decision Support Intervention source attribute information.

  • Test:
  • Update 3rd party decision support intervention source attribute field information.
  • Scenario:
  • The Pain Management Summary DSI engine is missing source attribute information on what kinds of patients the 3rd Party DSI will target in its AI engine. In order to provide clinicians safety information on the target populations of this DSI, this attribute needs to be updated.
  • Specific Testing steps:
Enter 'Smith' in patient selector at top right and then click the search icon/button.
Click the 'Smith, Ed' patient.
Find and expand the SMART Enabled Apps on the patient's Medical Record Dashboard
Click on the DSI Source Information to view the source attributes of the application
Find the source attribute for "Intended patient population(s) for the intervention`s use"
Find and click the 'Edit' button
Find the source attribute "Intended patient population(s) for the intervention`s use"
Update the source attribute to be "All patients who have had some form of chronic pain condition or diagnosis"
Find and click the 'Save' button
Verify you see the 'Save successful' message
Retrieved from "https://www.open-emr.org/wiki/index.php?title=Safety-enhanced_design_(MU3))&oldid=36038"