M&E

Foreword

The management of IHRA has determined to include rigorous monitoring and evaluation initiatives in its organizational scope of work starting from conceptualization stage to high quality monitoring reports for informed decision making process, and to strengthen organizational standard operating procedures/systems by promoting enabling environment to generate, capture, track, secure and reutilize its information base. The major objective of this M&E strategy is to connect the Authority’s officials with key performance indicators around departmental targets, quality standards, timeliness, transparency and accountability. These indicators will be tracked regularly and feedback loops will be established for timely decisions/corrective measures.

Monitoring & Evaluation at a glance

As a standard definition of M&E: Monitoring is the systematic and routine collection of information from organizational initiatives for four main purposes:

  1. To learn from experiences to improve practices and activities in the future;
  2. To have internal and external accountability of the resources used and the results obtained;
  3. To take informed decisions on the future of the initiative;
  4. To promote empowerment of beneficiaries of the initiative.

Monitoring is a periodically recurring task already beginning in the planning stage of a project or programme. Monitoring allows results, processes and experiences to be documented and used as a basis to steer decision-making and learning processes. Monitoring is checking progress against plans. The data acquired through monitoring is used for evaluation.

Evaluation is assessing, as systematically and objectively as possible, a completed project or programme (or a phase of an ongoing project or programme that has been completed). Evaluations appraise data and information that inform strategic decisions, thus improving the project or programme in the future.
Evaluations should help to draw conclusions about five main aspects of the intervention:

  1. Relevance
  2. Effectiveness
  3. Efficiency
  4. Impact
  5. Sustainability

Information gathered in relation to these aspects during the monitoring process provides the basis for the evaluative analysis.

Monitoring & Evaluation at IHRA

To achieve tangible target of provision of quality healthcare services to residents of Islamabad Capital Territory by engaging both public and private Health Care Establishments to ensure that all HCEs are duly registered with the Authority and maintaining minimum service delivery standards. M&E team proposes following phase based key performance indicators to record and track progress of the Authority;

 

Key Performance Indicators Phase-I

  • % of HCEs of ICT registered with IHRA
  • % of HCEs inspected by IHRA
  • % of non-registered HCEs followed up
  • # of quacks identified and sealed/corrective measures taken
  • % of activities monitored
  • % of complaints reduced to specific number

Key Performance Indicators Phase-II

  • % of licensed HCEs followed IHRA financial guidelines for pricing
  • % of licensed HCEs started sharing of patient information on IHRA portal
  • % of licensed HCEs obtained standard bench mark (>=75 score) during regular inspection process
  • # of quacks stopped malpractice – reduced to minimum number
  • % of IHRA’s operations transformed to online portal
  • # of learning and innovation themes adopted as best practices

Our Approach

On completion of targets setting and annual work plan of IHRA, M&E team adopted collaborative approach to monitor the implementation process of activities. The team adopted monitoring techniques of result based monitoring and process monitoring by regularly liaising with all the departments to acquire updated plan(s) and will adjust the frequency of monitoring activities accordingly. For result based monitoring organizational targets at monthly and annually basis level have been assessed against performance and in process monitoring SOPs defined in IHRA ACT 2018 along with regulations will be monitored to assess the implementation of set standards. For each department separate set of monitoring tools have been prepared. Each monitoring and evaluation tool will be pilot tested before initiation of monitoring activities. The idea to use mobile application for monitoring team is also envisioned in future plans of IHRA.

Monitoring Tools
S #Department% of monitoringSampling techniqueMonitoring Checklist / scorecard
1Registration process25%Systematic samplingTool – 1
2Licensing Process25%Stratified samplingTool – 2
3Training of IHRA Staff on MSDS100%N/ATool – 3
4Training of HCE staff on MSDS25%Systematic samplingTool-3
5Initial assessment of HCEs after provisional Licensing25%Systematic samplingTool-4
6General inspections25%Systematic samplingTool-5
7Clinical Inspections5%Systematic samplingTool-6
8Hearings5%Systematic samplingTool-7
9Quacks follow up50%Random samplingTool – 8
10Surprise visits to follow up recommended action by inspection team50%Systematic samplingTool-9
M&E Framework
S #Outcome IndicatorPlanned InterventionsOutput IndicatorBaselineTargetQTR 1QTR 2QTR 3QTR 4Means of verificationData SourceFrequency of data collectionResponsibility
1% of HCEs of ICT registered with IHRA - Provision of quality healthcare to residents of ICT, ensuring compliance of MSDSListing of existing HCEs of ICT# of HCEs registered04500150150150List of all HCEs in ICT, List of registered HCEs, Assessment report, Copy of license issuedDatabase of IHRAMonthlyDirector R&L
Issuance of provisional license
Initial assessments of HCEs
Assessment reports preparation and submission
Confirm licensing to qualified HCEs
2% of HCEs inspected by IHRA - Increase responsiveness by registered HCEs for compliancefollow up inspection visits to licensed HCEs# of HCEs followed up for compliance04500150150150Inspection reportsApproved Reports by Director R&LMonthlyDirector I,M&E
Surprise visits for compliance of MSDS
Legal notices on non-compliance of MSDS
3% of non-registered HCEs followed up - Reduce the health hazards by bringing unregistered HCEs on surveillance radarLegal notices on noncompliance of registration# of un-registered HCEs followed up02700909090Copy of Notices, Visit reportsDirector R&LMonthlyDirector Legal
Follow up visits to un-registered HCEs01800606060Director I,M&E
4# of quacks identified and sealed - Reduce risk factor related to healthcare for residents of ICT by malpractice of quacksListing of quacks# of quacks stopped from practice0900303030Pictures and locationDirector I,M&EMonthlyDirector I,M&E
Follow up visits of sealing, de-sealing
update list of existing quacks
5 % of activities monitored - to ensure compliance of quality, timeline, transparency and accountabilityMonitoring of assessment visits# of interventions monitored0900303030M&E reports, PicturesDirector I,M&EMonthlyDirector I,M&E
Monitoring of inspection visits
Monitoring of tribunal hearings
Monitoring of training sessions
6% of complaints reduced to - The voice of residents of ICT regarding Healthcare issues heard timely with rapid corrective actionRegistration of complaints# of complaints addressed0500151520Complaint Register, databaseDirector LegalMonthlyDirector Legal
Inquiry of complaints
Tribunal hearings
Complaints resolved
Record of all complaints with complete details
Departmental Indicative
S#DepartmentIndicatorsResponsibilityTimeline
1Human Resource / Admin / Operations1.   # of internal complaints/conflicts resolvedDirector HRMonthly Update
2.  Pay roll processed on defined date
3.  Leave calendar updated
4. Vehicle report prepared with fuel average
5. Inventory list updated
6. Procurement plan implemented
7.  % of indicators implemented
8.   % of staff retained
2Finance1.   Burn rate preparedDirector FinanceMonthly Update
2. Budget forecast
3. Budget versus actual
3Clinical Governance and Trainings1. # of MSDS approved by the technical committeeDirector Clinical - GovernMonthly Update
2. # of MSDS approved by the Board
3. # of training manual for each MSDS topic prepared
4. # of translation of training manuals completed
5. # of training session conducted for staff of IHRA
6. # of training sessions conducted for HCEs with # of participant (male and female segregation)
7. # of clinical audit visits conducted (SBMR/SS)
8. % of IHRA staff trained on MSDS
9. % of HCEs staff trained by IHRA on MSDS
10. % of HCEs staff trained by others on MSDS
11. % of HCEs staff training on MSDS
4Registration and Licensing1. Updated list of HCEs of ICT preparedDirector Reg & LicensingMonthly Update
2. % of HCEs applied for Registration
3. % of HCEs acquired provisional License
4. % of HCEs acquired confirm License
5. % of registration cases completed as per 2018 act - Timelines
6. # of Registration and Licensing committee meetings planned
7. # of cases presented for provisional licensing
8. # of cases presented for confirmed licensing
9. # of cases deferred by Registration and Licensing committee
10. # of deferred cases followed up for advise to resolve the objection(s)
5Inspection/monitoring & Evaluation1. % of registration files scrutinizedDirector Inspection and M&EMonthly Update
2. # of initial assessments conducted
3. % of initial assessments monitored
4. # of inspection visits conducted
5. % of inspection visits monitored
6. # of quacks identified
7. % of followed up for monitoring purposes
8.% of Health Tribunal Hearing attended for monitoring purposes
9. # of surprise visits conducted to HCEs for compliance of MSDS
10. # of evaluations conducted
11. # of surprise visits to sealed quacks
11. # of visits to quacks who declared registered HCE after corrective measures.
12. % of complaints followed up
6Legal1. # of complaints receivedDirector LegalMonthly Update
2. # of complaints registered for follow up
3. # of complaints resolved
4. # of cases consulted with Registration and Licensing Department
5. # of hearing tribunal conducted
6. # of court cases handled
7Media and Coordination1. # of coordination committee meetings plannedDirector CoordinationMonthly Update
2. # of coordination committee meetings conducted
3. # of public notices published
4. # of newsletter published
5. Website information updated
6. Annual report prepared and published
7. # of social media posts published