Foreword
The management of IHRA has determined to include rigorous monitoring and evaluation initiatives in its organizational scope of work starting from conceptualization stage to high quality monitoring reports for informed decision making process, and to strengthen organizational standard operating procedures/systems by promoting enabling environment to generate, capture, track, secure and reutilize its information base. The major objective of this M&E strategy is to connect the Authority’s officials with key performance indicators around departmental targets, quality standards, timeliness, transparency and accountability. These indicators will be tracked regularly and feedback loops will be established for timely decisions/corrective measures.
Monitoring & Evaluation at a glance
As a standard definition of M&E: Monitoring is the systematic and routine collection of information from organizational initiatives for four main purposes:
- To learn from experiences to improve practices and activities in the future;
- To have internal and external accountability of the resources used and the results obtained;
- To take informed decisions on the future of the initiative;
- To promote empowerment of beneficiaries of the initiative.
Monitoring is a periodically recurring task already beginning in the planning stage of a project or programme. Monitoring allows results, processes and experiences to be documented and used as a basis to steer decision-making and learning processes. Monitoring is checking progress against plans. The data acquired through monitoring is used for evaluation.
Evaluation is assessing, as systematically and objectively as possible, a completed project or programme (or a phase of an ongoing project or programme that has been completed). Evaluations appraise data and information that inform strategic decisions, thus improving the project or programme in the future.
Evaluations should help to draw conclusions about five main aspects of the intervention:
- Relevance
- Effectiveness
- Efficiency
- Impact
- Sustainability
Information gathered in relation to these aspects during the monitoring process provides the basis for the evaluative analysis.
Monitoring & Evaluation at IHRA
To achieve tangible target of provision of quality healthcare services to residents of Islamabad Capital Territory by engaging both public and private Health Care Establishments to ensure that all HCEs are duly registered with the Authority and maintaining minimum service delivery standards. M&E team proposes following phase based key performance indicators to record and track progress of the Authority;
Key Performance Indicators Phase-I
- % of HCEs of ICT registered with IHRA
- % of HCEs inspected by IHRA
- % of non-registered HCEs followed up
- # of quacks identified and sealed/corrective measures taken
- % of activities monitored
- % of complaints reduced to specific number
Key Performance Indicators Phase-II
- % of licensed HCEs followed IHRA financial guidelines for pricing
- % of licensed HCEs started sharing of patient information on IHRA portal
- % of licensed HCEs obtained standard bench mark (>=75 score) during regular inspection process
- # of quacks stopped malpractice – reduced to minimum number
- % of IHRA’s operations transformed to online portal
- # of learning and innovation themes adopted as best practices
Our Approach
On completion of targets setting and annual work plan of IHRA, M&E team adopted collaborative approach to monitor the implementation process of activities. The team adopted monitoring techniques of result based monitoring and process monitoring by regularly liaising with all the departments to acquire updated plan(s) and will adjust the frequency of monitoring activities accordingly. For result based monitoring organizational targets at monthly and annually basis level have been assessed against performance and in process monitoring SOPs defined in IHRA ACT 2018 along with regulations will be monitored to assess the implementation of set standards. For each department separate set of monitoring tools have been prepared. Each monitoring and evaluation tool will be pilot tested before initiation of monitoring activities. The idea to use mobile application for monitoring team is also envisioned in future plans of IHRA.
S # | Department | % of monitoring | Sampling technique | Monitoring Checklist / scorecard |
---|---|---|---|---|
1 | Registration process | 25% | Systematic sampling | Tool 1 |
2 | Licensing Process | 25% | Stratified sampling | Tool 2 |
3 | Training of IHRA Staff on MSDS | 100% | N/A | Tool 3 |
4 | Training of HCE staff on MSDS | 25% | Systematic sampling | Tool-3 |
5 | Initial assessment of HCEs after provisional Licensing | 25% | Systematic sampling | Tool-4 |
6 | General inspections | 25% | Systematic sampling | Tool-5 |
7 | Clinical Inspections | 5% | Systematic sampling | Tool-6 |
8 | Hearings | 5% | Systematic sampling | Tool-7 |
9 | Quacks follow up | 50% | Random sampling | Tool 8 |
10 | Surprise visits to follow up recommended action by inspection team | 50% | Systematic sampling | Tool-9 |
S # | Outcome Indicator | Planned Interventions | Output Indicator | Baseline | Target | QTR 1 | QTR 2 | QTR 3 | QTR 4 | Means of verification | Data Source | Frequency of data collection | Responsibility |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|
1 | % of HCEs of ICT registered with IHRA - Provision of quality healthcare to residents of ICT, ensuring compliance of MSDS | Listing of existing HCEs of ICT | # of HCEs registered | 0 | 450 | 0 | 150 | 150 | 150 | List of all HCEs in ICT, List of registered HCEs, Assessment report, Copy of license issued | Database of IHRA | Monthly | Director R&L |
Issuance of provisional license | |||||||||||||
Initial assessments of HCEs | |||||||||||||
Assessment reports preparation and submission | |||||||||||||
Confirm licensing to qualified HCEs | |||||||||||||
2 | % of HCEs inspected by IHRA - Increase responsiveness by registered HCEs for compliance | follow up inspection visits to licensed HCEs | # of HCEs followed up for compliance | 0 | 450 | 0 | 150 | 150 | 150 | Inspection reports | Approved Reports by Director R&L | Monthly | Director I,M&E |
Surprise visits for compliance of MSDS | |||||||||||||
Legal notices on non-compliance of MSDS | |||||||||||||
3 | % of non-registered HCEs followed up - Reduce the health hazards by bringing unregistered HCEs on surveillance radar | Legal notices on noncompliance of registration | # of un-registered HCEs followed up | 0 | 270 | 0 | 90 | 90 | 90 | Copy of Notices, Visit reports | Director R&L | Monthly | Director Legal |
Follow up visits to un-registered HCEs | 0 | 180 | 0 | 60 | 60 | 60 | Director I,M&E | ||||||
4 | # of quacks identified and sealed - Reduce risk factor related to healthcare for residents of ICT by malpractice of quacks | Listing of quacks | # of quacks stopped from practice | 0 | 90 | 0 | 30 | 30 | 30 | Pictures and location | Director I,M&E | Monthly | Director I,M&E |
Follow up visits of sealing, de-sealing | |||||||||||||
update list of existing quacks | |||||||||||||
5 | % of activities monitored - to ensure compliance of quality, timeline, transparency and accountability | Monitoring of assessment visits | # of interventions monitored | 0 | 90 | 0 | 30 | 30 | 30 | M&E reports, Pictures | Director I,M&E | Monthly | Director I,M&E |
Monitoring of inspection visits | |||||||||||||
Monitoring of tribunal hearings | |||||||||||||
Monitoring of training sessions | |||||||||||||
6 | % of complaints reduced to | Registration of complaints | # of complaints addressed | 0 | 50 | 0 | 15 | 15 | 20 | Complaint Register, database | Director Legal | Monthly | Director Legal |
Inquiry of complaints | |||||||||||||
Tribunal hearings | |||||||||||||
Complaints resolved | |||||||||||||
Record of all complaints with complete details |
S# | Department | Indicators | Responsibility | Timeline |
---|---|---|---|---|
1 | Human Resource / Admin / Operations | 1. # of internal complaints/conflicts resolved | Director HR | Monthly Update |
2. Pay roll processed on defined date | ||||
3. Leave calendar updated | ||||
4. Vehicle report prepared with fuel average | ||||
5. Inventory list updated | ||||
6. Procurement plan implemented | ||||
7. % of indicators implemented | ||||
8. % of staff retained | ||||
2 | Finance | 1. Burn rate prepared | Director Finance | Monthly Update |
2. Budget forecast | ||||
3. Budget versus actual | ||||
3 | Clinical Governance and Trainings | 1. # of MSDS approved by the technical committee | Director Clinical - Govern | Monthly Update |
2. # of MSDS approved by the Board | ||||
3. # of training manual for each MSDS topic prepared | ||||
4. # of translation of training manuals completed | ||||
5. # of training session conducted for staff of IHRA | ||||
6. # of training sessions conducted for HCEs with # of participant (male and female segregation) | ||||
7. # of clinical audit visits conducted (SBMR/SS) | ||||
8. % of IHRA staff trained on MSDS | ||||
9. % of HCEs staff trained by IHRA on MSDS | ||||
10. % of HCEs staff trained by others on MSDS | ||||
11. % of HCEs staff training on MSDS | ||||
4 | Registration and Licensing | 1. Updated list of HCEs of ICT prepared | Director Reg & Licensing | Monthly Update |
2. % of HCEs applied for Registration | ||||
3. % of HCEs acquired provisional License | ||||
4. % of HCEs acquired confirm License | ||||
5. % of registration cases completed as per 2018 act - Timelines | ||||
6. # of Registration and Licensing committee meetings planned | ||||
7. # of cases presented for provisional licensing | ||||
8. # of cases presented for confirmed licensing | ||||
9. # of cases deferred by Registration and Licensing committee | ||||
10. # of deferred cases followed up for advise to resolve the objection(s) | ||||
5 | Inspection/monitoring & Evaluation | 1. % of registration files scrutinized | Director Inspection and M&E | Monthly Update |
2. # of initial assessments conducted | ||||
3. % of initial assessments monitored | ||||
4. # of inspection visits conducted | ||||
5. % of inspection visits monitored | ||||
6. # of quacks identified | ||||
7. % of followed up for monitoring purposes | ||||
8.% of Health Tribunal Hearing attended for monitoring purposes | ||||
9. # of surprise visits conducted to HCEs for compliance of MSDS | ||||
10. # of evaluations conducted | ||||
11. # of surprise visits to sealed quacks | ||||
11. # of visits to quacks who declared registered HCE after corrective measures. | ||||
12. % of complaints followed up | ||||
6 | Legal | 1. # of complaints received | Director Legal | Monthly Update |
2. # of complaints registered for follow up | ||||
3. # of complaints resolved | ||||
4. # of cases consulted with Registration and Licensing Department | ||||
5. # of hearing tribunal conducted | ||||
6. # of court cases handled | ||||
7 | Media and Coordination | 1. # of coordination committee meetings planned | Director Coordination | Monthly Update |
2. # of coordination committee meetings conducted | ||||
3. # of public notices published | ||||
4. # of newsletter published | ||||
5. Website information updated | ||||
6. Annual report prepared and published | ||||
7. # of social media posts published |